Evaluation

Purpose
The purpose of this evaluation was to examine the development and implementation of Civic Engagement Programs (CEPs) on five college and university campuses. This project sought to gather in-depth information about the ways in which the five pilot schools have developed their individual Civic Engagement Programs, the challenges that they have encountered, and the intended outcomes of these programs.

Methodology
Data for this evaluation were collected through the following methods:

• The evaluation team conducted site visits to each of the five participating campuses. The schedule for each of these visits was developed in collaboration with the key program administrator(s) on each campus. During these visits the team met with key program administrators, faculty members, students, and community partners.
  
• The evaluation team met with the key program administrator(s) for an in-depth interview to discuss the program background and development to date. Key administrators provided the research team with a detailed account of the Civic Engagement Program (CEPs) on their campuses, including specific course requirements, community placement expectations, and any supplementary information about the programs.
  
• Site visits also included focus groups with faculty members involved in activities related to civic engagement, such as service-learning, community-based research, and experiential education.
  
• The key administrator(s) at each institution recruited students for focus groups at each campus. When possible, focus groups were held with students that are participating in civic engagement related activities on their campuses as well as and those that are not. Each student focus group participant was asked to complete a survey on civic engagement, focusing on current student activities as well as student perceptions of the meaning of “civic engagement.”
  
• Additionally, the team held interviews with relevant community partners on each campus to gather information about existing partnerships between the campus and the community and to learn about community partners’ perspectives on the developing CEP.
  
• When possible and relevant, the evaluation team interviewed school administrators to gather data about the relationship between the CEP and the administration.

Key Findings

• Generally, a student participating in one of these programs would be required to complete a series of courses that would include an introductory course and a capstone and they would complete an intensive civic engagement placement or internship.
  
• On every campus, the development of the CEP has been intentionally interdisciplinary, with an explicit desire to include students and faculty from a diverse set of disciplines.
  
• Each institution appears to be at a different point in the process of developing these programs, with some schools ready to begin their programs in the fall 2004 semester. However, other schools indicated their programs may not begin until the spring or even fall of 2005.
  
• For the most part, the courses and placement programs the schools intend to use for the CEPs already exist. The schools already have relationships with the community partners they intend to use for the programs. The CEPS will provide value-added in that they will create formal connections between coursework in different academic years and civic engagement placements in which students may already be participants.
  
• To date, student involvement in the development of the CEPs at the pilot schools has been minimal; however, program staff indicated that they do intend to seek student input later in the development process.
  
• In most cases, development of the CEPs has been influenced by the involvement of a small group of committed faculty members representing a variety of academic disciplines. These faculty members already work to integrate civic engagement experiences into the classroom, and in most cases are already communicating with one another through informal campus networks.
  
• Aspects commonly mentioned by the schools that facilitated the development of the CEPs thus far included a supportive administration and community, the location of the program in an academic department, dedicated and qualified program staff, and existing successful partnerships with community organizations.
  
• Generally, students and faculty reported that civic engagement opportunities existed in “pockets” around their campuses without much central coordination that could raise its visibility and generate greater awareness.
  
• Although the individual programs at the five schools will have specific impact goals, a few common themes emerged during the evaluation. In a general sense, the developing CEPs are intended to foster leadership, critical thinking skills, and awareness about civic issues among students.

Key Limitations of the Analysis
While a more complete list of the limitations to this analysis is included in the body of the report, the most crucial limitations are:

• Small Sample Size: Because the evaluation included only five institutions, it is not possible to generalize the findings from these campuses to other colleges and universities.
  
• Selection: Because the interviewees and focus group participants were selected by the key administrator on each campus, as opposed to by the evaluation team, it is possible that different campuses selected participants differently thereby affecting the responses that were received.
  
• Self-reporting: Since all data used in this analysis were self-reported, there are potential limitations in terms of inconsistent operationalization of concepts, intentional and unintentional misrepresentation, and tendencies towards reporting socially desirable answers.

Next Steps
Prior to this evaluation, it had been anticipated that the evaluation team would administer pre-surveys on civic engagement to students who would enroll in the CEPs at their institutions in summer/fall 2004. A key finding in this evaluation is that the five campuses are at vastly different stages of CEP development and implementation and therefore may not all be ready to begin surveying for student impacts by fall 2004. It is therefore the recommendation of the evaluation team that each campus carefully assess its own institutional readiness for participation in impact studies before surveys are administered.

• For campuses that do implement their CEP in fall 2004, pre-surveys will be administered to participating students as planned.
  
• All five campuses will be visited by the GWU evaluation team again in spring 2005. The goal of this follow-up site visit will be to assess any intermediary changes in program development for the CEP between the second and third years of the FIPSE grant.
  
• All five of the FIPSE pilot campuses will complete follow-up institutional matrices in spring 2005 to assist the researchers in determining changes in the numbers of students, faculty members, and courses currently connected to the CEP .
  
• Washington and Lee has had a formal program for a longer time than the other campuses participating in this pilot program. Because of their degree of implementation, a second group of evaluation students could assist them in measuring reported outcomes of individuals who completed the program in past years.