Evaluation/Planning Research Design for Year 2001,
based on conversations with individual centers, the January meeting, and feedback from participants:

This is a revised draft.  Please send your comments to randy.stoecker@utoledo.edu .  The purpose of this draft is to develop the overall research design more than the details of what methods to use.

There are three sections to this plan:

Principles of Participatory/Empowerment Evaluation

Draft Research Goals and Methods [methods are in brackets]

Next Steps

PRINCIPLES OF PARTICIPATORY/EMPOWERMENT EVALUATION:

1.  Any information about individuals belongs first to the individuals and is subject to revision by individuals before anyone else sees it.
2.  Any information about a group belongs first to the group members and is subject to revision by group members before anyone else sees it.
3.  All information compiled by the evaluation coordinator, Randy Stoecker, belongs to the project participants and nothing will be published without their consent and invitation to participate in the authoring process.
4.  The evaluation coordinator, Randy Stoecker, will not give any written judgements as to the quality or worth of anyone's work.  Rather, the purpose of the research is to provide information that project participants can use in evaluating their own work and making self-determined course corrections on an ongoing basis.
5.  Portions of the evaluation research will be used for reporting purposes to the funder, but not until participants have had the opportunity to review and revise draft reports.

DRAFT RESEARCH GOALS AND METHODS

People are interested in having research at three levels of the overall project:

• Overall network development (across the 7 sites)
• site network development (within each site)
• individual project CBR project development

 

1.  Overall network development

a.  strategy development

1. how were network participants identified and recruited to join in this project? [interviews with Bonner officials and staff, and main site contacts]
2. how were goals set initially for the network? [interviews; document analysis]
3. what strategies were put into place to meet overall network goals? [interviews; document analysis]

b.  strategy implementation

1. documentation of semi-annual meetings [field research; attendee survey]
2. documentatin of cross-site contacts [e-mail,  phone call, face to face tabulations; interviews]
3. documentation of referrals within or outside network [e-mail,  phone call, face to face tabulations; interviews]

c.  outcomes

1. to what extent did people develop new relationships and why? [interviews or survey]
2. to what extent did initial contacts sustain and why? [interviews or survey]
3. to what extent did participants make contacts outside of the network and why? [interviews or survey]
4. to what extent were other resources leveraged for the overall network and why? [interviews or survey]

2.  Site network development

 a.  initial strategy for building network

1. how are people initially recruited? [interviews]
2. what did people expect would effect levels of participation and how did they try to manage that using meeting frequency, incentives, identification of obstacles and other things? [interviews]
3. how important was identifying and attempting to manage CBO-academic tensions in the strategy? [interviews]

 b.  strategy implementation

1. historical case study research of networks already started in Trenton and Appalachia.
2. documentation of meetings used in network development.
3. documentation of non-meeting contact [e-mail,  phone call, face to face tabulations; interviews]

 c.  outcomes (long term and short term)

1. How do people change their thinking about CBR? [interviews]
2. What is the value-added of a network? [case studies of city-wide or area-wide policy impacts or coalition formation]
3. How do you sustain network? [case studies; interviews]
4. How does the network impact CBO capacity to do their own CBR?
5. What products came of the network?
6. To what extent did the network leverage extra funds to support individual projects or further network development? [case studies; interviews]
7. How much are relationships built that sustain outside of network activities? [e-mail,  phone call, face to face tabulations; interviews]
8. To what extent are other joint projects created outside of official network?
9. How are CBO and academy budget lines effected? [case study; document analysis]
10. How much influence does network have on people doing CBR outside of network? [snowball interviews]

3.  individual project development

a.  initial strategy for developing project

1. how do partners find each other? [interviews]
2. what do faculty, students, CBOs each bring to a project? [interviews or survey]
3. what infrastructure is in place initially to suppport a project? [interviews; document analysis]
4. how are decisions made about project design? [interviews]

b.  strategy implementation

1. what stages do projects go through?  [comparative case study]
2. what points of tension do projects confront? [comparative case study]
3. why do individuals continue or drop out? [comparative case study; survey]

c.  outcomes (long term and short term)

1. To what extent did the project achieve the intended goals.
2. To what extent do the projects leverage extra funds? [case study; document analysis]
3. To what extent are students taking jobs in organizations with which they were involved ? or with similar organizations? [survey; interviews]
4. How much are students' attitudes changing [survey; interviews]
5. How much are students learning [survey; grade comparison; interviews]
6. How much are students remaining involved with CBOs after completion of class or project?
7. How is policy impacted? [case study]
8. How is CBR functioning impacted, including funding, participation, other [case study, interview]
9. What projects developed as an outgrowth of original project [case study]

NEXT STEPS

Here are the research tasks to be completed by July, more or less in time order:

1.  Begin development of student, CBO, faculty outcome surveys.