CEO’s evaluation strategy builds on a
strong base of internal monitoring and assessment, and includes an
individualized plan for each initiative.
Each of these plans utilizes different strategies, which allows CEO to
better allocate its finite evaluation resources. In 2008 CEO commissioned Westat and
Metis Associates to conduct early implementation and outcome evaluations (called
program reviews) of 16 programs, representing the majority of CEO direct service
programs that were operational in fiscal year of 2008. CEO also conducted two internal program
reviews. Each review documents the
program’s implementation and its fidelity to the model.
CEO programs that did not receive a program review are
being monitored and evaluated using other mechanisms. For example, tax records enable CEO to
assess the impact of the EITC mailing.
Proven strategies, such as the Nurse Family Partnership (NFP), only
require careful program monitoring to ensure that the program is implemented
according to the model.
Several CEO programs are being
evaluated using the “gold standard” of evaluation: a random assignment study.
Because random assignment requires significant time and resources, it is only
appropriate for certain programs. In addition, programs must have a
waiting list or the potential to recruit sufficient numbers to serve as the
control group, and most CEO programs have not yet generated his excess demand.
Programs utilizing a random
assignment evaluation strategy include Opportunity NYC, NYC Justice Corps, and
the CUNY Performance Based Scholarship program. Another rigorous evaluation is underway for Teen ACTION that includes a
survey of approximately 3,000 youth and two comparison groups.
A small advisory group of evaluation experts periodically reviews
CEO’s evaluation planning and progress. Their input will help CEO to develop
additional studies in the coming year. CEO, working with the external evaluators
and City agencies, is also developing an evaluation strategy to measure findings
across programs that serve the same target groups. Such groupings could be based in
education, economic security, disconnected youth, and CEO’s role as an agent for
organizational change. This
approach should further help to identify effective strategies and
programs.
Performance Monitoring
