CEO’s evaluation strategy builds on a strong base of internal monitoring and assessment, and includes an individualized plan for each initiative. Each of these plans utilizes different strategies, which allows CEO to better allocate its finite evaluation resources.  In 2008 CEO commissioned Westat and Metis Associates to conduct early implementation and outcome evaluations (called program reviews) of 16 programs, representing the majority of CEO direct service programs that were operational in fiscal year of 2008.  CEO also conducted two internal program reviews.  Each review documents the program’s implementation and its fidelity to the model.

CEO programs that did not receive a program review are being monitored and evaluated using other mechanisms.  For example, tax records enable CEO to assess the impact of the EITC mailing.  Proven strategies, such as the Nurse Family Partnership (NFP), only require careful program monitoring to ensure that the program is implemented according to the model.

Several CEO programs are being evaluated using the “gold standard” of evaluation: a random assignment study. Because random assignment requires significant time and resources, it is only appropriate for certain programs.  In addition, programs must have a waiting list or the potential to recruit sufficient numbers to serve as the control group, and most CEO programs have not yet generated his excess demand.  Programs utilizing a random assignment evaluation strategy include Opportunity NYC, NYC Justice Corps, and the CUNY Performance Based Scholarship program.  Another rigorous evaluation is underway for Teen ACTION that includes a survey of approximately 3,000 youth and two comparison groups.

A small advisory group of evaluation experts periodically reviews CEO’s evaluation planning and progress. Their input will help CEO to develop additional studies in the coming year.  CEO, working with the external evaluators and City agencies, is also developing an evaluation strategy to measure findings across programs that serve the same target groups.  Such groupings could be based in education, economic security, disconnected youth, and CEO’s role as an agent for organizational change.  This approach should further help to identify effective strategies and programs.

Learn more