Evaluation Summary

Quantum Opportunities Program Evaluation Summary

PILOT PROGRAM

In 1989, researchers from Brandeis University, led by Eisenhower Trustee Andrew Hahn, began the evaluation of the first group of Quantum Opportunities Programs (not Eisenhower Foundation sponsored). Their findings clearly demonstrated the benefits of the Quantum model.

Study Methodology

Program designers randomly assigned 50 disadvantaged students in each of the five sites (Philadelphia, PA; Saginaw, MI; Oklahoma City, OK; San Antonio, TX; and Milwaukee, WI.) to either a program or a control group. Researchers compared the progress of the two groups with periodic questionnaires and basic skills tests.  Due to the failure to implement at one program site (Milwaukee), the analysis was completed on four of the original five sites.

Findings

The first group of Quantum sites were fully funded and the model rigorously followed.  Brandeis researchers evaluated the four Quantum sites.  Relative to a control group, Quantum students:

  • graduated from high school more often (63 vs.42 percent)
  • dropped out of school less often (23 vs. 50 percent)
  • went on to postsecondary education more often 42 vs. 16 percent)
  • attended a 4-year college more often (18 vs. 5 percent)
  • attended a 2-year institution more often (19 vs. 9 percent)
  • became teen parents less often (24 vs. 38 percent)
  • more often:
    • took part in a community project in the six months following Quantum (21 vs. 12 percent);
    • were volunteer tutors, counselors or mentors, (28 vs. 8 percent) and
    • gave time to non-profit, charitable, school or community groups (41 vs. 11 percent, only statistically significant at the Philadelphia site)

Contributing Factors

The Brandeis report concluded that the key contributing factors in the success of the program were:

Caring Adults
“If young people are connected with caring adults for sustained periods of time, year-round, positive results do emerge.” Program administrators and staff, as well as teachers and mentors, took an active interest in the welfare of the Quantum students, encouraging them, visiting them, following up and doing everything they could to keep them in the program. “Once in Quantum, always in Quantum” was the unofficial motto, and most program counselors took it to heart.

Sense of Community
The project sites were small, with only 25 students in each. Students were able to bond with each other and with adults in the program, particularly at the Philadelphia site.
“Simply put, when a quantum opportunity was offered, young people from public assistance backgrounds--African American males, females, whites, Asians, others -- took it! They joined the programs and many stayed with the programs or the staff associated with the initiatives, for long periods.”

Multiple Services Encompassing All Aspects of Youths’ Lives
The Quantum program was designed to address the many challenges and obstacles that disadvantaged youth face. Quantum focused on developing basic skills (academic and functional) for future success, strengthening life and social skills to make better choices and operate more effectively with families and peers, broadening horizons through cultural trips and other experiences, and taking pride in the community through active service.

Quality Staff
Results from the most effective project site--Philadelphia--show what can be accomplished with a dedicated, quality staff.  “The differences, for example, between San Antonio and Philadelphia cannot be attributed to the neighborhood setting, the characteristics of participants, or to the program model. What distinguishes these sites is the degree of buy-in from the host organizations and the commitment of staff at all levels.”

Financial Incentives As Part of a Comprehensive Program
While financial incentives were important to some students, and helped with family expenses, it appeared that they were not the decisive factor in Quantum participation. When they are part of a comprehensive, well-developed program, financial incentives can be effective in maintaining student interest in and attendance at program events. However, they do not appear to operate effectively in the absence of a strong program featuring much personal contact with staff.

Financial Resources
The Ford Foundation funded the Quantum program upfront, making it possible to plan for and deliver a host of services over an extended period of time. Both staff and students knew the resources were there to carry through on their commitments.

SECOND GENERATION

Following this initial success, Mathematica Policy Research, Inc. evaluated the demonstration programs, sponsored by the U.S. Department of Labor and the Ford Foundation. The results from this demonstration project were much less impressive, due in large part to the failure of the sites to accurately implement the model that had been shown to work in pilot phase.

YOUTH TODAY REVIEW

In an article entitled The Best Youth Program You Can Not Afford, Youth Today discussed both the problems and the benefits of the Quantum Opportunities model.

EISENHOWER FORUM

In November 2003, the Milton S. Eisenhower Foundation (MSEF) convened a group of evaluators, funders, and directors of Quantum demonstration and replication sites to a two-day forum. The meeting focused on practical questions about how best to implement Quantum and how to skillfully replicate the program on a larger scale. Their insights are telling, not only for providers of the Quantum model, but to anyone involved in youth development. To read a summary report of the 2003 Quantum Forum click here.

EISENHOWER'S FIRST COHORT

The Foundation implemented Quantum in four locations in Fall 2002 – Herndon, VA; Portland, OR; Dover, NH; and Columbia, SC.  These sites were similarly evaluated using control groups.  The outcomes for the first three sites were even better than for the students from the original Ford Foundation program, while the fourth site was an implementation failure.
Relative to the control group, Quantum Opportunities students in the three successful replications:

  1. Graduated from high school more often (seventy-eight percent vs. forty-                     four percent);
  2. Dropped out of school less often (ten percent vs. forty-eight percent); and
  3. Proceeded on to postsecondary education or training more often (seventy-                    eight percent vs. thirty-eight percent).

All of these differences were statistically significant.

In the successful replications, fifty percent of the Associates who dropped out entered Job Corps and were on track to receive a GED and a useable skill.  (We do not know how many of the Control Group drop-outs went on to Job Corps or similar programs.)

The participants in the Herndon, Portland and Dover replications experienced significantly lower arrest rates (eight percent), when compared to Control Group members (twenty-three percent).  In part because of the intervention of Quantum staff, all the Quantum participants arrested for various transgressions were able to avoid trial and conviction – and  therefore avoid a juvenile record.