During the 1990s, voucher advocates formed a number of organizations to provide privately funded vouchers to mostly urban, low-income students. The foundations supporting these groups also funded research in some of the cities, conducted by Paul Peterson and colleagues. The most prominent report to come out of this research to date has been the two-year study of private vouchers in New York City, Washington, D.C., and Dayton, Ohio released in August, 2000.28 Peterson wrote that the African American students using vouchers to attend private schools scored 6.3 percentile points better after two years on combined reading and math tests than the control group students who had applied for vouchers but remained in public schools. They found no statistically significant positive or negative effects for Hispanic or white students using vouchers, although 51% in the New York City study were Hispanic, for example, and 24% in Dayton were white.29
Since students applying for these private vouchers were entered into a lottery (separate in each city) and researchers were able to test the students and gather background data, Peterson argued that these studies are akin to randomized field trials or experimental studies in medical science, and - since they do not suffer from the shortcomings of the Milwaukee and Cleveland studies - the validity of the findings is beyond reproach. Peterson claimed that vouchers could lead to a "a nontrivial closing of the gap" in achievement between African American and majority students.30
The study was immediately criticized by researchers who pointed out a number of flaws. These included the presence of bias in the study potentially resulting from "disappointment effects" among some parents who lost in the voucher lottery, as well as "selection bias" due to the fact that significant numbers of students dropped out of the study in subsequent years. Another problem undermining the study's findings are the unexplained dramatic inconsistencies in achievement gains, where some groups do very well while others do poorly.31 The presence of these issues forced Peterson to use statistical corrections precisely like those he claimed to be avoiding - thus undermining his claim that this research adhered to a 'gold standard' of social science research.32
Unexplained, erratic test results
- One of the study's partner agencies, Mathematica Policy Research, Inc. took the extraordinary step of releasing its own press statement entitled "Voucher claims of success are premature in New York City." Mathematica revealed that, in New York City, all of the test score gains for African American students were concentrated in one of the four grades, with no explanation for this phenomenon.33 Yet the improvement shown by this one subgroup was so marked that it resulted in statistically significant improvement for all grade levels of African American students when the four grades were averaged together. The truth is that most African American voucher students in New York did not do better than their public school peers.
- Similarly erratic results were evident in Dayton, where students were in grades 3 through 9 in the second year of the program: African American students in three of the grades (3rd, 5th & 7th) showed large gains of between 14.5 and 19.7 national percentile points in combined math and reading scores while those in the other four grades (4th, 6th, 8th & 9th) actually declined -for the latter cohort by nearly 15 percentile points.34
No research basis for expanding vouchers
- Despite Peterson's claim that this three-city study indicated that vouchers might lead to the eradication of the achievement gap in this country, it in fact offers no evidence - even if the study's findings were valid - that would support larger scale, publicly funded voucher programs. The authors themselves caution that generalizing beyond this small, three city, privately-funded voucher program to larger-scale programs is problematic: "A much larger [voucher] program could conceivably have quite different program outcomes."35
- One reason for this is the presence of what are known as "peer effects". Much of whatever academic advantage private schools appear to have may likely be due to the fact that private schools can - unlike public schools - select and choose their students, thus having some control over the peer environment in the schools. As a few poor, urban students with vouchers enter these private schools that are made up predominantly of children of higher socioeconomic status, there may be a positive "peer effect" that helps to boost their academic achievement. To the extent this is true, this would indicate at best the possibility of potential gains for those few voucher students able to gain admission to established private schools, but not for larger numbers entering new private schools, were there to be a growth in the number of private schools as a market response to a large expansion of voucher programs.36 Metcalf's findings in Cleveland point to exactly this problem: Voucher students in already established private schools did as well as their public school peers - better in one subject, science. But those who attended new schools - the ones opened precisely to accept voucher students - did worse in all subjects than their peers in both the older private schools and public schools.