Fact Sheets: The Truth About Vouchers

Research on the Impact of Vouchers on Public Schools

Research on the impact of vouchers on public schools In a recent study widely touted by advocates of private school vouchers, researcher Jay Greene claims that the threat of vouchers caused significant improvement in Florida public schools rated 'F' in 1999 on the state's Florida Comprehensive Assessment Test (FCAT) exams.37 Greene established that F-rated schools showed greater gains than schools with higher grades, and went on to argue that it was the threat of vouchers that motivated these schools to do better. In fact, as several researchers have shown subsequently, the study has serious flaws and does not demonstrate that the improved test scores of students at these 'F'-rated schools was caused by or in any way related to the threat of vouchers.

Greene's study flawed

  • Significant methodological flaws in Greene's analysis have been identified by Rutgers University researchers Gregory Camilli and Katrina Bulkley, including the fact that he uses school-level statistics rather than the more conventional student-level ones and yet asks in effect that his resultant "effect size" data be interpreted in the usual way. This leads to greatly inflated estimates of the gains made by the 'F'-rated schools. They find Greene's conclusions to be "implausible."38
  • A second study, by Haggai Kupermintz of the University of Colorado at Boulder, demonstrates that the "gains" made by Florida's 'F'-rated schools may have been more apparent than real. Kupermintz' analysis supports what had become the conventional wisdom in Florida at the time, shortly after the FCAT test results were announced in June 2000.39 This explanation was that the schools knew what was required to avoid an 'F' grade: they needed only to pass any one of the three sections on the FCAT, reading, math or writing. Of the three, writing was a far easier target for specific test preparation. Schools did not want to be branded again as "failing schools," they knew the easiest route to escape that fate, and they succeeded - but without necessarily improving actual classroom learning. As Kupermintz concludes, Greene's report provides a "a false sense of a dramatic success" with respect to vouchers.40
  • Greene failed to pursue and investigate other plausible explanations and causes for the improved (FCAT) scores in 'F'-rated schools. For example, Greene overlooks the significant amount of extra resources, both state and local, that were directed towards the 'F'-rated schools. This enabled schools to reduce class size and extend the school day and year in order to help their students improve their performance, and may well have been one of the real causes of improvements in these schools.41

Resources and accountability can improve public schools, not vouchers

  • Greene also never considered whether the very fact that schools were rated 'F' - the so-called 'Scarlet Letter effect' - regardless of the possibility of vouchers, stimulated the observed performance improvements. An Economic Policy Institute (EPI) report by Stanford researcher Martin Carnoy did examine this question empirically, testing Greene's "voucher theory" against a "scarlet letter theory."42 Like Florida, Texas and North Carolina have accountability systems that draw public attention to school failure, and Florida had a similar system prior to the introduction of vouchers in 1999. Carnoy and colleagues compared the post-voucher test score gains in Florida against test score gains in these three non-voucher contexts:
  1. Specifically, EPI utilized the fact that Florida's FCAT had been the basis of rating schools even before vouchers were added in 1999. They compared gains in Florida in 1997-98 - before vouchers were introduced - by students in schools given a "critically low" rating in 1996 (comparable to the 'F' rating of the current system) with Greene's data on the gains made by students in F-rated schools 1999-2000 (post-voucher). They found that improvements in math scores were larger in 1997-98 when there was no "voucher threat," while gains in reading and writing were larger after vouchers. They concluded that vouchers do not "have a significant positive impact on public school performance."43
  2. Comparing Texas with Florida, the EPI study found overall test score gains (not just for failing schools) in Texas and Florida to be similar. However, for the failing schools, gains in Texas were as large or larger than the gains calculated by Greene for Florida's schools, "casting doubt on Greene's claim that a voucher threat was the impetus for such growth in Florida".44 Indeed, Greene's own research in Texas points to an alternative explanation for Florida's rising test scores: accountability, testing, and increased resources can promote school improvement, independent of vouchers. Greene himself, in an earlier article, found that the state's Texas Assessment of Academic Skills (TAAS) - the set of tests that is the centerpiece of the Texas state accountability system - was critical to what he has called the "Texas school miracle" (the significant gains in some Texas achievement results during the 1990s).45
  3. In North Carolina, the EPI study found that the low-performing schools had larger test score gains than higher-rated categories of schools, as in Florida - again, with no threat of vouchers. The authors, therefore, find that Greene "inappropriately attributed the differential gains to the voucher program rather than to the other effects of being labeled a failing school, such as shame, increased scrutiny, and possibly additional resources,"46 and that "the results that Jay Greene found for Florida probably have little or nothing to do with vouchers."47
Share this page: Facebook Twitter Digg SU Digg Delicious