A recent Upshot article written by Susan Dynarski from the University of Michigan suggests that charter schools provide the most benefit to students in poor, urban areas. Recent research I conducted with my colleague Elizabeth Bersin on DC Charter Schools also bears out this conclusion.
Who Benefits? Non-White, Low-Income Students in Urban Areas
Dr. Dynarski’s research mostly focused on charter schools in Boston. Her research uses students who were randomly assigned to a charter school by a lottery system. Dynarski’s work assumes that because those who win the lottery and those who lose the lottery are essentially the same in regards to academic history and overall demographic make up, any observed differences between the students after the fact can be attributed to charter school attendance. Analysis was done based on who won or lost the lottery rather then who actually attended the charter school, which means that the results are not biased by transfers after the lottery. In other words, if a charter school decided to expel low performing students, this did not impact analysis.
The results suggest that for students who are low performing, poor and non-white in urban districts, charter schools tend to increase student test scores more than public schools. By contrast, in suburban areas where students are more likely to be white and middle class, charter schools do not do better then public schools, and sometimes perform worse.
Our Research
Elizabeth Bersin and I conducted our own research on charter schools in the District of Columbia. We were interested in exploring the impact of attending a charter school in DC on academic outcomes for students. For our outcome variable, we used DC-CAS data from three school years, 2012, 2013, and 2014. The DC-CAS assessment is administered in the spring for students in grades 2 -10. These tests were aligned specifically with DC English/Language Arts, Math, Science, and Health Standards. We used test scores for Math and English/Language Arts, which have been in place since 2006 for students in grades 2- 10. The data that we utilized contained the number of students who scored at the proficient or advanced level for each school, and we had an indicator for whether a school was a charter school or part of the DC public school system.
While the studies carried out by Dynarski and other researchers take advantage of the lottery system used to randomly assign students to charter schools or their regular public schools, we were only able to use school level data. Our non-experimental design and results obtained are thus subject to two major sources of bias. One source of bias is that students (or their parents) who attend charter schools differ from those who do not attend charter schools. For example, parents who want their child to attend a charter school may know that the public school in the area is not high-quality whereas another parent might just not know about the differences in quality between the public school and the charter school and thus not apply to attend a charter school. The student in the former situation is likely to have a better academic outcome then the student in the latter situation because the parents are more involved in the student’s education regardless of whether the instruction methods or curriculum at the charter school improve academic outcomes. The research designs used by Dynarski and others eliminate this issue by only following and using data from students who apply to attend the charter school. Another source of bias is that some of the charter schools in our study could have used non-random means to select students (i.e. test scores) or that schools could have expelled low performing students and thus sent them back to public schools before the DC-CAS was administered.
We used OLS multivariate regression to analyze the data and produce our results. One of the control variables we used was the ward number for each school, a proxy variable to measure the socioeconomic status of the students who attend that school (assuming that it is likely that students attend charter schools close to home). The table below provides an indication of some of the key attributes of each ward. Wards 7 and 8 especially have a high African American population and a low median income.
Key Attributes of DC Wards
| Ward | Percent of Population that is Black[1] | Median Household Income | Percent of Families Below the Poverty Line[2] |
|---|---|---|---|
| Ward 1 | 45.74% | $36,902 | 19.7% |
| Ward 2 | 19.93% | $44,742 | 11.6% |
| Ward 3 | 5.78% | $71,875 | 2.7% |
| Ward 4 | 70.73% | $46,408 | 7.9% |
| Ward 5 | 86.65% | $34,433 | 14.3% |
| Ward 6 | 62.73% | $41,554 | 19.1% |
| Ward 7 | 96.84% | $30,533 | 21.6% |
| Ward 8 | 92.41% | $25,017 | 33.1% |
The graphs below present the differences in percent of students who scored at the proficient or advanced level for charter schools versus public schools for each ward. For both math and reading, there is an especially large difference for Wards 7 and 8 and much smaller differences for the other wards. We also found a statistically significant difference for charter schools versus other public schools, which suggests that the number of students scoring at the proficient or advanced level was more than 10% higher for charter schools rather than other public schools (see Appendix).
Our research is by no means as rigorous or as conclusively causal as the research conducted by Dynarski and others, but it is encouraging to see that the urban poor attending charter schools may have higher test scores and thus may have more opportunities in the future to attend college or receive a scholarship. In future research, it would seem even more important to get a better understanding of why urban schools in poor, non-white areas are struggling to increase scores of the population they serve and to determine policy solutions that would help increase achievement for all students.
Appendix
Results for Charter Schools and Percent of Students Scoring at the Proficient or Advanced Level (Math)
| Variable | Coefficient | Standard Error | t-value | Confidence Interval | |
|---|---|---|---|---|---|
| Charter School | .1353*** | .0291 | 4.64 | .0778 | .1927 |
| Ward 1 | -.3493*** | .0622 | -5.61 | -.4721 | -.2266 |
| Ward 2[3] | -.1403** | .0652 | -2.15 | -.2689 | -.0117 |
| Ward 3 | Used as Base for Other Ward Number coefficients | ||||
| Ward 4 | -.3624*** | .0474 | -7.64 | -.456 | -.2688 |
| Ward 5 | -.4169*** | .053 | -7.86 | -.5214 | -.3124 |
| Ward 6 | -.3916*** | .0468 | -8.37 | -.4838 | -.2994 |
| Ward 7 | -.4638*** | .0397 | -11.68 | -.5421 | -.3856 |
| Ward 8 | -.5007*** | .0396 | -12.65 | -.5787 | -.4227 |
| Constant | .8226*** | .0287 | 28.67 | .7660 | .8791 |
Results for Charter Schools and Percent of Students Scoring at the Proficient or Advanced Level (Reading)
| Variable | Coefficient | Standard Error | t-value | Confidence Interval | |
|---|---|---|---|---|---|
| Charter School | .1156*** | .0255 | 4.53 | .0653 | .1659 |
| Ward 1 | -.389*** | .0562 | -6.93 | -.4998 | -.2783 |
| Ward 2[4] | -.142** | .0688 | -2.06 | -.2777 | -.0063 |
| Ward 3 | Used as Base for Other Ward Number coefficients | ||||
| Ward 4 | -.3575*** | .0465 | -7.68 | -.4493 | -.2658 |
| Ward 5 | -.4062*** | .0501 | -8.11 | -.5049 | -.3074 |
| Ward 6 | -.4035*** | .0465 | -8.69 | -.4951 | -.3119 |
| Ward 7 | -.4945*** | .0371 | -13.34 | -.5676 | -.4214 |
| Ward 8 | -.5353*** | .0351 | -15.26 | -.6045 | -.4662 |
| Constant | .8209*** | .0286 | 28.74 | .7646 | .8772 |
Notes
[1] The authors calculated the percent of the population that is Black by dividing the population of Black individuals over the total population. The population data appears to be from 2000 and the income data may be from 2002.
[2] This column in the data is POVFAM. The authors assume that this is a percentage. Poverty guidelines for 2012-2014 are available in Appendix 1, figures 2-4. This also includes a note about the difference between poverty thresholds and poverty guidelines.
[3] The coefficient on Ward 2 is significant, but this may not be a good estimate of the true value because of the small sample size for that Ward.
[4] Similar to footnote 3, the coefficient on Ward 2 is significant, but this may not be a good estimate of the true value because of the small sample size for that Ward.
0 Comments