A Path Forward for Literacy in DPS
Dear Members of the School Board, Dr. Harris, Dr. Mozingo, Ms. Wilson-Norman, and Ms. Thomas,
Too many students in Durham Public Schools are not achieving at desired levels. The problem is particularly acute for our children of color and low income children. To address this problem, Durham Public Schools has mandated full implementation of the Reading Street curriculum for all teachers. We understand their goal in doing this but believe this approach is wrong for several reasons.
1. The Reading Street Curriculum is not effective. The publisher, Scott Foresman, contracted for an independent 2-year study of the efficacy of the Reading Street Curriculum. The study found that students in Reading Street classrooms performed worse than students in the control group classrooms. According to the report, “whether a student participated in the Reading Street program did not have a significant effect on reading performance measured by the GMRT-4 or DIBELS gain scores. While students in the control classrooms gained more than students in Reading Street programs did, it is important to note that the effect sizes for these analyses were extremely small, -0.003 and -0.06, respectively, and represented less than one- and three-point percentile differences” (Magnolia Consulting, 2007, p. 49—emphasis added).
2. The more teachers used Reading Street, the worse students performed. In the same study, classrooms that had higher implementation fidelity with Reading Street did overall worse than classrooms with lower implementation fidelity. According to the study, “higher implementation fidelity was associated with lower posttest performance on the GMRT-4 across [all] grades and higher performance on the DIBELS PSF test for first-grade students” (Magnolia Consulting, 2007, p. 38—emphasis in text). Since the study did not experimentally test different levels of implementation, we see a couple of possible reasons for this. One reason could be that using Reading Street more frequently caused test scores to decline. The other reason could be that better teachers used Reading Street less frequently.
3. The implementation of this initiative has been done in such a way that failure will be very hard to avoid. School reforms fail for a variety of reasons including lack of buy-in from teachers or from parents and the broader community (Edmunds, 2005). The short notice given teachers, the lack of professional development for teachers and coaches, and the mandated nature of the reform make it highly unlikely that this effort will be successful. Richard Allington (2005) wrote that “relying on a one-size-fits-all curriculum – even one found to be ‘evidence based’ – was not one of the characteristics of the nation’s most effective teachers. Instead, we observed these teachers selecting and experimenting with multiple instructional approaches, always driven by the responses of their students” (Allington, 2005, p. 1).
Based on the above reasons, a forced mandate of Reading Street will almost assuredly not have the desired results; however, we agree that the district needs to do something to ensure that all students are literate at high levels. Going back to way things were is not acceptable.
Evidence indicates that it is not adherence to a specific package or curriculum, but utilization of best practices, regardless of the curriculum, that has the greatest impact on achievement and teacher success (Allington, 2005). We therefore encourage and will support the district in efforts to implement the following evidence-based best practices (most of which are consistent with the districts’ literacy audit):
Obtain funding to support expansion of pre-school programs to expose more children to developmentally appropriate early literacy activities;
Work with teachers to help them incorporate the core components of reading development including phonemic awareness, phonics, reading fluency, and comprehension, which includes helping students develop their vocabularies through hands-on, multi-sensory learning activities (National Reading Panel, 2006). For some teachers, using the Reading Street curriculum may be the best way to incorporate these components. Other teachers may already have successful strategies in place.
Work with principals to help them support teachers in literacy instruction and help them monitor instruction effectively. One particular model may be the observation protocol developed for the Chicago Reading Initiative by one of the consultants who did the literacy audit.
Differentiate professional development for teachers. Use student data to identify teachers who are not successful in literacy instruction and provide them with more intensive professional development. Explore opportunities to challenge teachers who are already being successful.
Assist teachers in implementing formative assessment strategies (not more formal testing) to determine where students are and how they can be moved forward.
Ensure that all students have access and opportunity to read rich, interesting, authentic texts on their level. The use of the Lexile framework to identify books at the appropriate level is promising.
Provide an evidence-based intensive intervention, such as Reading Recovery, for the students who read substantially below grade level.
Help teachers integrate comprehension instruction into the content areas. As teachers have students read science or social studies texts or primary sources, they should purposely instruct students in how to comprehend these texts. Professional development in strategies like Visualization and Verbalization (Bell, 1991) and Story Grammar Markers should be available for teachers.
Provide a structured opportunity for teachers within a school to engage in vertical planning to ensure that each grade level appropriately builds upon the previous one and is not repetitive. This could easily happen during part of the time dedicated to Professional Learning Communities.
Reduce the class size to a maximum of 15 students in Kindergarten through second grade, and a maximum of 17 students in second grade through fifth grade, as the evidence has shown good effects for literacy development and math achievement in at-risk, minority, and ELL children with small class size (Arias, Walker, Douglas, 2004, and American Educational Research Association, 2003). Researchers provide the following recommendations for policymakers to have maximum effect on achievement: “First, early intervention is important. Start in kindergarten or first grade. Second, the number of students in a class should range from 13 to 17. Third, if resources are scarce, target implementation by focusing on at-risk students. Fourth, maintain intensity by ensuring that students experience small classes every day, all day. Fifth, small classes should last at least two years for initial benefits and three to four years for longest-lasting benefits after the small classes are over” (American Educational Research Association, 2003, p. 4)
Maximize the amount of time teachers can spend on instruction. This can be done by reducing unnecessary paperwork and unnecessary testing that serves little to no instructional purpose.
We hope that Durham Public Schools will take advantage of the passion and energy that has been uncovered as part of this effort to improve literacy instruction for all of our students.
References:
Allington, R. (2005). A Special Section on Reading Research – Ideology is Still Trumping Evidence. Bloomington, IN: Phi Delta Kappan.
American Educational Research Association (2003). Class size: counting students can count. Washington, DC.
Arias, Walker, Douglas (2004). Additional evidence on the relationship between class size and student performance. Washington, DC: Journal of Economic Education.
Bell, N., (1991). Vizualization and Verbalization for Language Comprehension and Thinking. Nancibell, Inc.
Edmunds, J. (2005). Learning from failure: a discussion guide on high school reform.
Greensboro, NC: SERVE Center at the University of North Carolina at Greensboro.
Magnolia Consulting, LLC. (2007). An efficacy study of Pearson Education’s Reading Street curriculum: Year two report. Louisa, VA: Author.
National Reading Panel. (2006). Report of the National Reading Panel: Teaching children to read. Washington, DC: National Institutes of Health.
Filed under: literacy | 2 Comments
Tags: best practices, literacy, Reading Street
DPS Parent Advocacy Blog 
On behalf of Magnolia Consulting, LLC and the research studies we conducted on the Reading Street program, I’d like to take this opportunity to provide an accurate interpretation of the findings. My intention is not to give favor to Pearson or to the concerned parents and teachers, but to clarify particular assertions that are being attributed to Magnolia Consulting’s research. We conducted two separate studies during the 2005-2006 and 2006-2007 school years in which teachers were randomly selected to use Reading Street or their existing curriculum.
The studies did not find that students in Reading Street performed worse than control students. There were no statistically significant main effects between treatment and comparison groups that would support this claim or the claim that the program was not effective. If you say the Reading Street program is “not effective,” you also would have to say the comparison programs were equally ineffective. Or, you could say they were equally “effective” at promoting student gains in reading, but it is inaccurate to say that one was more effective or ineffective than the other. Groups performed comparably, even though Reading Street teachers had only used the program for one year. Saying the programs were ineffective would imply that students actually showed decreases in student achievement, which was not the case for either group.
Descriptively, but not statistically, there were some differences between treatment and control gains on assessment subtests, but in many instances, these differences favored Reading Street, and in other instances, they favored the comparison groups. Differences in these instances were very small and not statistically significant in most cases. It is also important to note that sub-group findings by grade, subtest, and demographic categories divide the sample into smaller groups; thus, results are for descriptive, exploratory purposes and should be interpreted with caution since apparent differences could be due to chance.
The study findings about implementation fidelity and student achievement are correct, yet they do not indicate that “Reading Street is not likely to have desired results” considering the significant gains Reading Street students made at all ability levels (below, average, and above) in reading achievement—wouldn’t that be a desirable result? It should be noted that in a one-year study it is difficult to provide any explanation of the relationship between implementation fidelity and achievement, especially because Reading Street teachers were new to the program. From a research perspective there could be other possible explanations in addition to the ones voiced by the DPS parent advocates. Implementation fidelity was based primarily on teachers’ weekly self reports through online implementation logs, so there could be some respondent bias. Furthermore, it is possible that because Reading Street purports to “raise the bar” for teachers and students, it takes longer to see the positive effects when implemented properly, and sometimes interventions will see a first-year slump.
It needs to be very clear that Magnolia is not employed by Pearson. We are an independent evaluation firm specializing in curriculum efficacy studies, and we conduct studies for a variety of publishing companies, including publishers of other elementary reading programs. At the core of our practice and our profession are ethical standards we take very seriously, and we stand as unbiased, objective parties in the research process. Therefore, Magnolia Consulting neither endorses nor devalues Reading Street, as that is not the purpose of our work.
I understand and completely respect that DPS parent advocates feel very strongly about what is best for their children. Likewise, I hope they respect my need to clarify what can and cannot be claimed about Reading Street based on the research. Accurate information is critical to purposeful and meaningful dialogue.
Respectfully,
Stephanie Baird Wilkerson, Ph.D.
President, Magnolia Consulting, LLC
Dr. Wilkerson,
Thank you so much for taking the time to respond thoughtfully to our comments. As someone who does evaluation work, I have respect for your report and for Pearson for posting it on their website (too many evaluations get buried in the circular file).
I agree with you that students made gains from the beginning of the year when using Reading Street and that those gains were not statistically significantly different from the gains made by students using other programs, although many of the Reading Street gains were descriptively lower. The question is not whether students made gains or not (at the age you are talking about, almost all children make some level of gains in reading simply by absorbing information from their environment). The question really is: did they make more gains by using this program than they would have done with whatever else people were already doing? Thus “effectiveness” needs to be defined as doing something better than the status quo; this is how the What Works Clearinghouse interprets effectiveness. This is the definition of effectiveness used when we say that Reading Street is not effective.
This issue is particularly important since Durham Public Schools has caused significant upheaval by requiring everyone to switch from what they were already doing to Reading Street. Because your study showed that using Reading Street did not result in bigger gains than what people were already doing (lower gains in the most appropriate multilevel analyses), this switch is more than likely not worth all of that upheaval.
In addition, because the sample in your study was deliberately selected to exclude low-performing schools and schools with significant percentages of ESL students, it is not at all clear that Reading Street will be effective in Durham schools.
I agree with you that, in general, it is wise to give a program more than one year to show results. I look forward to seeing the results of the large-scale experimental study of Reading Street being undertaken and would encourage Durham Public Schools to pay attention to those results as well. I also hope this study will compare Reading Street to other approaches to reading in addition to the other basal series you examined in your study.
I am glad that you agree that our interpretation of your findings on implementation fidelity was correct. I question, however, your suggestion that response bias is possibly responsible for these findings. I see no reason that teachers who are getting better results from their students would respond to the online data collection any differently than lower performing teachers. Do you have any data from your study that suggest this is the case?
Our main point here is that, when a program is mandated, the standards of proof should be high. Programs that are mandated should be virtually guaranteed to provide significantly better results than what is already being done. As shown in your study, this is not the case with Reading Street.
Thank you again for your willingness to take the time to respond to us.
Julie Edmunds, Ph.D.