In a recent article published in the City Journal, Stern (2007) defends Reading First, the reading portion of No Child Left Behind. Stern (a Contributing Editor to City Journal and a Manhattan Institute Senior Fellow) claims that Reading First has worked, and cites evidence from three places: Virginia (specifically, Richmond City and Fairfax County), the state of Alabama, and New York City (comparing Region 5 with the rest of the city).
I will discuss each of these sites in turn. In some cases, a closer look at the data shows that the results are not what Stern says they are. In other cases, I will conclude that it is premature to give Reading First the credit for gains that are present. I will then deal with several of Stern’s unfounded statements about reading instruction in general in the United States.
Virginia: Richmond City and Fairfax County Stern claims that Richmond, a city that embraced Reading First, made great gains, but Fairfax, an area that “disdained” Reading First, has not done as well. Note that Stern is comparing the city of Richmond (25,000 students) with Fairfax county (190,000 students).
The data he presents is only from African-American students. He notes the percentage of African-American third graders who passed the state’s reading exam in Richmond rose from 62 percent in 2004 to 74 percent in 2005, but for African-American third graders in Fairfax, the percentage rose from 52 to 59 percent. So Fairfax increased by seven percent, Richmond increased by 12 percent.
But let’s add 2006 to the picture:
African-American third graders: percent passing | | 2003-4
| 2004-5
| 2005-6
|
Richmond
| 62
| 74
| 76
|
Fairfax
| 52
| 59
| 69
|
Between 2005 and 2006, there was a ten percent increase in Fairfax, and only two percent in Richmond. Looking at the difference between 2004 and 2006, Richmond increased 14 percent, and Fairfax increased 17 percent. Now Fairfax is the winner.
But the crucial group is not African-American, it is children of poverty: Reading First is aimed at this group. Here is the data. The difference is not large, but again, the winner is Fairfax, not Richmond, with a gain of 17 percentage points from 2004 to 2006, compared to Richmond’s 15 points.
Children of Poverty: percent passing | | 2003-4
| 2004-5
| 2005-6
|
Richmond
| 60
| 73
| 75
|
Fairfax
| 57
| 62
| 74
|
Stern also claims that Richmond’s third grade scores are “closing in on wealthy Fairfax’s scores for all its students, 79 percent of whom passed the third-grade reading test in 2005.” It looks like this when we only consider the one year interval 2003-4 to 2005-6, but when we add 2005-6 to the analysis, things look very different. Although Richmond gained slightly more over the last two years, Fairfax improved much more the second year. Richmond is no longer “closing in.”
District/city wide: Percentage passing | | 2003-4
| 2004-5
| 2005-6
|
Richmond
| 63
| 76
| 78
|
Fairfax
| 76
| 79
| 87
|
None of Stern’s claims about Fairfax and Richmond hold when we consider a wider set of data.
Alabama Stern notes that Alabama “enthusiastically welcomed” Reading First, and was the first state to be approved for it. Reading First was in place early in all eligible Alabama schools, in 2002-2003. Stern reports that “On state reading tests, Reading First students rocketed from 29 percent at grade level in 2004 to 39 percent in 2005 and 46 percent in 2006.”
The US Department of Education’s recent report on the progress of Reading First students does not match these cheerful numbers: On the Dibels tests, Alabama’s gains were indeed spectacular from 2004 (46.2% reaching proficient) to 2006 (70.8% reaching proficiency). This, however, is Dibels, a test of pronouncing texts, reading outloud, with no requirement that the child understand what was read. There are numerous reasons to doubt the validity of this test as well as doubt that it has anything to do with real reading (Goodman, 2006). Dibels is also easily available on the internet, which means teachers can drill students on the actual test.
On the SAT10 reading test, which includes reading comprehension, Alabama students didn’t “rocket” at all: In 2004, 44% of Alabama Reading First children scored at proficient (the 40
th percentile on the SAT10), which increased to 49.5% in 2006, a 5.5% increase, much less than the gain seen on Dibels and much less than that claimed by Stern. According to my analysis, this is slightly less than the average gain for all children in Reading First nation-wide (Krashen, 2007).
Even this 5.5% increase does not demonstrate that Reading First has succeeded. First, there was no control group; test scores have been rising for all students for a variety of reasons that may have nothing to do with increased proficiency. Second, Reading First students have an extra 100 minutes per week in reading instruction. Over two years, this could amount to an extra semester. Over a four semester/two year period, this means we might be comparing five semesters of instruction with four semesters. We should thus expect a gain with any method that provides some exposure to comprehensible text. As Coles (2003) puts it, we are comparing Reading First to doing nothing, to doing no reading instruction at all.
Stern also claims that in Alabama, “On diagnostic reading tests for early-grade children, the Reading First cohort has—astonishingly, since it encompassed the lowest-performing students in the state—almost reached parity with Alabama’s broader student population.” This assertion is hard to pin down, because the state presents overall scores in terms of average percentile, not in terms of percent of students who reach the “proficient” level. Even so, Stern’s claims are not confirmed by data from the State of Alabama website: On the SAT10, the overall third grade mean for 2004 was 46, in 2005 it was 48 and in 2006 it was 50. That’s a four percent improvement. On the same test, the percentage of Reading First third graders who reached the proficient level rose 5.5% in the same time period. The data thus provides no evidence that Reading First has resulted in any noticeable gap-closure.
Stern’s claims for Alabama do not hold.
New York City: Region 5 vs. the rest of the city According to Stern, most of New York City adopted a “loose” phonics-based program, but in contrast, Region 5 has participated enthusiastically in Reading First, with a superintendent who favors “tightly scripted phonics programs.” Region 5, according to Stern, has “twice as many in the program as the next-highest city region.” And of course Stern claims that “early results in Region 5 are positive …”.
Stern maintains that third grade scores in Region 5 have increased, “with all of the schools that have gone through the two years of the program so far seeing their third-grade reading scores go up. The increases range from 10.5 percent at P.S. 215 to 36 percent at P.S. 65.”
The New York City Department of Education website provides overall scores for grades 3 and higher in reading, giving the percentage of children who scored in the two highest levels (out of four). Region 5 did indeed make an impressive gain in reading from 2004 to 2006, with 26% more scoring at the top two levels in 2006 than in 2004. The citywide gain was 16%; Region 5 exceeded this by 10%. (Note: The citywide percentages remain nearly identical when Region 5 is removed: 2004 = 41.8, 2005 = 53.7; 2006 = 60.8).
Percent of students in two highest levels: New York City | | 2003-4
| 2004-5
| 2005-6
|
city
| 45.7
| 53.5
| 61.5
|
region 5
| 37.4
| 46.3
| 63.6
|
(Note: I assume that the New York tests are in fact tests of reading comprehension, which the website indicates they are, and not tests of pronouncing words outloud.)
Moreover, inspection of the data from individual districts reveals that the other nine districts’ gains are very similar to each other (note the small standard deviations; mean gain from 2004 to 2005 = 7.8, sd = .87; mean gain from 2005 to 2006 = 7.01, sd = 1.68). The exceptional gain is Region 5 from 2005 to 2006. Region 5’s gain from 2004 to 2005 was 8.9, but from 2005 to 2006 it was 17.2, much greater than the gains for the city as a whole (far outside the confidence interval).
The reason that nearly all the Region 5 advantage is in 2006-2005 could be because Reading First targets the lowest grades, and it took some time for the effect to be felt by the third grade; those tested in 2005 may have had less exposure to Reading First.
The Region 5 case, thus, is the only one of Stern’s cases that appears to hold up after inspection of the data.
Before declaring victory for Reading First, however,
it must be noted that Reading first entails an extra 100 minutes of reading instruction per week. As noted earlier, this could amount to an extra semester over a two-year period. It is quite possible that a different kind of supplementation would have resulted in better gains.
I am assuming that Region 5 did indeed implement Reading First as intended, and that the other regions had, in fact, a “looser” approach to reading. If this is so, the overall gains in New York City are due to factors not specific to Reading First; at best Reading First can claim credit for the additional ten percent gain in the percent of children in the upper two levels in 2005-6 in Region 5.
Stern also claims that failure to do Reading First met with disaster: “Several Bronx schools dropped out of Reading First after two years, basically wasting millions of dollars in federal funds, and returned to balanced literacy. And the children still can’t read.” Unfortunately, Stern does not tell us which schools they were, so this claim cannot be checked.
As Stern claimed, Region 5 did, in fact, make better gains in other districts. They occurred only for one year, however. The gain was an additional ten percent who reached the highest two levels, compared to the rest of the city. But we must consider the fact that children in Reading First had extra instruction. (1)
Post-script to New York discussion: Bubble children in New York? The gains in 2005-2006 in Region 5 and city-wide came mostly because of movement into level 3, the lowest of the two levels of “proficient” reading. In fact, the percentage of students in level 4 was lower than in previous years, not just in grade 3 but in all grades (overall citywide (grades 3-8), the percentage of children in level 4 in 2006 = 5.7%; in 2005 = 14.6%, in 2004 = 9.6%).
Region 5 | | level 3
| level 4
|
2006
| 58.2
| 5.4
|
2005
| 33.1
| 13.2
|
2004
| 27.1
| 10.4
|
City-wide | | level 3
| level 4
|
2006
| 56.3
| 5.2
|
2005
| 34.2
| 19.3
|
2004
| 30.5
| 15.1
|
This raises the suspicion that throughout New York there has been an emphasis on “bubble children,” (Boother-Jennings, 2005) getting those slightly below the proficient level to slightly above, as a response to Reading First/NCLB pressures. Unfortunately, the city does not provide overall means for 2006 (but does for previous years).
Unfounded Accusations Are American children poor readers?
Stern asserts that American children don’t read very well: “40 percent of U.S. fourth-graders are reading below the minimally acceptable level, according to the gold-standard NAEP test.”
There is no convincing evidence that “American children don’t read very well.”
Gerald Bracey (see Bracey, 2007) has pointed out that the NAEP standards, e.g. knowing exactly what score one must get on the NAEP to be a “minimally acceptable” reader, are arbitrary and have been rejected by several important organizations: the Government Accountability Office; the National Academy of Sciences; the National Academy of Education; and the Center for Research on Evaluation, Student Standards and Testing.
When estimates are made of how well students in other countries would do on our NAEP, many of those in high-ranking countries would fall short of the “proficient” level. Sweden was ranked number one in the world in the last international study, but Bracey notes that only 33% of Swedish fourth graders would score at the “proficient” level on NAEP, nearly identical to the overall average for fourth graders in the US (29%). According to Bracey, “The great majority of the remaining countries would have fewer proficient students than the United States. Using the NAEP standard, no country comes close to having a majority of proficient readers.”
The “problem” with American children and reading is the fact that children of poverty typically do not read well, which is directly related to the fact that they have little access to reading material at home, at school, or in their communities (Krashen, 2004).
Nostalgia for the past Like many others, Stern suffers from an unjustified “nostalgia for the past”: “Making the situation more tragic, nineteenth-century American children learned to read very well, thank you, in one-room schoolhouses, with nothing more than a single determined teacher wielding Noah Webster’s Blue-Backed Speller and the McGuffey readers. Even before a public school system existed in America, Alexis de Tocqueville had marveled at the country’s extraordinarily high literacy rates.”
Just as they do now, people in the nineteenth century complained about low levels of literacy. In 1874, Harvard University instituted written entrance examinations, and many (more than half) of the applicants failed. Ten years later, another study yielded the same results, which resulted in the establishment of remedial writing classes. As a result of an analysis of essays written in 1894, the Harvard Board of Overseers criticized high school writing teachers for the poor performance of the students. Note that these were the best students in the country attending the best university of its time (James Berlin, cited in “The Role of Prebaccalaureate Programs (AKA Remediation) in the California State University,” (http://www.calstate.edu/AcadSen/Records/Resolutions/2004-2005/2687.shtml)
The complaints about the low levels of literacy of American students have continued since then: According to Hofstadler (1963), in 1930 Thomas Biggs of Teachers College wrote that high school English classes resulted in written English that was “in a large fraction of cases shocking in their evidence of inadequate achievement” (Hofstadler, p. 304).
If we believe all this, our schools were terrible in 1874 and have been getting even worse ever since. Another interpretation is that there has been no decline in performance, that we have always been expecting too much. For evidence against the claim that performance has been declining, see Bracey’s discussion of “knowledge nostalgia” in Bracey (2004).
Back to the basics “The most effective reading instruction for most children—especially for those from disadvantaged homes—begins by training them to recognize the relationship between letters and the sounds they make (phonemic awareness), moves on to teaching them how to sound out whole words (phonics), and then focuses on fluency, vocabulary, and comprehension.”
Stern confuses “phonemic awareness” (the ability to divide a word into its component sounds), and phonics (knowledge of the relationship between sounds and letters, known as “sound-spelling correspondences”). His claim that the most effective instruction begins with phonics and then moves to words, and then comprehension, has been widely challenged in the research literature by highly respected scholars (e.g. Smith, 2004; Goodman (see Flurkey and Xu, 2003), who present evidence showing that our ability to use phonics, read words, and read fluently with comprehension is the result of real reading, of making sense of texts. (A modest amount of phonics instruction can help make some reading more comprehensible, but heavy, intensive phonics instruction has limited value in learning how to understand text; see especially Garan, 2001a, 2001b).
Stern dismisses this approach with this statement: “The balanced-literacy cultists believe that learning to read is a natural process and that most children can intuit the alphabetic principle and the meaning of printed words with a little guidance from a teacher and through pleasant cooperative classroom activities such as ‘shared reading’ and ‘reading circles.’ Basically, this approach says that kids can learn to read by reading—by immersing themselves in print. And for some children from literate homes, where print and articulate conversation abound, this approach can work.”
This approach can work for everybody simply by making sure “print and articulate conversation” are in abundance in all classrooms. Unfortunately, Stern and others never mention this as a possibility. In fact, Stern maintains that spending more on books is undesirable, despite massive evidence that providing access to print is a powerful means of encouraging literacy development, and strong evidence showing that children of poverty have little access to print (Krashen, 2004):
“It’s not hard to see why an independent grant reviewer, picked for his knowledge of science-based reading instruction, would find the city’s submission troubling. For starters, the city proposed spending a big chunk of the first year’s $37 million grant on things that had zilch to do with science-based reading instruction—$3 million on library books, for example. (It is balanced literacy, not explicit phonics, that fetishizes the idea of surrounding children with “authentic literature,” believing that the backdrop will stimulate their reading lessons.)”
It is not the “backdrop” of literature that stimulates reading; it is reading literature that promotes literacy development: Study after study confirms that reading for meaning (especially “free voluntary reading”) is the source of our reading ability, our ability to write well, much of our vocabulary and spelling knowledge, and our ability to handle complex grammatical constructions (Krashen, 2004). Spending money on school libraries is the best investment we can make, especially for children of poverty, who have few sources of reading material elsewhere. Studies in fact confirm that library quality is related to reading achievement (Krashen, 2004).
Note: (1) Region 5 fourth graders also made the largest gain in the city in percent of children in the two highest levels in reading proficiency between 2004 and 2006 (12%, compared to the city mean of 9%), but the increase came the first year after Reading First was introduced (2004-2005). The second year, the percentage actually declined slightly both in Region 5 and citywide. It is unlikely that Region 5 deserves the credit for the gain between 2004 and 2005. As noted earlier, Reading First features an intensive phonics approach which is used mostly in the early school years; not many fourth graders in 2004-2005 would have been affected.
References Allington, R. (Ed.) 2002. Big Brother in the National Reading Curriculum: How IdeologyTrumped Evidence Portsmouth, NH: Heinemann
Boother-Jennings, J. 2005. From classroom
to emergency room:
Educational triage
in American schools.
http://nochildleft.com/2005/nov05triage.html
Bracey, G. 2004. Setting the Record Straight. Portsmouth: Heinemann. Second Edition.
Bracey, G. 2007. A test everyone will fail. Washington Post, May 3, 2007.
Coles, G. 2003. Reading The Naked Truth: Literacy, Legislation, and Lies. Portsmouth, NH: Heinemann.
Flurkey, A., and Xu, J.. Eds. 2003. On the Revolution in Reading: The Selected Writings of Kenneth S. Goodman. Portsmouth, NH: Heinemann.
Garan, E. 2001a. Beyond the smoke and mirrors: A critique of the National Reading Panel report on phonics. Phi Delta Kappan, 82(7), 500-506.
Garan, E. 2001b. What does the report of the national reading panel really tell us about phonics? Language Arts, 79, 61-70.
Goodman, K. 2006. A critical review of Dibels. In K. Goodman (Ed.) The Truth about Dibels. Portsmouth: Heinemann.
Hofstadler, R. 1963. Anti-intellectualism in American Life. New York: Vintage Books.
Krashen, S. 2004. The Power of Reading. Westport, CONN: Libraries Unlimited and Portsmouth, NH: Heinemann.
Krashen, S. 2007. Reading First: “Impressive” gains? The Pulse. http://www.districtadministration.com/pulse/commentpost.aspx?news=no&postid=18974
Smith, F. 2004.Understanding Reading. Sixth Edition. Hillsdale, NJ: Erlbaum.
Stern, S. 2007. This Bush educational reform really works. City Journal, 17 (1).
