As Mark Twain famously observed way back in 1906, “There are three kinds of lies: lies, damned lies, and statistics.”
We’re not suggesting in this editorial that anything like that occurred with Maryland’s scores on the 2013 National Assessment of Educational Progress test, but read on.
There appears to be a fair amount of confusion and skepticism in the public mind surrounding educational “assessment” in general. What do all these tests really measure? What is the significance of rankings? Are schools too focused on “teaching to the test”?
Now we learn that Maryland’s scores and national placement on the 2013 NAEP test are skewed. The reasons why raise a few questions about both this national test and reading education in Maryland.
As with all such tests, we assume that NAEP results are supposed to indicate how well a state such as Maryland is doing in teaching subject matter — in this case, reading — and how it stacks up against other states.
A recent Associated Press story, however, reported that Maryland had excluded a significant percentage of students in two key categories from taking the test — English language learners and students with learning disabilities. Obviously, this bumped up the state’s scores and national placement.
In fact, Maryland blocked 62 percent of fourth-graders and 60 percent of eighth-graders in these two categories from taking the NAEP test.
According to the AP story, Maryland’s exclusion rate was “more than double that of any other state” taking the test.
The story went on to report that “The governing board overseeing the test has set a goal that states exclude just 15 percent of learning-disabled and English language learners.”
“Set a goal”? What does that mean? Apparently nothing, since Maryland’s exclusion rate was four times that high.
How is the state Board of Education explaining all this? This way: Maryland accommodates learning-disabled students on yearly exams by permitting the text to be read to them by a person or computer, as opposed to requiring them to read it themselves. And since NAEP doesn’t permit this read-aloud accommodation during its annual test, those students were excluded from taking it. Voila!
Where to start? First of all, if NAEP results, especially comparative ones, are to mean anything, every state must be playing under the same rules. Clearly that was not the case in the 2013 test. Second, how and to what extent does reading test material aloud to students, in lieu of their reading it themselves, actually measure their reading ability?
What’s the verdict of Maryland’s 2013 NAEP scores and comparative ranking? Referring to the Maryland exclusion percentage, Lindsay Jones, of the National Center for Learning Disabilities, says, “That number is a red flag. It stands out this year in particular because NAEP’s (average) exclusion rate has dropped so much.”
Clayton Best, who is NAEP’s Maryland coordinator, says he’s concerned about “the implication that this is a conscious process to eliminate students taking the test to improve the NAEP scores. There is no motivation to do that at all.”
Let’s say that’s true. But even so, it doesn’t change the fact that Maryland’s scores on this test, and its ranking among the other states, require asterisks.
Sorry, Maryland, but the chest bumps and bragging rights will have to wait for another day. Meanwhile, NAEP needs to issue some specific new rules as to exactly who must take this test — and dump the nonbinding “goals.”