Standardized Tests Cannot Measure Skill
In being boring, standardized tests do not clearly measure what they intend to. It’s a design flaw, and while not fatal to everyone, it is fatal to enough students that it means the tests don’t accomplish their job with nearly the accuracy we need. In fact, for many students, standardized tests actually create the impression of ignorance, because in being boring, the test drives students away from actually demonstrating what they know.
Too often, we don’t really listen to students when they complain. And the single most common complaint I’ve heard over 20 years in education is that Standardized Tests are boring.
It may be that the term ‘boring’ is simply the only word to come easily to the minds of sixteen year olds. We often latch on to the easiest answer when faced with complicated obstacles. There are lots of good reasons to dislike tests, but students have little experience beyond school, and to them the tests are simply parts of their lives they must accept. They lack both the depth of experience to understand how standardized exams affect their personal lives and the professional experience to understand how the tests are supposed to exist in the larger world. Sitting in the middle of that experience, uncertain of its purpose, slaved to readings and questions over which they have no control, eager to escape but not exactly suffering in any tangible way, the only answer to their discomfort and confusion is ‘boring’.
But there is also a more profound reason students lack the vocabulary to explain their struggles with standardized exams. The tests really ARE boring, and it has nothing to do with the students themselves. It’s in the nature of the tests to be boring. Not because they are uninteresting, or dry, or tasteless and bland. Not because they lack energy or vitality. Not because they are repetitive, or simplistic, or opaque. Though they are all those things at times. It is in the nature of Standardized Tests to be boring because their Standardization must necessarily remove context. And in avoiding context, one removes the essential component of all human knowledge and experience: its purpose in the moment.
‘Boring’ isn’t a simple matter of missing interest — the causes of the disconnect are varied and significant. It’s difficult to answer a question, to ‘connect’ to it, if one doesn’t know the point of the question. It’s difficult to connect to an experience if the conditions are unjust or unfair. And it’s quite difficult for many young adults to perform well at a process which has no clear purpose or inherent value to them.
Reading Is contextual, tests are not
Let’s examine a Standardized Reading Exam, such as the SAT or ACT or MAP, as an example.
Reading (like most things) is a contextual experience. The written word does not exist alone, it acts as a bridge between two things in the real world, between the reader and whatever the writing is about. An automobile mechanic uses the words in his manual to connect his actions to the actions of the engine. A surfer reads a surfing magazine to connect his interest in surfing to the wide range of complex details inherent to the sport. Novels provide a link between the reader and the meaning of experience. This essay is on a website for writing and argument, directed at readers (hopefully) interested in education reform, among other things. When the reader engages in the text, they do so in an environment in which the written words are merely a part. They engage with the content.
But the reading portion of a standardized exam deliberately tries to avoid content and context altogether. More to the point, the reading portion of the exam isn’t about the content at all, it’s about the skill of reading words and sentences and paragraphs, independent of any larger purpose the words serve. As such it actively tries to avoid context because in principle it isn’t a test of content but a test of skill, a test of the skill in gleaning content from sentences, as if, somehow, these two components are completely independent of each other. Students are to prove they understand how writing functions without much clear connection between the words and the subject of the writing as it exists in the world.
Most often, paragraphs are lifted from the middle of essays, textbooks, or novels and then students are asked questions related only to that specific passage. The disconnect here for poor or reluctant readers can be profound. Intuitively, students understand the purpose of the written word as a bridge, yet the test itself seems to completely ignore writing’s purpose. Rather than a test of skills, the exam seems to many some sort of trick designed to…well, they aren’t quite sure. What they do know, though, is that the passages seem to lack whatever vitality it is that makes things ‘interesting’ instead of ‘boring’.
It’s in the nature of Standardized Reading tests to be boring because they deny the reason any reading is engaging in the first place. Standardized tests break the contextual energy that makes reading interesting.
The tests also are boring because they demand students read in a way that is unnatural to reading. Students will never elsewhere be asked to read in the manner required by the exam, because, quite simply, it’s not the way (or ‘why’) we read in the first place. Nobody ever reads a passage in order to answer specific questions about it. There’s always a context, and for most readers the context is fundamental to the understanding. If, for example, I am reading a novel and I find myself in a descriptive passage about the weather, I instinctively know that the author is likely setting a mood or tone and that the passage is setting the place of the events. But I know this because I am already in the book, already ‘in’ the experience of the novel. Well-designed questions will focus on this kind of reading, but they will still lack everything that made one read the passage in the first place, much less fully comprehend the purpose of the passage. And such a focus is irrelevant to a student who doesn’t read many novels, much less one who is either confused by the purpose of the exam in general or — as is so often the case given the sheer pressure the tests create — suffers from test anxiety.
The actual assessment aspect of a Standardized exam is also different from that which students usually experience in the classroom. In the classroom, tests related to readings involve specific lessons and details from the recent past, which are reviewed and then built upon (or discarded) for the next lesson. They are part of an ongoing experience, with all sorts of visible and invisible purposes. Vocabulary tests, for example, usually concern words studied or reviewed previously in class. These are lists of specific words defined and spelled. Students know that learning the words is part of a wider process and purpose, the amassing of vocabulary, the recognition of roots, prefixes, suffixes, and cognates, deeper understanding of a text through its vocabulary, etc. (To be sure, the absence of clear purpose within a classroom is also one significant reason students struggle.) None of that larger purpose or extended history is evident on a Standardized exam. Vocabulary skills on a Standardized test are mostly about gleaning definition from extremely limited context clues. While this is no insignificant skill, the full depth of it is lost if the passage has no larger context. Again, students are asked to prove they can glean the meaning of a word or passage from the very small fragment they have before them, a passage often sliced from the body of a much larger creature.
Avid reader, poor test taker
Many students who are enthusiastic readers perform poorly on reading exams. It’s frustrating, particularly since the issue is not skill. Because the point of a reading exam is so different from the reading they do elsewhere, that student actually ‘reads’ differently. When we read, we don’t hold all the information the words contain in equal measure. Without much conscious decision, we evaluate the purpose of the text and group the information accordingly, holding usually only a few common threads and crucial supporting evidence throughout the reading experience. (This is how one can read a 1,000 page novel and not get lost.) Since a reading exam, however, is testing comprehension removed from context, students don’t know where to start. As a result, a weak test taker tries to hold every single word of a reading in their minds, to in effect memorize the passage before they get to the questions. A paragraph into any reading passage and they have lost all track of the beginning. By the end of the passage, they have to go back and start all over again.
Later, they wonder, ‘What happened?’ The standard answer: ‘It was a boring selection.’ But behind that answer is real damage, for these enthusiastic, avid readers exit the experience being told that they are poor readers. Many of them lose the passion.
There are answers to this nightmare, answers actually discovered by many people some time after they leave school, all those folks who go on to great success despite lackluster academic performance. Yet presently, the school system relies primarily on testing to mark, monitor, measure, and motivate. A student might understandably survive their academic years under the impression that the world of successful professional adults operates under the same general guidelines. To many, the thought of this is enough to steer them towards less professional fields. The world outside school, however, accommodates everyone to varying degrees, and it does not rely on testing — it relies on individual product. Like reading itself, the world students graduate into is purpose driven.
Standardized tests do work, of course. They do measure many student’s reading ability, but in many ways what they actually measure is kind of peripheral to the skill of reading comprehension. Good students are very often simply more curious than poor students. Good students are open to ideas, and are willing to gaze past the limitations of a passage to the material within. Good students also read a lot, and over a wide range of topics; thus, they are more likely familiar with the variety of passages and language that shows up on the test. Often their ability is not so much their skill as it is their experience. But it is by no means a democratic or equal measure, and all too many students are not being accurately judged. The big question, then, is this: How can an assessment accurately gauge or evaluate the skill of a process that isn’t actually being fully tested? Is it even an accurate assessment of a student’s true reading skills if the test doesn’t allow the student to demonstrate the skill?
We’ve been evaluating student ability this way for a long time now, but the past does not quite serve as a standard for the present when it comes to standardized exams. The two big hitters — the SAT and the ACT for college — are relatively new to the scene, and for a long time they were hand-graded essay exams. Only recently have the public schools themselves gone full-on with standardized exams. In other words, these tests are not classics. Moreover, the model for the standardized exam is drawn from tests that measure advanced academic potential; that is, success in school. There’s nothing wrong with this approach, so long as it applies to those young adults with an interest in pursuing academic fields. Those are the ones who do well on exams. Currently, however, we apply the model to all students, regardless of their interests or abilities, and bemoan the lack of universal success. The tests are also a product of their time and circumstance. Until recently, these kinds of exams were the easiest, most efficient, and accurate way to measure large groups of essentially anonymous applicants. That is not true anymore. If we so desired, and were willing to undertake the (admittedly strenuous) task of changing direction, we could create an evaluation system that would more accurately gauge ability, not to mention serve — far more accurately — a much wider range of demands.
It may be that the standardized exam is here to stay. Accountability in an institution as entrenched, bureaucratic, and large as the education system is difficult, to say the least, and consistent, across the board, exams are quite possibly the only reasonable path. But let’s at least be honest about both what should be measured — not student academic success on a large scale, but possible student strengths and ability on the scale of the individual — and how best to democratically measure those skills. At the least, we should give the students power over their own destiny. Let them choose what tests to take. Give them some choice in the reading. Let them sense that the exam is merely a tool for direction and proof of achievement.
If we are to use an exam to measure our kids, we should provide as many exams as possible, and allow the scores on those tests to be the badge by which the students earn entry into the work they choose. We don’t need two exams — English and math — or three — writing — or four — add science. We should have hundreds, for everything they choose. And we should not measure them by our choices (which I doubt we even understand), but by the paths they choose. Our students should not be forced to come to us with a piece of paper that judges them by our standards. They should come to us with a piece of paper that says, “This is what I care about and what I’m good at.”