How to Assess the Educational Health of a School District?
by Kathy Emery
last update October 2008
The last two superintendents of the San Francisco Unified School District have based their reputations on raising standardized test scores. Both claimed to have done so, but the FACTS belie the claims. High-stakes testing advocates such as Mayor Gavin Newsom and Superintendent Arlene Ackerman claim that they are "not ideological, but results driven." But test scores as "results" are based on ASSUMPTIONS about what signifies intelligence and which knowledge is important. Basing assessment on test scores alone, or even primarily is highly ideological.
"It is wrong to evaluate schools based on students' standardized achievement test scores. Educators and non-educators alike need to understand why." James Popham, The Truth About Testing: an Educator's Guide to Action
1. Standardized tests are created so that the results closely correlate with socio-economic status. This is done ON PURPOSE. This ensures their "reliability," i.e., students get the same score every time they take the test. For an example of how tests are strongly correlated to SES view a map comparing selected census track's median income with the API of the neighborhood's elementary school in San Francisco in 2003. More data: look at a map of median income and compare it to API scores of all elementary schools in San Francisco, all middle schools and all high schools. (see a POWER POINT presentation of test scores and income correlation in SF)
You can look at the test score data from one elementary school in SF during the last 5 years. And then look at Popham's analysis of a standardized test to determine how many items are correlated to SES and how many to "inherited aptitude." Not much is left over for other kinds of interventions. [more by Popham and Berlak]
2. There are many and various "threats" to the "validity" of a test score. Here are just a few:
See Harold Berlak's Handbook for more information about why using test scores to evaluate student achievement is not a good idea.
What is the Alternative to Using Test Scores to Measure Achievement?
One can start thinking about how to develop alternatives by looking at Debbie Meier's Six Alternative Assumptions to the high-stakes testing agenda. Meier expands upon these assumptions in her book, In Schools We Trust. The real question today, a question provoked by the Business Roundtable's successful implementation of high-stakes testing, is how can parents, teachers and other local community members work together so they begin to trust each other enough to have honest, deep and ongoing conversations about educational goals, assessment, pedagogy, curriculum, standards, governance, and so forth? If the members of a school's community cannot develop alternative vision to high-stakes testing, then our public school system will continue on its current path of punishing "low-performing" schools and allowing "high-performing" schools to produce task-completers rather than problem-solvers. This distinction is addressed in Don Arnstine's Democracy and Arts of Schooling in which he distinguishes between education and socialization. He argues that it is crucial for any possibility of democracy that we do both.
SEE alternative reform models in SF and the Bay Area.