It’s time to take a careful look at the whole process of annual school rankings. Reporting rankings based on data that doesn’t consider the makeup of the student base or prior results doesn’t tell the correct or full story.
It usually happens in the spring – newspapers across states and provinces publish the annual ranking of elementary schools, middle schools and high schools in that jurisdiction. Educators and parents read the tables with great interest to find their schools and see how they fared – up, down or unchanged from the prior year. Well-meaning third parties often perform the actual rankings as a service, and they comment on the macro trends that the data suggest. Equally well-meaning writers opine on what the rankings say about local schools and the school systems. Parents wring their hands or rub them together because school rankings and home values are often tightly aligned. Teachers are praised or criticized for their school’s placement.
And, almost all of this reaction is meaningless.
Measuring a school’s success
While it may be easy to rate and rank a school population against all others in a jurisdiction based on the average of its test scores compared to other schools or to compare a percentage of graduating students who go on to postsecondary education, this ranking is of almost no value. The only true measure of a school’s success is its performance against its own prior results. Is the individual school improving, worsening or standing still relative to its performance last year?
And here is the simple reason why this is the case – schools, with some notable exceptions, serve a population base that remains largely the same from year to year. (Over several years, a school’s population may change quite significantly, but that change usually does not happen overnight.) For schools where students are underperforming expectations, continuous improvement in results must be the unwavering target, while the maintenance of the status quo could be a good outcome in a high-performing school.
Impact of socioeconomic status
Many know that academic achievement is highly correlated to socioeconomic status. On average, children from more advantaged families do better on standardized tests than children in poor families. So the high performance of students in a private school that draws its students from affluent families with university-educated parents who value education and are actively engaged with their children’s education is expected and is not a particularly notable fact. What may be far more laudable is the school in an area with a predominance of children who qualify for free or reduced lunch that improves its performance from one year to the next.
Taking a detailed look
A much more detailed look at the existing data and the availability of different data are needed to fully appreciate the progress that is being made, or not made, within a school. There is nuance in the data of a student population that must be understood by teachers, administrators, politicians, parents and community members that a simple ranking by test scores obscures. Is a significant percentage of the student body English as a Second Language (ESL) learners or special needs students? Have students been in the reporting school for some time or are they new?
Compensation system
As more jurisdictions look at tying compensation to teacher performance (which translates to student outcomes), accurately understanding performance is important. Think about it – if a compensation system seems to be stacked against teachers being rated as doing well, then what teacher will want to take the compensation risk of going to underperforming schools? No one, not even the most highly, intrinsically motivated teachers, is the simple answer.
Improvement year to year
In a more data-driven world where standings matter and seem to gain a life of their own, the matter of school rankings is not trivial. It’s not about splitting hairs – it’s about getting people to understand the job to be done and the actual results achieved. The name of the game is improvement from year to year, not standing relative to other schools.
The continued, simplistic evaluation and ranking of schools needs to change. As with most things, this starts with the education of politicians, reporters, parents and the larger community as to what the data mean. It also means the communication of improvement metrics for the measured schools so that there is a clear picture of teacher impact. Praise the gains achieved and work to resolve the gaps that remain. Makes sense to me and to many.
