VAM: A meaningless, dangerous number

By Warren Fletcher
UTLA President

I have been an English teacher in LAUSD for many years. I have always understood that my primary duty and loyalty must be to my students, first and last. Teaching is a profession; it’s not merely a job. At a very deep level, we all understand this. And the public understands this too. Parents and the community don’t expect that we will robotically “deliver” instruction to students. They expect that we will passionately advocate for their children and that we will speak up when the District tries to implement policies that will degrade the quality of instruction.

Like what is happening right now.

Superintendent John Deasy has decided to implement a new system of teacher evaluation in a pilot program throughout LAUSD and to do so without negotiating it with UTLA. This is illegal, and UTLA has filed an Unfair Labor Practice charge against the District to stop it. Last month, the Public Employment Relations Board issued a complaint on the matter, the first step toward making the District obey the law.
Deasy’s system—Academic Growth Over Time (AGT)—bears a striking resemblance to the Value- Added Measurements (VAM) being pushed across the country. In both cases, teacher evaluations are tied directly to annual student standardized test scores. The long-term goal of VAM/AGT is to eventually tie teacher pay and teacher retention directly to those scores.

Individual VAM/AGT evaluation degrades instruction

As educators, we all understand how a narrow and punitive system of numerical evaluation like VAM/ AGT can seriously degrade instruction and, as a consequence, hurt students.

I mentioned earlier that I’m an English teacher. I know that if my pay, or even my continuance in the profession, were to be tied directly to my students’ annual CST scores, I would be under intense pressure to spend less time teaching writing and teaching literature; to spend less time teaching my students how to clearly express themselves and to understand complex written material; to spend less time teaching the actual skills that my students need and that their parents want them to develop for college or careers. In a VAM/AGT environment, there is constant pressure to teach less and to reduce instruction to test prep. And that kind of pressure would apply to every teacher, regardless of grade level or subject field.

Many of the people (including elected leaders) who push the hardest to link teacher evaluation directly to raw test score numbers are well intended. They believe that directly linking test scores to teacher employment decisions will transform all schools for the better. This belief is based on two obvious fallacies: first, that test scores are a direct and accurate (rather than approximate) measure of “student achievement”; and second, that attaching test score incentives and threats to teachers’ jobs will somehow “revolutionize” how teachers approach their teaching. People who believe these things are not evil—they’re just wrong. And it’s our job as professional educators to set the record straight.

Individual teacher VAM/AGT scores are virtually meaningless

The U.S. Department of Education is headed by Arne Duncan. Duncan is a strong proponent of VAM/ AGT-type teacher evaluations. But Duncan’s own department has released studies showing that VAM evaluations of individual teachers consistently have an error rate of more than 25%. In other words, one-fourth of all individual teacher VAM scores will be flat-out wrong. That’s why the U.S. Department of Education recommends against using VAM scores for teacher evaluation decisions. Respected researchers from all over the country have rejected their use and have warned of their unreliability. An individual teacher’s VAM/AGT score, whether it was calculated by the District or by the L.A. Times, is virtually meaningless as an indicator of effectiveness or competence.

And speaking of the L.A. Times, it appears that they may be planning another round of articles on VAM, once again accompanied by a searchable database of individual teacher VAM scores, based on data from the District. While UTLA will vigorously pursue all legal channels to keep the District from releasing teacher-level raw data, it is important to remember that any individual teacher score that the Times may calculate and post is, by definition, a VAM score, and thus is little more than a random and meaningless number determined through a formula that has been widely discredited.

While the obvious purpose of the last L.A. Times VAM series was the public belittling of public school teachers, it is important to remember that your or my individual “VAM number” is about as reliable an indicator of teaching quality as your or my driver’s license number. When we forget that, we empower the people who want to disparage our profession.

It doesn’t matter whether the proponents of VAM are wealthy or powerful; they are still wrong, and they are pushing a teacher evaluation system that is at best meaningless and at worst dangerous to instruction. We have a duty to our profession and to our students to resist its implementation. When any teacher starts to feel inadequate (or for that matter superior) because of something as meaningless as an individual VAM score, the enemies of our profession score a small victory.

The advocates of individual VAM score evaluation all have one thing in common: They don’t know the first thing about quality teaching. You do.

Never forget that.