Dog Eats AFT Homework
A teachers union's dishonest study of charter schools.
BY WILLIAM G. HOWELL, PAUL E. PETERSON AND MARTIN R. WEST
Wednesday, August 18, 2004 12:01 a.m.
It is not unusual for interest groups to issue misleading reports that
further their political agenda. And for this reason, newspapers
generally ignore them, treat them with great skepticism, or make sure
they vet the study with independent observers.
Not so in the case of the recently released study of charter schools
issued by the American Federation of Teachers, which, after receiving
top billing in the right-hand corner of the front page of yesterday's
New York Times, was picked up by news media across the country.
According to the Times, the AFT had unearthed an apparent coverup by the
Department of Education, which had buried key findings in "mountains of
data . . . released without public announcement." The department, it
seems, is taking a long time to issue its report on charter schooling in
America.
So the AFT took matters into its own hands, attributing its success in
conducting the study to "a combination of intuition, prior knowledge,
considerable digging, and luck." Perhaps. But within a few hours of its
release, we were able to replicate its results--and conduct a similar
"study" of religious schools besides.
The AFT's conclusion: "Charter schools are underperforming."
Data from the National Assessment of Educational Progress (NAEP), often
called the nation's report card, show students in charter schools doing
less well than the nationwide public-school average, which includes
middle class students from well-heeled suburbs. Similar results are
obtained within selected states.
Big deal. These results could easily indicate nothing other than the
simple fact that charter schools are typically asked to serve
problematic students in low-performing districts with many poor,
minority children.
Indeed, if the AFT believes these findings, it must also concede that
religious schools excel. According to the same NAEP data from which the
AFT study is taken, religious schools outperformed the public schools
nationwide by nine points, a gap that is as large as the public
school-charter school difference AFT is trumpeting.
On other occasions, the AFT has objected to interpreting such findings
as evidence that religious schools are superior, on the grounds that
they attract an especially able group of students. But for charter
schools, apparently, similar student differences are less important.
"To enhance the fairness of the analysis," the AFT study makes
comparisons among students eligible for free lunch and in various kinds
of communities, which again shows public school students doing better
than those in charter schools. But if these simple comparisons prove the
AFT case, then they also prove that religious schools are better than
public ones--for within these same categories, religious schools
outperform public ones.
Indeed, the AFT's most telling comparisons-- the ones within ethnic
groups--cut against the case it is trying to make. This comparison is
vital, precisely because prior research has found ethnic differences to
be large. Yet when the authors look just at African-American or Hispanic
children, they find no statistically significant difference between
public school students and those in charter schools.
But do any of these findings--within ethnic groups or otherwise--say
anything meaningful about the quality of charter schools? Not a bit. For
starters, one must do much more than look separately at students grouped
by free lunch status, ethnicity or school location, in order to take
into account family influences on a child's learning capacity. All of
these factors--and many other considerations--must be combined into a
sophisticated analysis in order to begin to gauge how well students
perform.
What makes such a task essential is the simple fact that charter schools
are usually placed in challenging situations. Most states allow charter
schools to form only where students are having difficulties, and charter
schools are, in many cases, then asked to accept the most challenging of
students. Any credible analysis of their effectiveness must account for
these facts on the ground.
But this just touches the surface. The AFT study only looks at student
performance at a single moment in time. One needs to track student
progress within a school over multiple years in order to ascertain how
much the child is learning. Moreover, nothing in these data accounts for
the length of time that a charter school has been in place--a factor
known to have an impact on a school's performance. First-year schools
usually have difficulties. Having just hired new staff and teachers,
implemented new curricula, and acquired a building facility to use, new
schools often face considerable start-up problems. Almost one-third of
the charter schools nationwide were less than two years old when the
NAEP was administered, raising doubts about whether even a sophisticated
analysis of NAEP data would be relevant once charter schools have had
time to become well established.
According to AFT official Bella Rosenberg, "Analyses are always welcome,
but first things first. . . . Surely the interests of children are
better served by timely and straightforward information about whether
charter school performance measures up to the claims made for it."
Of course timeliness is important, but bad information is worse than
none. To know whether charter schools are doing better, careful analyses
are essential. For all of the reasons outlined above, the Department of
Education is well advised to prepare its report on charter schools
carefully, taking as much information into account as possible. If this
explains the official report's delay, this can hardly be called a
coverup.
The limited information currently available prevents anyone, including
the AFT, from taking even the most modest steps toward addressing these
issues. In short, the AFT's report tells us hardly anything about the
relative effectiveness of charter schools. But one thing is sure:
Charter schools do not appear to be bastions of privilege. What remains
unclear is how much they can do for the underprivileged. Sadly, the AFT
report tells us nothing about that.
Messrs. Howell, Peterson and West are researchers in the Program on
Education Policy and Governance at Harvard. Mr. Peterson is also a
senior fellow at the Hoover Institution and a member of the Koret Task
Force on K-12 Education.
Copyright (c) 2004 Dow Jones & Company, Inc. All Rights Reserved.
Mike
(Attachment image001.gif is missing)
(Attachment image002.gif is missing)