Politics & Research - Not Perfect Together

by Raymond R. Arons, Dr. P.H., M.P.H.
(Dr. Arons is Assistant Professor of Sociomedical Sciences, Mailman School of Public Health, Columbia University.)

The Mathematica Case Study

In politics, an agenda is decided first, and how to accomplish it is decided second. In science, no agenda is decided first. A question is asked without preconceived answers. Strict methods are then used to get information about the question. Analysis is made. And finally, recommendations are developed based upon the findings. That did not happen in a study commissioned by the Bush administration of Early Head Start programs and conducted by Mathematica Policy Research, Inc., located in Princeton, New Jersey, entitled Early Head Start Research and Evaluation Project-Leading the Way: Characteristics and Early Experiences of Selected Early Head Start Programs.

Mathematica was asked to evaluate the implementation of Early Head Start (EHS) programs in America, or those specifically addressing the needs of infants and toddlers, ages 0-3. This was a multi-million dollar study that collected extensive data on 17 different EHS programs over a five-year period. Throughout its interim and final report, many references are made to "Part C" programs and services. Yet, these are never designated as special education, and few questions were asked about special services for this population. In fact, the report suggests specific efforts to avoid collecting this data even though special education cannot be separated from Early Head Start programs. Particularly disturbing was the timing of the final report in June of 2002, or during the hearings and testimony on IDEA reauthorization. No one appeared to question the Mathematica report except the Melody Arons Center. We reviewed it, never expecting to be as distressed as we became, anticipating the paper to have the quality of any other genuinely scientific publication. It did not. It was a monograph that, in my opinion, would never pass muster in any scientific journal or community. Its techniques and statistical methods were flawed and highly questionable. It never submitted its findings for peer review before holding a press conference. When a request was made for public access to the original data used by its researchers in order to replicate the findings, no response was received.

The credibility, manipulation, selectivity and censoring of findings to achieve a political goal is a central issue in the Mathematica study. It attempts to prove the worth of Early Head Start at the expense of accuracy. This is inexcusable, particularly with a leading research center, and raises a legitimate argument that research is precisely the opposite of politics. The first searches for truth. The other seeks only to control.

The current IDEA recommendations before Congress emphasize the creation of scientific rigor and peer review among and between various federal education and health agencies and experts. If the multi-million dollar Mathematica study on Early Head Start is an example of what the Bush administration envisions, we are all in trouble. There is clear and present danger in merging scientific inquiry with the political needs to prove the efficacy of a program. No one can argue that early intervention is essential to improve the outcome for disabled infants and toddlers. But to bury early intervention questions relative to Part C within the umbrella of Early Head Start does a disservice to both. Politics and Research are not perfect together.