Are College Rankings Based on Circular Logic?

When students are making the all-important decision of where to apply to college, they often turn to U.S. News and World Report’s rankings of colleges and universities. According to one study, more than half of incoming college freshmen surveyed said the rankings were important when making their decision about which school to attend.

But a new study calls into question the methodology used to rank the schools. Specifically, it examines the “reputational” survey that makes up 25 percent of the score that the magazine uses for its rankings. The research shows that reputational scores are based on the previous year’s rankings, and rankings, in turn, are based on the reputational scores. In other words, the scores are circular.

The study, published in the American Journal of Education, was conducted by Michael N. Bastedo, an associate professor of education at the Center for the Study of Higher and Postsecondary Education at the University of Michigan, and Nicholas A. Bowman, a postdoctoral research associate in the Center for Social Concerns at the University of Notre Dame.

In an interview with Inside Higher Ed, Bastedo said that the study raises serious questions about the validity of the reputational survey. The findings suggest that reputational rankings won’t change, even as the quality of colleges does change. But Robert Morse, director of data research for U.S. News, says it’s not surprising that reputational scores are “relatively stable from one year to another,” adding that “schools themselves say they change slowly, not rapidly.”