Understanding University Rankings

University rankings have become increasingly important in terms of the power they wield throughout the world. Governments are using them to determine how education budgets are dispersed, parents and students are using them to make decisions about where they should study, and the corporate sector are increasingly using them to determine the quality of candidates’ education. But are they a valid and reliable set of data for doing so?

It is often very confusing for students and parents to make sense of university rankings. A brief analysis of the rankings and their comparison raises some interesting questions. Why are they so different in their results? How can a university rise or fall so many places in just one year? These are all very legitimate questions, and ones that we will endeavour to briefly explain in this blog. The aim of this article is to help you use rankings more sensibly, or… perhaps ignore them altogether. 

There are so many rankings systems these days, but for the purposes of this blog, we will look at the three most popular:

1. The Academic Rankings of World Universities (ARWU)

2. Times Higher Education World University Rankings (THE)

3. QS Times World University Rankings (QS)

Some points to begin with:

1. Universities are very complex organisations. They typically have between 15,000 - 50,000+ students, and 3,000 - 15,000 staff. They house vastly different faculty’s and departments, and diverse research centres and institutes. They have complex partnerships and affiliations with industrial, governmental and medical organisations, and their focus is vastly different according to their context. To expect that you could create a uniform instrument that could adequately and successfully measure this complexity is far fetched for many critics of the rankings systems. 

2. Wealthy and old universities rule the rankings. With hundreds of years of brand development, history, and wealth, the older established universities are significantly advantaged by the rankings. This is largely for two main reasons. Firstly, many of the rankings have a reputation  survey that constitutes a large percentage of the methodology. For the well known universities with some history, this can mask actual performance. The very famous Sorbonne of Paris is ranked 214 in the QS rankings for instance, but doesn’t make the top 500 in the ARWU. This is because the QS rankings place more emphasis on reputation than actual performance. Secondly, the university philanthropic recipients with billions in endowment funds can buy a lot of prized researchers, can fund countless research projects, and can basically do much more with much more. This doesn’t necessarily translate into a better experience for an undergraduate though. 

3. The rankings systems are packed full of bias. If you speak English; if your university is in loads of American feature films per year; if your university is predominantly science rather than humanities; if you are lucky enough or wealthy enough to land a Nobel Prize winner; and if some super wealthy oil tycoon alumnus from Texas leaves you an endowment fund of a billion dollars. Again, all this bias significantly affects the rankings, but doesn’t necessarily translate into outcomes that will positively affect your education or experience there. 

4. Rankings don’t measure the social outcomes that a university delivers to the community for which they serve. Many universities train thousands of teachers and nurses, or social and community workers each year. They contribute vastly to their communities through their pedagogy research and teaching, and public health. However, their research is unlikely to make it into the journal Nature and Science, or the Lancet. They don’t have a Nobel Prize category for teaching or social work (although they should perhaps consider it). All this means it is unlikely that these universities appear as high up the rankings as they should had another methodology been applied. These may be the most appropriate places for you to study your chosen path in life though. 

When you use the rankings, its important that you do so with a clear understanding of their methodology. We have included a break down on each of the data points that were used to determine the rankings. You can also following the links for the complete explanation of the methods used. We suggest you look at these with a very critical eye. It may help you use the rankings more objectively. 

Acadmic Ranking of World Universities (ARWU) Methodology

The ARWU is largely considered to be the most consistent and defensible ranking system. Unlike the other popular rankings systems that rely largely on an often poorly responded repetitional survey, the ARWU uses hard data. It is awkwardly weighted towards big scientific discoveries, and places a lot of emphasis on Nobel prizes and highly cited research that tends to favour the sciences, although it also has a 20% weighting towards indexed research output. Saksara prefers this ranking system over others due to its focus on actual outputs and the weighting of those outputs according to the size of the university. You can explore the complete details of the methodology here. 

The QS rankings are the most reliant on repetitional surveys. A massive 40% is dedicated to a survey that they send out to academics all over the world. On average only around 20,000 responses are received annually, which accounts for only 2 staff from each university throughout the world. Likewise, around 27,000 industry responses were received, for the employer reputation survey globally. Saksara generally feels as though the weighting of 50% based soley on these dubious repetitional surveys is bad science, and probably wouldn't make it past a preliminary thesis defence at any of its top ranked universities. On the positive side, we think that faculty staff to student ratio and the proportion of international students and staff are good things to measure. Overall, we think you should only look through the corner of your eye at the QS rankings. 

The main issue with the THE rankings for Saksara is the extent to which they change their methodology so regularly. They are perhaps the most widely cited rankings system and have often come under much criticism for their methods. They have responded to this criticism with annual amendments to their methodology and subsequently vast fluctuations in their results. Although the weighting of 30% to research citations is a positive point, it also has a disproportionately large emphasis on their reputation survey, which only received around 10,000 responses in the last survey. Again, its worth noting there are around 10,000 universities throughout the world. Its worth looking at this methodology very carefully to see the precise formulation of the score, and the data that supports it. Saksara likewise believes this would be unlikely to make the grade for a Masters dissertation. 

It’s difficult not to be skeptical about the ranking of universities. On the other hand, it’s good that the discovery and dissemination of knowledge for the advancement of people, communities and the planet is getting global attention. It is unlikely that the position of the top 50 universities will change much over the next 50 years. Universities all have something valuable and meaningful to contribute to society. The top 5-10% of world universities all have something exceptional to contribute. This is 500-1000 of the top universities around the world. Whilst rankings are here to stay, Saksara believes that a more holistic approach to determining your study destination should be pursued. 

There is no doubt that a place at MIT, Harvard, Oxford, Cambridge etc will be a life changing experience and open many doors. The truth is that most candidates are unlikely to find a place in these institutions. There are great universities that are accessible to most internationally bound students. Finding the right place for you will require some dedicated investigation and some good advice from professional student advisors. We hope that this brief look at the rankings will help you look more critically and less adoring at them. Perhaps ignoring them altogether is not such a bad thing either. They are perhaps more beauty pageant than beauty. 

If you would like to know more about another way to approach your future university studies, you can make an appointment to discuss options with Saksar’s expert advising team here: