Shock and horror as the THE World Top Ten Universities 2022 are revealed as….exactly the same ten names as 2021. A small shuffle of the deck saw Stanford drop from second to fourth but The Stanford Daily seemed more concerned with the question Who Is Elizabeth Holmes, The Stanford Dropout Now on Trial? As someone probably once said – if you’re truly world class you don’t say it and you certainly don’t need the THE to tell you.
LinkedIn was full of university marketing chiefs and even some academics, who should probably know better, trumpeting their performance. Newcastle University’s marketers expressed pride that it had moved into the top 150 but it had simply returned to 146th – exactly the position it occupied in the 2011-12 rankings. There were plenty of other institutions with short term memories talking without any regard for whether their ranking meant real, sustainable or even meaningful progress.
It’s a merry go round that was called out recently by Vincenzo Raimo who noted that universities tend to celebrate advances but complain about the distortion and negative impact of the rankings. When leading academics do call into questions the methodology, as David Price, UCL’s Vice Provost of Research did recently, they get snide responses from the promoter in chief. Perhaps the THE is becoming The Borg and thinks that “resistance is futile”.
What the THE has certainly seen is that university compliance and hypocrisy has enabled them to exploit the “trusted rankings” as a platform for THE Student. To the mix they add a spiel about “hand-picked partners” who will help student “make the right choices”. A cynic might suggest that the many privately financed partners on the list are much more likely to ensure a result which is in their own interests.
But It May Be Worse Than That
It would seem harmless to simply accept that the World Rankings have become a university version of the Sunday Times Rich List where envious glances are occasionally followed by spectacular falls from grace. Maybe The Stanford Daily is offering a metaphor by focusing on a cautionary tale of hubris and deceit just as these rankings were published. But the THE doesn’t appear to be in any doubt about the game it is playing.
They say that “even if you do not meet the inclusion criteria, you will be entitled to a university profile on our website that will increase your visibility to our audience of academics, prospective students and their parents.” It is a university version from the “Toxic Sludge is Good for You” playbook which Publisher’s Weekly called “a cautionary reminder that much of the consumer and political world is created by for-hire mouthpieces in expensive neckties.”. Even the most limited institution, regardless of reputation or quality, can benefit from reflected glory as part of this commercial enterprise.
The THE sells the benefits of the rankings very hard and articulates them as global exposure with tens of millions of page views, data trusted by governments and universities, and a vital resource for students when they are making decisions about where to study. The point about ‘trusted by governments’ is a big part of the sales patter including a recent Tweet which highlighted the EU Commission’s, Gerard de Graaf saying, “We know that rankings do more to direct universities’ attention, policy makers’ attention, students’ attention than any other policy tool… “.
Surprising then that in 2014, the very same year of de Graaf’s comments, the European Commission gave €2m funding to establish U-Multirank explicitly, “to avoid simplistic league tables which can result in misleading comparisons between institutions of very different types”. Dr. Simon Marginson, Professor of International Higher Education, UCL Institute of Education, University College, London called U-Multirank, “a vital corrective to the “football” league mentality that has crept into higher education…”. The point is that the EU did not see ‘rankings’ as the answer to anyone’s problems or need for better quality information.
Gaming The System
The tweet also claims that de Graaf “urged@timeshighered to develop rankings on impact” which they framed around the UN’s SDGs and first published in 2019. To be included in the overall ranking an institution has to self-select and submit data on SDG 17 and at least three other SDGs of its choice. It’s difficult to see, however, that an institution can’t selectively manage its performance in three SDGs and SDG 17 while being a mediocre or even poor actor in the other thirteen.
The University of Manchester’s top spot in the 2021 Impact Rankings suggests how partial this process can be and why students looking for insights might do well to look elsewhere. An alternative might be the People and Planet UK-based student network that has been running an environmental and ethical performance league tables since 2007. The organisation also does useful things like training and mentoring young people, campaigning and challenging vested interests locally and internationally.
Its 2019 League Table gave the University of Manchester a low-ranking in the Upper Second-Class Honours bracket and 59th in the UK. To be totally fair it also notes that the University has fully completed a commitment to divest from all fossil fuels. It is arguable that the THE rankings give too much opportunity for institutions to game the system and, as a Professor of History in a 5* department once said to me, “we are all here because we are good at passing tests”.
If the principle is that the THE Impact Rankings are a “vital resource” for students wanting to make a choice they might do well to consider giving a broader context. Students travel internationally to share in a cultural experience and could easily find that selecting a university based on the Impact rankings leads them to places where the off-campus setting is a little less in tune with their sensibilities. It’s not necessarily that the universities aren’t trying hard but there are very real limits to their power.
The country with the largest representation in the Impact rankings is Russia with 75 institutions which seems counter-intuitive given that the country is only 46 of 165 in the UN’s own SDG rankings. In early 2020 Transparency International ranked the country 137th out of 180 in its Corruption Perceptions Index at a point when the Russian Academy of Sciences was reported as finding “widespread plagiarism in Russian academic journals, with more than 850 articles rescinded from 263 journals after an initial review.” More concerning is the repression, sexual harassment and intimidation of students and faculty outlined by the Russian student magazine DOXA.
At 27 in the Impact rankings is Princess Nourah bint Abdulrahman University in Saudi Arabia – a country down at 98 in the UN rankings. The university scores well on SDG 10 for reducing inequalities at a point when the UN does not appear to have information available to give the country a score. Meanwhile Finland, which is top of the UN league table, doesn’t have a university ranked until the 201-300 bracket by the THE.
When Gaming Becomes Cheating
League table manipulation is a theme that Malcolm Gladwell picked up in his Revisionist History podcast series. Calling the U.S. News & World Report college rankings an “abomination” might sound harsh but his analysis points to the way the rankings can distort perceptions of higher education. The edition on Project Dillard focuses on the specifics of how a historically black university in New Orleans is disadvantaged “even though, on a number of very objective measures, it does an outstanding job of educating the students who go there.”
His argument is that, fundamentally, the league table gives no encouragement for small and rich colleges to use their advantaged position to serve larger numbers of students. The corollary is that Dillard University could leap sixty places up the US News rankings by cutting 75% of its students. All of this is before the various scandals of colleges manipulating data to improve their place in the US News rankings.
In this vein the THE Impact rankings have a corrections page where any errors in data collection and changes to rank as a result are listed. The notable thing about this is that every case where incorrect or incomplete data was submitted the university’s ranking has either not changed or they have gone up the table. It’s a relatively small sample but one might imagine that institutions are keen to, legitimately, correct the data when they feel they have done poorly but less likely to review data when rankings have gone well.
Earlier this year a report by the Center for Studies in Higher Education produced an analysis suggesting the QS World Rankings had a conflict of interest due to its consulting business. QS responded that the consulting contract with the university stipulated that there was no link to rankings and that they had policies to ensure staff were “free from personal or commercial bias”. Readers will make up their own minds but as league tables become increasingly commercially exploited the risks becomes greater.
If Resistance Is Futile…Consider Changing the Rules
Nobody should kid themself that league tables have not had a material impact on decision making within universities. Hours, days and weeks of planning and strategy have been exhausted on understanding the levers that can be pulled to move institutions up various rankings and this effort would not be made unless it fed into actions. The available tools are relatively blunt but increasing the number of ‘good degrees’ always looked manipulable and it is arguable that the 90% growth in first class degrees awarded in the UK between 2010/11 and 2018/19 is one visible sign of that pressure.
But Forbes tells us some interesting things about “no win scenarios and ethical leadership” and draws on Star Trek’s Kobayashi Maru scenario as its exemplar. Famously, Captain James T. Kirk overcame the no-win training scenario by reprogramming the simulation and has led a fierce debate over whether he cheated or was simply creative. Author Janet D. Stemwedel cuts through this by suggesting “it’s important to be able to deal with trying to live up to our ethical obligations while knowing full well that circumstances and our own limitation cannot guarantee we’ll succeed.”
University league tables won’t go away and universities may feel obliged to play the game because of the political, social and recruitment leverage they might offer. However, academics do not have to join in by offering their opinions about other universities and institutions do not need to manipulate their decision making with one eye on the league table impact. There could also be more concerted pushback against the dumbing down that emphasizes overall rankings and oases of excellence in a sea of mediocrity or even corruption. If the aim is to help students faced with the biggest decision of their lives it’s worth the effort.
Notes
The complexity of league table methodology is the stuff of legend but it does not really aid understanding. The commentary on the THE approach to the overall SDG table reflects my understanding of the paragraph – A university’s final score in the overall table is calculated by combining its score in SDG 17 with its top three scores out of the remaining 16 SDGs. SDG 17 accounts for 22 per cent of the overall score, while the other SDGs each carry a weight of 26 per cent. This means that different universities are scored based on a different set of SDGs, depending on their focus.
As always I am happy to review authoritative comment which may aid understanding and will reflect this in an update if necessary.
Image by OpenClipart-Vectors from Pixabay