THE WUR 2024: New Methodology, New Questions

The composition of the Top-100 usually remains very stable. The US still has the most entries (36) in this tier, although the number has been decreasing year after year; in 2010, for example, the US had 53 universities in the Top-100.

When we look at a bigger range — the Top-500 — there are many more changes. On average, universities that were in the Top-500 in THE WUR 2023 (published in 2022) have moved down in THE WUR 2024 (published this year) by more than 25 positions. In contrast, universities that had been in the Top-500 in THE WUR 2022 (published in 2021) lost about 10 positions in THE WUR 2023.Front matter, or preliminaries, is the first section of a book and is usually the smallest section in terms of the number of pages. Each page is counted, but no folio or page number is expressed or printed, on either display pages or blank pages.

In terms of country representation, here are those which lost the most positions in the Top-500: Iran (–6), the UK (–5), and India (–4) (Table 1). Saudi Arabia lost two positions in this tier, while three countries — Greece, Nigeria, and Vietnam — also lost two positions each and completely dropped out from the Top-500. China and Japan, on the contrary, were able to boost their presence in the Top-500 with four more entries each; Canada, Malaysia, South Korea, and Taiwan (China) have three new entries each; France, Switzerland, and the US — two more each.

We have looked at THE WUR 2024 through the prism of five top-level indicators — which have considerably changed this year. How has the new methodology affected the positions of countries and certain universities? And are there any peculiar results in this year’s list?

A commonly recognised weak point of THE WUR (before the changes in the methodology) was the small expert base. Just two years ago, only around 10,000 authors with publications in Scopus used to participate in the annual survey. For comparison, QS has been surveying over 100,000 experts for a long time — but some of these respondents may be HE managers, not necessarily researchers. It is an issue for QS; however, THE was also subjected to criticism, for example for having only 100 experts to judge the reputation of all law schools in the world.

Now there are hopes to resolve skewness: almost 40,000 experts participated in the latest THE survey (four times more than before). The current rankings take into account survey results from the previous two years, and consequently the expert base has grown to more than 68,000 people.

Let’s now take a closer look at the lists which appear when THE WUR 2024 are filtered by individual indicators — Teaching, Research Environment, Research Quality, Industry, and International Outlook.

On average, universities that were in the Top-500 by Teaching in THE WUR 2023 (published in 2022) have moved down in THE WUR 2024 (published now) by about 20 positions. In contrast, universities that had been in the Top-500 by Teaching in THE WUR 2022 (published in 2021) lost about 10 positions in THE WUR 2023.

As for countries, the US has now lost the most universities (–7) in the Top-500 by Teaching, while India has gained the most in this tier (+8) (Table 2). Within the Top-100 by Teaching, Germany has lost the most positions (from seven to four) and Japan has gained the most (from four to seven).

Although the rankings by Research Environment (formerly Research) have gained some new entries and lost some participants compared with the previous year, we do not observe striking changes which could have been caused by the new methodology.

One of THE’s goals was to change the methodology so as to balance the effect of the indicator reflecting the quality of research. In 2022, this indicator was called Citations Impact — equal to Elsevier’s Field-Weighted Citation Impact (FWCI). In the new methodology, the weight of FWCI has been halved and three more new parameters have been introduced, forming the aggregated Research Quality indicator. In this light, although the rankings by Research Quality / Citations Impact (we will be using the new name throughout the rest of the text) are a little hard to compare year-over-year, this indicator aims to reflect the overall quality of science — and that is why the comparison seems valid.

Last year’s list of the Top-6 universities by Research Quality looked quite odd as it did not feature any of the “usual suspects” but included entries from Vietnam, Türkiye, Iran, and Ethiopia (Table 3). This year, these universities have moved far down the table. At the same time, the changes in the methodology have brought some of the world’s (probably) best-known universities to the top positions by Research Quality (Table 4). We look forward to seeing next year’s rankings to check whether the methodology continues to deliver similar results.

Just over a half (58) of the universities in the Top-100 by Research Quality that were there in 2022 have remained in this tier in 2023. Geographical representation, too, has experienced major changes; of the 35 countries in the Top-100 by Research Quality in 2022, 21 have dropped from the tier, while eight new have entered. Iran has lost the most universities (–8) but still kept one entry (Table 5). At the same time, Egypt and India have lost three universities each; Saudi Arabia and Vietnam have lost two each — making these four countries drop from the Top-100 altogether.

On the other hand, we can highlight some success stories. The Netherlands has strengthened its presence in the Top-100 by Research Quality from one to six universities; Hong Kong is now represented by four entries instead of one; and Switzerland has three universities in the Top-100 compared with none in 2022.

The changes in the Research Quality indicator were meant to reflect the quality of science at various universities in a more comprehensive way, and that is why the indicator now contains several metrics. We speculate that the changes in the methodology have resulted in a yet higher number of the entries from the US (35) and the UK (22) in this year’s Top-100 by Research Quality.

China has lost the most universities (–10) followed by Türkiye (–8) and Russia (–5) (Table 6). Iran, Saudi Arabia, and Taiwan (China) have lost two positions each. Only China and Taiwan (China) have been able to remain in the Top-100 by Industry while the rest of the mentioned countries have dropped out completely.

Just over a half (52) of the universities in the Top-100 by Industry that were there in 2022 have remained in this tier in 2023. At the same time, those universities that have dropped from the Top-100 by Industry have lost a lot of positions: 25 entries have moved down by more than 100 positions each, and 11 — by more than 500. The Top-5 “outsiders” from last year’s Top-100 have lost more than 650 positions each (Table 7). On the other hand, when we examine the universities which have gained the most positions, their Top-5 consists of entries that have jumped up from 1500+ by more than 1000 positions (Table 8).

In the case of the Industry indicator, the changes in the methodology are meant to improve the evaluation of how effectively universities interact with business and engage in technology transfer. This year’s results, calculated using the new approach, have shown radical shifts in the rankings — both at the university level and at the country level.

This top-level indicator has received a new component, Studying Abroad; however, the move has not affected this year’s rankings, because Studying Abroad has received zero weight (for now). Although the new metric will play a role next year, the methodology of the current rankings has also seen changes in the International Outlook indicator — which has now been normalised by population.

Among this year’s success stories are the UK which now occupies almost half (48) of the Top-100 positions by International Outlook (up from 38 positions last year), and the US which has returned to the Top-100 with two entries in 2023 after absence in 2022 and 2021.

We believe that it is better not to attract criticism or provoke universities to boycott the rankings. Greater transparency of the methodology would be beneficial; it would also help to publish the recalculated results of the previous year using the new methodology. In this case, it would be easier to see that some universities’ positions have changed sharply because of the new methodology — and not as a result of internal transformation.