Universiti Malaya's (UM) poor performance inevitably came under scrutiny at a seminar to discuss the World University Rankings 2005, with the maze of disclosures meshing into a thinly-veiled attempt at damage limitation.

Beleaguered UM vice-chancellor Prof Dr Hashim Yaacob used the event as a platform to defend the university's performance in the Times Higher Education Supplement (THES) World University Rankings 2005.

At the same time, Nunzio Quacquarelli, the managing director of QS Quacquarelli Symonds - which designed and conducted the THES survey - also conceded an error in data collection last year.

This had resulted in UM being placed higher than warranted - therefore implying that the drop this year was not as severe as it looks.

The university plunged from 89th spot to 169th position this year, while Universiti Sains Malaysia dropped off the list of top 200 universities after being ranked 111 last year.

About 400 management personnel and academics from government and private higher education institutions attended the seminar organised by UM at its campus yesterday, including the university's Pro-Chancellors Raja Nazrin Shah and Dr Aishah Ong.

Hashim, who chaired the event, insisted that the university had not only improved over last year's performance, but also performed better than many prominent universities in specific fields of study.

He said UM's score of 23.5 this year is more than the 16.6 it obtained last year, while it was ranked among the world's best 100 in the categories of arts and humanities (45), social sciences (83), and biomedicine (82). UM was not listed in any of these last year.

"The position depends on (how we and others have performed). It is not within our control. What is important is that each year we show an improvement in the marks - and this is something we can control," he said.

Undeserved humiliation

Hashim said UM's performance had improved in terms of three criteria - peer review, staff-student ratio and citations.

The drop registered against two other criteria - international student intake and staff - was attributed to UM being a national university that has to consider the interest of the Malaysian workforce and students.

(The sixth criterion, 'employer satisfaction' was only introduced for the 2005 ranking.)

UM's performance is close to that of the University of Newcastle in Australia (ranked 127, with a mark of 26.4) and the University of Frankfurt in Germany (142; 25.3), said Hashim.

"UM is only a little bit lower...Malaysia is still a new country," he said, also rebuffing critics who pointed a finger at him for the plunge in ranking

"We do not deserve to be humiliated and looked down upon because we have done very well and we have proved it," he added.

Copies of an article written by Hashim, entitled UM in the International Rankings: the True State of Affairs, were distributed during the seminar. The New Straits Times and Utusan Malaysia published it as an advertorial on Sunday.

Limitations of survey

Nunzio (right) explained that the UM was wrongly ranked last year because the survey had treated all Chinese and Indians as foreign students. This led to high marks being awarded for the 'international students' criterion.

He said this was due to an error in communication and that the mistake had been corrected this year.

No one in the audience asked what UM's real position would have been, if the mistake had not occurred.

Nunzio also pointed out the ranking is not as important as the ways in which universities make use of the data collected.

"There is no indication that a university ranked high is better than a university ranked lower," he said.

He praised UM's performance in the 'peer review' criterion, in which UM had advanced 41 places compared to last year.

Explaining the methodology used, Nunzio provided the breakdown of marks for the six criteria: peer review (40 percent), citations (20 percent), staff-student ration (20 percent), employer opinion (10 percent), number of foreign students (5 percent), and number of foreign staff (5 percent).

A total of 560 universities which attained the highest citations score were selected for ranking and the 200 best were shortlisted and assigned final placings.

Equal numbers of academics and employers from Asia-Pacific, America, Europe, Middle East and Africa were included in the survey, with academics only providing opinions in their areas of expertise to avoid possible bias.

Nunzio conceded that the survey has limitations. It cannot cover the teaching quality, student satisfaction and university investment in infrastructure.

He said journals publicised in languages other that English were neglected in the survey, which only considered those written in English over the past 10 years as recorded in Thompson ISI.

Nunzio advised universities to improve its performance in three areas - to produce research of higher quality; ensure that graduates are attractive to international employers; and to internationalise the university in order to secure a higher position in the ranking.

Questions ignored

Several members of the audience pointed out the limitations and inaccuracies of the survey. Some even doubted the independence of the survey as they quizzed Nunzio on the profits made and how many members of his staff are academics. However, he ignored these questions.

A few described the survey as "comparing apples to oranges", because the survey compares universities of different sizes and with various functions.

A warning was also sounded against campus administrators getting caught up in "a beauty contest of universities" and changing policies according to the ranking, rather than objective criteria.

Nunzio, however, defended the survey by saying that he had used all available and comparable data that could be obtained from about 30,000 universities.

Asked for details of the methodology, he said this would be published in January. As to how the percentage for each criterion was determined, Nunzio said his partner had decided this.

The survey is in its second year. Next year, the ranking format will be changed to a set of data that will enable the public to arrive at their own rankings based on their own criteria, he added.