Tag: studentchoice

SHORELIGHT REPORT AND STUDENT SAFETY – ILLUMINATION OR OBFUSCATION?

We are familiar with the notion that there are “lies, damned lies and statistics” so whenever an organisation throws up figures to paint a scenario that is in their interests it’s always worth taking a second look at the source data.  Students, parents and agents should particularly beware claims made primarily for marketing purposes when safety issues are at stake.  Caution is certainly a good approach to take with the recent collaboration between Shorelight and US News Global Education (USNGE) which includes a, so called, University Safety Index and league table of the safest States for international students.

There is no place in the world that is entirely free of potential trouble, and international students should be alert to both the joys and the potential troubles of studying overseas*.  The widespread rise in hate crimes around the world and specific incidents of racism are as concerning for the UK, Australia and Canada as the USA. Reputable universities work hard to make their campuses as safe as possible but the advice to incoming international students should be pragmatic rather than marketing gloss. 

There is no reason to believe that the numbers used are incorrect but the way the Index is constructed shows Washington D.C., Vermont and Massachusetts as the top three states of “the “extremely safe” category for international students.”  In September 2021 these States were listed as the top three of the list for Hate Crimes per capita in a 24/7 Wall Street report using FBI Uniform Crime Data and the FBI data shows Washington D.C. as having the US’s most violent crime rate per 100,000 inhabitants in 2020.  Further analysis suggests these anomalies reflect a selection and conflation of data that may mislead international students about the relative safety of their study destination.

Table Source: University Safety Index: State Safety by International Student Enrollment Percentage from How Safe Are U.S. Campuses?

Not All Crime on US Campuses is In Decline

While focusing on a comparison to criminal offenses two decades ago the Shorelight/USNGE’s own graph (below) shows that the “criminal offenses on campus” were comparatively higher in 2019 than five years before.   Also, the long term decline is largely due to a fall in motor theft, robbery and burglary which masks other trends on offenses against the person.  The U.S. Department of Education’s Campus Safety and Security Survey (CSS Survey), the source of the Shorelight/USNGE data on criminal offenses, shows that hate crimes, “sex-offences-forcible” and violence against women (VAWA) have increased in recent years.

Source: University Safety Index: State Safety by International Student Enrollment Percentage from How Safe Are U.S. Campuses? By Selene Angier 

Since 2014 the number of cases historically recorded by the CSS Survey as “Sex Offenses – Forcible”, increased 65% from 7,955 to 13,121 by 2019.  Since 2014 these offenses have been reported separately as “rape” or “fondling” with the former growing 33.5% and the latter by 124.2%.  A National Center for Education Statistics summary reflects these figures and notes that “according to reports in a student survey administered at several dozen large universities, officially reported sexual assaults represented only a minority of sexual assaults that occurred.”  

There has been excellent progress in reducing motor theft, burglary and robbery but the situation appears to have worsened in terms of sexually related offenses.  Offenses recorded as “aggravated assault” also remain stubbornly around the 4,000 mark. Mixing and matching the categories of crimes against property and crimes against the person fails to offer clarity that might be helpful in assessing risk.

Source: U.S. Department of Education Campus Safety and Security Trend Data

The CSS Survey also shows, separately, a 47% growth in reported offenses of violence against women, from 12,232 in 2014 to 17,578 in 2018 (the most recent data).  These have been registered since the Violence Against Women ReAuthorization Act 2013 (VAWA) brought changes to Clery Act reporting requirements.  With estimates that over 40% of international students are women it would seem reasonable to reflect this information in an article on campus safety.

Source: U.S. Department of Education Campus Safety and Security Trend Data

International Student Safety Off-Campus Matters

The CSS Survey data only includes cases, “…on campus, on property controlled by the university or a student organization, or on public property immediately adjacent to campus.” By using this measure the Shorelight/USNGE Index removes any information about the towns and cities where international students will hope to be welcomed.  This contributes to the leap of logic that establishes a league table of “the safest states to study in” which doesn’t include any city-wide or state-wide crime data.

The principle of aggregating data across a state is, itself, highly questionable when it comes to giving a student information on selecting a specific destination.  The statistician joke, credited to C. Bruce Grossman, that with your head in the oven and your feet in the freezer you are comfortable on average, comes to mind.  This consequences are evident as soon as you begin to consider more wide-ranging data about the crime rates in different cities.

In 2021 US News and World Report considered Massachusetts the 7th safest state in the US (although only 24th lowest for violent crime) but COVE Home Security in 2017 suggested the chances of being a victim of violent crime in Boston made “the city less safe than 83 percent of US cities”.  Neighbourhod Scout indicates that UMass Amherst is in a town that has a crime rate of 5.99 per 1,000 residents while UMass Boston is in a city with a crime rate of 26.45 per 1,000 residents.  Shorelight/USNGE use their Index to say both universities “…are located in the “extremely safe” category for international students” even though the numbers suggest the locations are quite different in terms of crime.

Washington DC, according to the Index is an ‘Extremely Safe’ State despite a 2019 crime rate which some sources indicate is 1.8 times higher than the US average and higher than in 95.5% of US cities.  American University’s campus may be a haven of civility and courtesy but students would probably be wise to exercise appropriate caution when they move onto the surrounding streets.  The university provides personal safety tips to international students which is both responsible and appropriate.       

Hate Crime is Relevant to International Students

The report is heavy on presenting data to reassure international students, yet surprisingly silent on the incidence of Hate Crimes recorded by the CSS Survey.  It was a 2008 amendment to the Clery Act which required post secondary institutions to report these incidents.  In 2018 the National Center for Education Statistics indicated that 43% of reported Hate Crimes in 2018 were motivated by racial bias.

The data presents a grim picture with a spike over the most recent years which is of relevance to students travelling to the US from abroad.  This may not fit the Shorelight/USNGE narrative but it is an important issue if students are to be given the most complete picture.  The Australian response to international students who are victims of crime might also inspire positive initiatives to engage productively with the issue rather than ignore it.

Source: U.S. Department of Education Campus Safety and Security Trend Data

At a state level Shorelight/USNGE report considers Massachusetts “extremely safe” for international students but the state’s campuses rank behind only New York (250) and California (174) in terms of reported Hate Crimes in the Survey.  The trend has been remorselessly upward for a decade.  In the broader Massachusetts context even the Editorial Board of the Boston Globe has recently argued that the situation in the state is serious enough to warrant its legislature updating hate crime laws.

Source: U.S. Department of Education Campus Safety and Security Trend Data

Whose Facts for What Purpose?

It is not unusual for organizations to give the most positive presentation of their situation and the Index is positioned as a response to a situation where “news headlines and social media shares can unfairly grab attention and raise concerns”.  But it seems specious to suggest that “U.S. News and Global Education (USNGE) and Shorelight — two leaders in U.S. higher education — have partnered to develop the University Safety Index” when one is owned by the other.  It also seems misleading to present the item as news on a website where the branding gives the gloss and reflected credibility of US News and World Report’s league tables.

While the article is designated as “News” the authorship, data and presentation of universities looks like an inside marketing job.  The writer was once on staff for Shorelight, has written regularly for the company’s website and describes herself as a “content manager specializing in e-commerce marketing, UX messaging and lifestyle brands.” The statistics were compiled by Shorelight’s vice president of data science and strategy.

The marketing dimension becomes even more clear when “Notable U.S. News Global Education Universities” are highlighted – they just happen to be Shorelight partners.  There is, however, no mention that the lowest “Somewhat Safe” category of the Index features Florida and Illinois where Shorelight has partnerships with Florida International University, University of Central Florida and University of Illinois at Chicago as well as new partner Eureka College.  The implication of the Index is that international students have more reason to be concerned about safety if they go to those institutions but that seems a less palatable marketing statement.       

Summary

Several countries and many universities are in a headlong dash for more international students and most recently Colleges Ontario commented on the need to recruit them to fill funding gaps. CBC News recently reported on the problems for students from south Asia who had arrived to study in Calgary but couldn’t find jobs and were unprepared for the winter weather. It’s a toxic mix where students are not getting realistic information about the situations they will encounter and there is every chance it will end in tragedy for individuals as well as blemishes on institutional reputations.

Fall intakes have shown that international students are returning to the US in significant numbers after the pandemic but it is entirely possible that some will have lingering doubts about attitudes towards foreign visitors. It is, however, unhelpful to underestimate the importance of ensuring that young people are given balanced information and not lured into a false sense of security.  International students are courageous, committed and deserve more respect than that.       

The US should also be applauded for publishing campus crime data in a consistent manner and might consider positioning this as a competitive advantage over the UK where there is a growing clamour for better data on student-related crime. While the Complete University Guide is to be commended for giving comparative information on an issue where one in five students are likely to be a victim, action from HESA or the Office for Students would be welcome.  For international students, agents and other decision makers the best advice is to demand information directly from your university of choice and avoid sales promotion gimmicks.

NOTES

*  I am not aware of any comprehensive and credible research on which countries are safest for international students. Various guides exist but tend to base their outcomes on overall country statistics. The Founder and CEO of iSchool Connect based a recent table in The Tribune of India on indexes covering factors such as Global Peace, quality of life etc. It includes Singapore at number five – a country where the Prime Minister has recently acknowledged “resentment over foreigners”.

**US News Global Education was formed as a collaboration between US News and World Report (USNWR) and Shorelight but is a subsidiary of Shorelight. The University Safety Index is a reminder of the link to USNWR’s own league tables whose methodology ex-Editor Peter Bernstein, in a classic Freudian slip, called “this mythology.”

***This blog relies, in part, upon my understanding and interpretation of various data sources and media reports. While data is almost always partial in the way it is collected, categorized and presented I have considered a range of sources in an attempt to ensure the points made about specific locations are reasonable. I am happy to correct any material errors brought to my attention by an authoritative source.

LEAGUE TABLE CLICK-BAIT COMPLICATION FOR UNIVERSITIES

It is standard to hear a manager in the English Premier League say “the table doesn’t lie” as they bemoan their lowly position or celebrate their success.  By contrast it has been equally standard to hear university recruiters put the case that various league tables are wanting in terms of nuance, specificity or even veracity.  But it may become even more complicated if university league table compilers have a direct, commercial interest in the outcome of the table and its impact on students.

In a recent article in The PIE, the Chief Development Officer of Times Higher Education (THE) outlined plans for millions of international students who consult its rankings website each year.  He said, “We want to stay top of the funnel and maximise the number of students coming to the site. What we will then do is identify a network of complementary, trusted partners that we will send those students to.”  The potential for universities to find themselves excluded or obliged to pay large sums for access seem obvious.

Regulators, governments and the sector’s networking bodies would do well to consider whether this manipulation of the recruitment process through commercially driven league tables is in the interests of the institutions and the students.  Back in October 2018 the Office for Students Director of External Relations wrote of the “challenge for policymakers….providing information responsibly and well as accessibly” but it is difficult to see any action to head off the private sector. Allowing brands that have been built with substantial public funding to be used as click-bait providing a return to private money certainly does not seem the best way forward.

Selective, Subjective and Subject to Manipulation

It is equally troubling to think that students may find themselves railroaded into choices by an organization that decides how the league table is compiled and has commercial partners who may have more than a passing interest in the result.  Elsevier have quoted Lydia Snover, director of institutional research at the Massachusetts Institute of Technology, as saying, “every ranking is based on the available, comparable data, and is built on the subjective judgement (over indicators and weightings) of its compilers.”.  Even when league tables are independently audited, consulted upon and done with good intentions they are about choices. 

UC Berkeley’s Center for Studies in Higher Education has suggested that “universities with frequent QS-related contracts experienced much greater upward mobility in both overall rankings and in faculty-student ratio scores over five years in the QS World Rankings”. HEPI’s president, Bahram Bekhradnia, did not find this a surprise and noted, “QS is a commercial organisation. They’re there to make money and their rankings are not objective.”  The higher education sector, while complicit in working with rankings media, is aware that this is a double-edged sword, and it may be that commercial imperative is sharpening one side. 

Those factors are made worse by the documented cases of universities deliberately manipulating the data they submit in order to secure a place higher up the ranking.  A University World News article in 2019 highlighted how the University of Oklahoma had been supplying US News and World Report rankings with incorrect information for nearly two decades.  Occasional errors seem forgivable but the more complex and wide ranging the tables the more scope there would seem to be for manipulation.

Legitimization and Lost Perspective

It seems a long time ago that in the late 1990s a few national UK newspapers would produce university league tables once a year as part of their wider agenda of news coverage.  But since the early 2000s league table compilation and publication has become increasingly central to the activity and business model of some HE sector-oriented media organisations.  Universities have played their own part in legitimizing the ranking races that may undermine their reputation and their ability to compete for students.

Many university planning offices have also spent hundreds of hours analysing league table performance and working to advise their senior colleagues on the levers that can elevate the institution’s position.  It would be difficult to believe any Vice Chancellor who says their university’s league table performance is not considered in strategic discussions.  League tables have become silent and increasingly oppressive enforcers influencing decision making, reputations and student experience.

It is certainly plausible that one of the factors influencing grade inflation at UK universities has been the weighting of a ‘good degree’ in the league tables.  When one university sees a perceived competitor getting league table marks for awarding a higher proportion of ‘good degrees’ the argument to amend marking criteria can be positioned as not disadvantaging students.  Almost without realising it institutions and academics may find their autonomy compromised by external factors.

Methodology, Misalignment and Misunderstanding

Over and above that, the dizzying array of league tables has become a way for compilers to open new routes for advertising income and securing influence.  Universities under 50 years old may welcome the chance to trumpet their performance against similar institutions and it allows the sector to applaud its own achievements.  But when high placings are used as advertising and marketing fodder to attract students the institutions are validating a process which is almost entirely out of their control and where interests may not be aligned. 

In 2004 the Times Higher Education (THE) began its University World Rankings but that has now been joined by 18 other main categories including World Reputation Rankings, Young University Rankings, Emerging Economy Rankings, Subject and Teaching Rankings.  The latest addition of Impact Ranking assesses universities against the UN Sustainable Development Goals (SDGs).  The accompanying launch events, announcements and conferences drive substantial content, which may be the purpose of media organizations but that is not the same as the purpose of universities.

The QS Rankings also began in 2004 and now covers 11 main categories, with several similar to THE but some noticeable differences such as Employability and System Strength.  They have built a student-facing event series – the QS World Tours – to bring students together with admissions directors at events.  Conferences and consultancy services also build out of the rankings as a source of revenue.

The variability of methodology that universities are trying to deal with shows in the league table results.  The THE and QS most recent “top 10 global universities” and “top 10 under 50 years old” show seven as being the same in each category but three different.  It’s a discrepancy which seems unhelpful if you are a student really wanting to know which were the best of breed in either category.

So, even when compilers are gauging similar categories they are making subjective choices about what to include, how to weight it and whether it will be important to their readers.  But in what is largely a game of statistical musical chairs there is some evidence that there are also fundamental misunderstandings about what is driving the performance of institutions.  Research by QS has suggested, for example, that students believe that a university’s ranking is substantially linked to employability of graduates when this only makes up a small element of the overall score.

It seems indisputable that league tables have become very big business for organizations that compile them and are influential enough to be a source of power over university decision making.  The prospect of them being leveraged to influence student choice and the recruitment potential of institutions has been made clear.  An informed, open discussion leading to collective action by the sector would be a step towards restoring balance. 

Image by Firmbee from Pixabay