How can we measure the true impact of universities?

How can we measure the transformative power that higher education institutions (HEIs) have in addressing global and local challenges and contributing to a more sustainable world? Join us as we explore the multifaceted dimensions of university impact.

What is impact?

Higher Education Institutions (HEIs) have a huge potential to benefit society: academics are currently exploring some of the world’s most pressing challenges such as climate change, poverty and hunger. This societal benefit does not just come through citations and volume of research published, but through impacts outside the immediate academic sphere – real change in the real world. The university’s ‘’third mission’’ refers to its broader role beyond teaching and research. It encompasses the institutions commitment to societal engagement, community outreach and its contribution to creating a fairer more sustainable society. Third mission activities may include knowledge transfer, valorisation of research output and collaborative initiatives that address real world challenges.

Research Excellence Framework (REF) defines impact as “an effect on, change or benefit to the economy, society, culture, public policy or services, health, the environment or quality of life, beyond academia”^[1].

Why measure impact?

Traditionally, university rankings have primarily measured academic excellence in research and publications, and neglected this real world impact. It is important that going forward university ranking systems reflect impact and the third mission of the university as well. Measuring impact is a form of accountability, concretely demonstrating that something positive is being achieved; and research funders are now increasingly looking at impact beyond academia when they are allocating funds. Now, more than ever, universities need to demonstrate the work that they are doing to positively influence the world around them.

Impact is a fuzzy concept

Measuring publications and citations is straightforward and can be automated. Gathering comprehensive and reliable data on a university’s impact (in all its multivariate forms), however, is challenging and more complex.

Impact projects at universities often involve interdisciplinary efforts and data from different projects or departments is rarely centralised if it is captured at all. It is also challenging to establish a direct causal relationship between a university’s activities and a specific impact and impact may take years or even decades to fully materialise, it is not a linear process and assessing immediate results might not capture the true long-term impact. On top of this, universities operate in diverse geographical contexts, each with their own challenges and opportunities. Measuring impact, therefore, across diverse regions requires context-specific approaches.

Measuring impact

Due to increasing importance, certain ranking systems have emerged to try and create a solution to this immeasurability. They aim to help bolster the importance of impact on the agenda of universities and evaluate how universities are actively working towards addressing the world’s most pressing challenges, such as climate change, poverty, inequality, and access to education, etc. By recognising and highlighting universities efforts to make a positive impact, impact rankings can motivate institutions to embrace social responsibility and sustainability as key components of their mission and vision.

One such ranking system that attempts to capture impact on a global scale is the Times Higher Education (THE) Impact Rankings. Developed in collaboration with the United Nations (UN), Impact Rankings make a break with traditional university rankings that focus primarily on research output and reputation. Instead, the Impact Rankings focus on university’s efforts and performance in addressing global challenges and advancing progress towards the UN Sustainable Development Goals (SDGs). These goals address pressing global challenges of our time such as poverty, hunger, education, health, gender equality and climate change.

The metrics to measure a universities performance on each goal are broken down into various factors. Whilst the factors for each SDG vary, overall, they focus on the following:

• Research: This dimension evaluates the university’s research output related to sustainable development, using measures such as number of publications and citations on work directly addressing SDGs, as well as extent of interdisciplinary research projects that address the SDGs.
• On-Campus Activities and Consumption: This metric assesses the university’s efforts to implement sustainable practices and reduce its environmental footprint on campus. It includes factors such as energy efficiency, waste management and the use of renewable energy.
• Student Wellbeing: This dimension focuses on the well-being of students and their experiences at the university. It includes financial aid to students and whether the university is providing financial aid to its students who require it.
• Proportion of Graduates in Relevant Careers: This metric measures the percentage of graduates who pursue careers aligned with each of the goals. SDG3 ‘Good Health and Well-being’ measures the amount of students with a degree associated with a health-related profession.
• Community Programs: This dimension evaluates the university’s engagement with the local and global community through community-oriented initiatives.
• Diversity, Equity and Inclusion (DEI): Metrics to assess DEI focus on the university’s effort to promote diversity, inclusivity, and equitable representation among students, faculty and staff. These metrics consider factors such as gender balance, representation of underrepresented groups, and accessibility of education.

Times Higher Education report that the record number of submissions from universities globally to their rankings demonstrates that HEIs worldwide are serious about sustainability, committed to making change on the issue and willing to be held accountable for their progress towards SDGs^[2]. As of 2023, the THE Impact Rankings have expanded to encompass 1,705 universities representing 115 ^[3]. This year Australia’s Western Sydney University have taken the top spot for the second year in a row.

Moving beyond quantifiable metrics

Of course, it is unlikely that one ranking system is able to capture all of the elusive, multivariate and long term impacts that universities and their strategic partnerships have on the world. Rankings often rely on specific metrics, which may not capture the full breadth or depth of a university’s impact. Focussing solely on quantifiable metrics can lead to neglect of essential aspects like teaching quality, student experience and the vital role that the institution plays in the local region.

Whilst the THE Impact Rankings are aligned with the UN Sustainable Development goals which are pretty comprehensive, non-quantifiable impacts, such as cultural influence or local community engagement may be overlooked. If universities are ranked based on the metrics of this Impact Ranking, they may pivot their resources towards bettering these metrics and neglect less tangible or measurable impacts elsewhere.

It is therefore important to note, that while impact rankings can provide impetuous for universities to work hard on creating impact and making a difference in the world, they are just one tool for assessing universities and should be used alongside other qualitative and contextual information in order to make informed judgements on the quality and external impact of institutions. Universities should prioritise their unique missions, contribution to local contexts and focus on meaningful contribution to society rather than merely chasing rankings.

References

[1] REF Impact (no date) UKRI. Available at: https://www.ukri.org/about-us/research-england/research-excellence/ref-impact/ (Accessed: 01 August 2023).

[2] Impact ranking (2023) Times Higher Education (THE). Available at: https://www.timeshighereducation.com/impactrankings (Accessed: 01 August 2023).

[3] Times Higher Education (2023) Impact Rankings 2023. THE. Available at: https://flipbooks.timeshighereducation.com/19712/92579/index.html?70818# (Accessed: 01 August 2023).

Ready for more?

If you enjoyed this article then you might like our podcast episode about how academics can make an impact on sustainable development goals.

Our Impactful Academics program is designed for ambitious academics across all disciplines at every career stage to boost both the scientific and societal impact of their academic work through external engagement and partnering, SDGs as well as effective communication and personal branding. The next addition of the program will run from March 2024.

Tasha Day is a Project officer at UIIN, where she undertakes research activities and creates content on a wide variety of topics including entrepreneurship education, sustainability and research valorisation. Tasha has an Msc in Urban Geography, has worked in urban sustainability and placemaking, and is interested in how universities can be at the forefront of the transition from an extractive to regenerative and sustainable economy.