Is it impact? (Helen Slaney)

Posted by Tseen Khoo on

 

Photo by Johan Nilsson | unsplash.com

Do you think your research will change the world? Many of us do!

The element of ‘research impact’ is now a widely considered part of the research life cycle. By necessity, discussion about impact must be more evidence-based and demonstrable than aspirational when you’re including it in grant applications, impact assessment exercises, or job applications.

As the University’s Research Impact Manager, one question I’m frequently asked is whether certain outcomes “count” as research impact. This post aims to demystify what can be claimed as research impact and how you might talk about it.

It’s not straightforward to define precisely what research impact is However, in alignment with the Australian Research Council and international equivalents such as UK Research & Innovation, La Trobe understands impact broadly as “measurable, demonstrable and beneficial change occurring outside the university as a result of our research”.

Each component of this statement is important:

  • We need to be able to measure what has changed against a baseline condition and provide evidence to support our claims;
  • the change should benefit organisations or communities beyond the academic sector; and
  • original research must be associated with the impact, ideally recorded in peer-reviewed publications (although reports and other non-traditional outputs are also valid).

Engagement and impact: what’s the difference?

There is often a bit of confusion around what constitutes engagement or impact. Engagement refers to activity that you have undertaken to make your work available to external users. It might involve networking, presentations to stakeholders, serving on advisory boards, community co-design, or commercial licensing. The most effective modes of engagement will depend on your field and intended beneficiaries. Impact refers to the change which happens as a result. Engagement is how you influence your stakeholders to apply your research and make that change.

Primary and secondary impact

If your research findings have been applied by an external organisation – for example, they have changed policy or guidelines – this constitutes primary impact. It is not necessary to pursue evidence for secondary impact, or the follow-on effects of the policy change, although it can certainly enhance your impact narrative.

For example, the work of Dr Chris Maylea (Law) on involuntary mental health treatment led to legislative change via the Victorian Mental Health and Wellbeing Act 2022 (primary impact). While it may then be informative to consider how the Act is in turn having an effect on practice (secondary impact), this next level of evaluation is not essential.

Impact for Indigenous communities

Distinct impact criteria may apply to research involving Indigenous community collaboration. The exact criteria in each case will depend on the outcomes established as meaningful by the community. For example, impact may include preservation of cultural knowledge, such as language or oral history; provision of employment; or funding for community initiatives. A truly decolonising approach may require professional self-effacement from the researcher to ensure that the university is not profiting from Indigenous knowledge. Please see the end of this post for further reading in this area.

What is not impact… and the exceptions

1) Knowledge transfer to students enrolled in degree courses.

Exception: Short courses offered as in-service professional development to existing practitioners (e.g. the SOLAR lab).

2) Citations in academic publications. Confusingly, the same word – “impact” – is used in both contexts, but it means two different things.

Exception: If you publish in biomedicine or elsewhere in fundamental science, and your results or methods are applied in adjacent field en route to public or commercial benefit. However, in this case citations constitute a “pathway to impact”, rather than the impact itself.

3) Media coverage of your work. This may be evidence of attention, but it is not evidence of change.

Exception: If one of the indicators you are measuring is a shift in public discourse or the terms of a debate, media coverage can demonstrate where this is occurring. However, you would need to identify a specific metric such as a key phrase to track usage.  

4) Consultancy without associated publications.

Exception: If you are able to use the data gathered over the course of a consultancy as the basis for a research output, your narrative can state that the impact resulted from engagement conducted prior to publication. Alternatively, you may be able to make the case that prior original research determined your approach to the consultancy.  

Other FAQs

Q: Can I count citations in Royal Commissions, Senate inquiries, Parliamentary debate, etc?

A: Yes, as evidence, but ideally you will also need to show the legislative change that resulted. You do not need to demonstrate the mechanism.

Q: Are audience or participant surveys evidence of impact?

A: Not really, because they don’t show change. You will need to conduct a baseline survey and/or follow-up.

Q: My drug or therapy is undergoing clinical trials. Has impact resulted yet, or does it need to be on the market?

A: It depends who is conducting the trials, as in this example from University of Queensland.

Q: Is a patent evidence of impact?

A: It should be included as part of the narrative, but doesn’t necessarily prove change or benefit in itself.

Q: I have done R&D (research and development) for an external company which improved one of their products. Is that impact?

A: Yes, as in this example from UTS – provided you have also published the results.

---------------------------------

I still have questions:

---------------------------------

Further resources:

There is a growing body of literature on decolonizing research practice. Some discussions mentioning impact in this context include:

  • Datta, R. 2017. ‘Decolonizing both researcher and research and its effectiveness in Indigenous research’. Research Ethics2, https://doi.org/10.1177/1747016117733296
  • Jones, J. 2021. ‘Acknowledging Sovereignty: Settlers, Right Behaviour and the Taungurung Clans of the Kulin Nation’, Law and History 2 (117-43).
  • Tsey, K. et al. 2016. ‘Evaluating research impact: the development of a research for impact tool’. Frontiers in Public Health 4,  https://doi.org/10.3389/fpubh.2016.00160
  • Tuhiwai Smith, L. 2018. ‘The Art of the Impossible: defining and measuring Indigenous research’ in Spooner & McNinch (eds.) Dissident Knowledge in Higher Education. Regina: University of Regina Press.

---------------------------------

Dr Helen Slaney is La Trobe University's Research Impact Manager, located in the Research Performance team.

Having worked in research management since 2016, Helen completed her PhD at Oxford University in 2012, where she was based at the Archive of Performances of Greek and Roman Drama. She subsequently held a Junior Research Fellowship at St Hilda's College, Oxford, followed by a British Academy Postdoctoral Fellowship (2014-16).

In a research administration capacity, she was previously Research Bids Manager at Roehampton University in London. In 2020-21, she led the team responsible for implementing OPAL (Open@LaTrobe), the university's Open Access platform.

Comments

Post a Comment