The Traditional Owners of this land are those who identify as
Aboriginal and Torres Strait Islander Peoples.

Sovereignty was never ceded.

ANTAR pays respect to Elders past, present, and emerging through our dedicated advocacy for First Nations Peoples’ justice and rights.

ANTAR acknowledges the responsibility of committing to a truth-telling process that promotes an honest and respectful path forward for future generations to build upon.

Enter website
6 minutes

Australia’s Indigenous evaluation strategy

Dr Danielle Campbell Dr Linda Kelly Marlkirdi Rose Napaljarri
Last edited: April 2, 2024

In October 2020, the Australian Government took a major step towards understanding what First Nations policies and programs work – and why – by releasing the first national Indigenous Evaluation Strategy.

The Indigenous Evaluation Strategy embraces the overarching principle that First Nations peoples, perspectives, priorities and knowledges need to be front and centre in evaluations that concern them and the Strategy Guide provides useful information on how to do this.

But, having worked for more than 10 years with the Central Land Council (CLC) to help them monitor and evaluate their Community Development Program (CDP), we think the documents don’t go far enough in recognising monitoring and evaluation (M&E) as a challenging, non-linear, time-consuming and deeply political process. It would also benefit from more practical guidance. The strategy guide includes only one reference to checking whether Indigenous people can participate in evaluations on an equal footing and makes no reference to evaluation politics. The literature increasingly points to engaging with worldviews and power differences to co-create Indigenous evaluations that acknowledge merit in multiple perspectives (see here and here). This is hard to do and organisations thinking of embarking on evaluating Indigenous programs should be prepared for the challenge.

Take the following program administered under the CLC’s CDP. Set up in 2005, the Warlpiri Education and Training Trust (WETT) is governed by Warlpiri landowners and educators (including co-author Napaljarri). WETT has so far invested over $30 million of gold mining royalties in education and training programs that benefit four remote Warlpiri communities in the Tanami Desert, plus extensive community engagement and governance processes. WETT M&E has been facilitated by the CLC since 2009, informed by the staff (including co-author Danielle) belief that it would support improved process, outcomes and accountability to Indigenous people and help demonstrate that supporting Indigenous locally-led development is more effective than centralised, top-down programming.

Over the last decade the CLC, WETT and the Institute for Human Security and Social Change (where all co-authors now work) have experimented with many different evaluative approaches and methods in trying to assess WETT process and impact. Three things we’ve learnt stand out.

First, because Indigenous and non-Indigenous stakeholders operate from fundamentally different worldviews, we don’t necessarily value the same outcomes. Non-Indigenous stakeholders are generally interested in education, training, employment and Indigenous capacity development outcomes. Warlpiri aspirations include these objectives, but they are underpinned by a broader set, including maintenance of Warlpiri culture, language and connection to country, respectful relationships and self-determination.

Second, views on the purpose of M&E, the type of information that constitutes quality evidence and the best methods to generate it differ. Everyone agreed that accountability to Indigenous people was the primary intention but CLC staff were also focused on a secondary accountability to the Australian Government. This was due to their ultimate control of resourcing through the Commonwealth legislation that provides for royalties and the payment of CLC operational costs. On top of regular opportunities for Warlpiri feedback and reflection through WETT’s CD process, formal M&E methods and processes were adopted. This included interviewing Warlpiri decision-makers and participants, focus group discussions with CLC staff, and analysing project reports and expenditure. But CLC staff worried that this may not be rigorous or ‘objective’ enough to convince external stakeholders so explored other approaches including an ‘evidence-based’ participatory capacity development framework. The WETT committee enjoyed the first workshop but saw it as too time consuming and disconnected from their WETT work to repeat and it was shelved. In another example, the CLC looked into doing a cost-benefit analysis but did not proceed (the required data wasn’t available and the methodology limited local decision-making and control).

These attempts took the M&E away from the Warlpiri focus on mutual accountability between WETT decision makers, landowners, communities, project partners and the CLC. Warlpiri prefer methods that support relationality and the co-creation of WETT knowledge through sharing stories, interviews and collaborative paintings.

We have our own Yapa (Warlpiri person) ways of watching closely to assess how things are going, giving encouragement and feedback so things can improve, and sharing information through stories and paintings. But Kartiya (non-Indigenous people) don’t see this or maybe they don’t understand it.

(Marlkirdi Napaljarri)

Third, you can’t simply ‘mix’ evaluation processes that work well for Indigenous and non-Indigenous stakeholders. This ignores power dynamics that can skew evaluation and privilege certain ways of assessing programs. The earlier focus on government priorities, using forms of evidence valued by governments was a pragmatic approach – and accountability to Warlpiri was happening in other ways – but it was at the expense of setting up a co-created evaluation approach designed to meet everyone’s needs. On reflection we made the mistake of approaching evaluation as a technical process and an add-on to WETT, which was otherwise Warlpiri-led.

A new 18-month WETT funded project brings the Institute, CLC and WETT together to co-create an evaluation framework that will try to weave together different priorities and knowledge systems. The Ground Up Monitoring and Evaluation Approach being developed in Northern Australia and the Indigenous Canadian concept of “two-eyed seeing” – bringing together the strengths of Indigenous and non-Indigenous ways of knowing – are proving helpful. We are paying close attention to differences in power and perspectives, including that of the Institute’s non-Indigenous staff, to deliberately build a shared approach.

This is slow and complex work. It builds on 15 years of M&E learning about what does and doesn’t work. It draws on long-term relationships grown advocating for WETT in meetings from Lajamanu to Canberra and working together on the ground in the Tanami. It involves experienced Warlpiri researchers and WETT champions with a deep understanding of the local context and non-Indigenous researchers with knowledge of the strengths and limitations of different evaluative forms, who are committed to valuing Indigenous knowledge and shifting power. Valuing and navigating different knowledge systems is not something that can be workshopped, written into an evaluation plan and delivered. In our experience, it’s facilitated by a shared commitment to ‘two-eyed seeing’, trusting relationships which allow for frank cross-cultural conversations, and endless learning and adaptation. The fact that Warlpiri are funding the evaluation design work for the first time has also sharpened the focus on Warlpiri priorities.

We are still in the talking, trialling and refining phase but we all agree that approaching evaluation as a complex, political process that needs careful navigation and co-creation seems more likely to generate a rich picture of WETT’s ongoing journey and achievements for all stakeholders. The Indigenous Evaluation Strategy is right to call for centring Indigenous people in evaluation, but it presents it as a pretty technical and apolitical process. It would benefit from more examples of the challenging reality of bringing together different knowledge systems and more detail on what it takes to support good Indigenous evaluation in practice.

Dr Danielle Campbell

Danielle Campbell is a Senior Research Fellow at the Centre for Human Security and Social Change (La Trobe University) who works on Monitoring, Evaluation and Learning. She has over 20 years’ experience designing, delivering and evaluating locally-led development initiatives in Indigenous Australia and more recently the Pacific. During this time, she has moved between working as a senior manager to build and deliver development programs and as a researcher with community, national and regional development programs on monitoring, evaluation and learning. This has involved working extensively with Aboriginal land councils, international and domestic non-government organisations, university institutions, government departments and industry partners. Her interests focus on locally led and co-designed processes, local governance, Indigenous self-determination and de-colonising monitoring, evaluation and learning practice. Danielle has a strong background in adaptive programming and management and expertise in bringing together diverse stakeholder groups to develop shared understandings. Before joining the Institute in 2019, Danielle led the establishment of both the Central and Northern Land Council’s successful Community Development Programs across Central and Northern Australia, including collaborating with Aboriginal people to design, trial and adapt locally appropriate monitoring, evaluation and learning frameworks, processes and tools. Now with the Institute, Danielle is leading a project to further refine these existing frameworks, as well as leading a major new initiative with the Warlpiri Education and Training Trust to co-design a ‘Tracking and Learning’ Framework. In the Pacific, Danielle is also working closely with the Australia Pacific Training Coalition’s Monitoring, Evaluation and Learning Team using a two-way learning approach to further develop their program wide systems, data collection methods and tools, as well as build capacity. Danielle completed her PhD in Public Health at Flinders University in 2003 with a scholarship from the Cooperative Research Centre for Aboriginal and Tropical Health.

Dr Linda Kelly
Co-Director, Institute for Human Security and Social Change Latrobe University
Dr Linda Kelly is Co-Director, Institute for Human Security and Social Change Latrobe University.

She has over twenty-five years’ experience in international development with non-government organisations, DFAT, university institutions, and UNDP Pacific Office. She has authored and co-authored numerous research reports, papers and publications, and presented at industry conferences. Throughout her career, she has advocated for and integrated gender equality, disability and social inclusion in her practice.

Marlkirdi Rose Napaljarri

Marlkirdi Rose Napaljarri is a Warlpiri woman who works as an advisor and part time researcher with the Institute of Human Security and Social Change (La Trobe University). She is a qualified teacher and highly experienced Warlpiri educator.

Marlkirdi was a driving force in setting up the successful Warlpiri Education and Training Trust (WETT) since the idea was first discussed 20 years ago. She worked with WETT and the Central Land Council on the co-creation of a WETT monitoring and evaluation framework. Marlkirdi is passionate about mentoring the next generation of Warlpiri researchers.