Measuring impact in our evaluation framework
The Watertrust model presents a unique challenge for evaluation because we seek to effect change in complex systems. Rather than trying to measure impact, our evaluation approach needs to test our relevance, effectiveness, and contributions while remaining humble and avoiding claims that cannot be substantiated.
In December 2022, our evaluation framework was approved by our board. We spoke to our evaluation specialist, Amanda Wealands, about the choices we have for measuring and tracking our impact.
Q: Can you explain the unique impact that Watertrust aims to achieve?
Our mission is to improve the decision-making processes for water and catchment policies in Australia. We envision a future where citizens perceive these decisions as fairer, more reasonable, and consistent with available evidence, ultimately enhancing their legitimacy. What sets us apart is our focus on the decision-making process itself, rather than advocating for specific outcomes.
Q: Could you elaborate on how Watertrust brings about change and achieves this unique impact?
Our approach revolves around people and process, rather than predetermined outcomes. We prioritise co-design, broad participation, and creating independent spaces for the dialogue and fruitful conflict that allows for collective problem-solving.
As a non-governmental organisation, we leverage our independence, specialised knowledge, and determination. Building trust is paramount, and we foster productive relationships while upholding our values of inclusivity, humility, honesty, integrity, transparency, and accountability. This helps us to facilitate workable agreements among conflicting interests.
Q: Trust certainly plays a crucial role in your work. How does Watertrust measure the impact of its initiatives?
Given the complex and uncertain nature of our work, where change emerges from rapid interactions, we adopt a developmental evaluation approach. We focus on three key questions: the relevance of our organisation, the effectiveness of our initiatives, and the difference we are making.
Understanding the intricate social and political context, we don't attempt to claim causality or attribution. Instead, our evaluation framework facilitates learning, improvement, and tracking our contributions and influence as we make progress.
Q: Why is all of this important for Watertrust?
At Watertrust, we are committed to making a lasting impact on water and catchment management policy. Our unique approach drives towards decision-making processes that are fair, reasonable, and evidence-based. While tangible evidence is important, enduring impact often takes time and may not be immediately visible.
We know our work can effect change, even if it goes unnoticed or is attributed to others. By focusing on our core values and effectively communicating our activities, we can bring about meaningful transformations in water policy.
NOTE: You can access the Watertrust Evaluation Framework here.
Our developmental evaluation approach focuses on three questions - are we relevant, were we effective, did we make a difference?