Measurement and Evaluation: How Do You Know You’re Making an Impact?

How do you know you’re making an impact?

This has become one of the most frequently asked questions by philanthropic leaders in recent years. Perhaps the question gets asked so often because a simple answer is elusive. There is no silver bullet. But answer the question we must.

In many ways Fluxx is positioned in the middle of the philanthropic ecosystem – where funders and nonprofits intersect. I talk to many on both sides of the equation almost every day and I never forgo the opportunity to ask them what they think about measuring impact. The same themes pop up over and over again during these conversations:

• Not everything is quantifiable.
• Storytelling is powerful.
• Perfection is the enemy of the good.

Not Everything is Quantifiable

Reporting on impact is particularly difficult in our world because progress isn’t always linear and it’s not always quantifiable. Tracking exactly when and how impact is achieved can be difficult, says Matthew Maguire, finance business partner at the Children’s Investment Fund Foundation (CIFF). Maguire cites climate change policy as an example. You may be doing huge amount of work for three, four, five years but, effectively, until that legislation is passed, it appears as though you haven’t achieved any impact. Sometimes impact is a thumbs up or thumbs down proposition.

Storytelling is Powerful

In some cases, eliciting anecdotal data can be the difference between success and failure. Take, for example, the work that Maguire and his team at CIFF support in Malawi.

When about 5,000 children weren’t showing up for World Food Program nutrition supplements, CIFF wanted to know why. Initiative partners spoke with mothers in the program to gather anecdotal evidence on the ground. They learned that it was too difficult for many families to reach the nutrition access points.

With new evidence in hand, they decided to increase access points from 27 to 80 and sent more volunteers out to spread the word to more mothersin the area. As a result, the consumption of nutritional packets rose from 65.6 percent to 82.9 percent.

The example shows that combining quantifiable data with anecdotal evidence can lead to a course correction that ultimately boosts impact. For CIFF, it’s a low level outcome that will eventually roll up to help achieve the impact that it’s looking to achieve at the top level.

Don’t Let Perfection Become the Enemy of the Good

When it comes to impact measurement, we have a 100% problem. Though we all agree on the critical importance of measuring impact, more can be done.

We’re letting perfection become the enemy of the good. Let me explain.

My roots are in design. As a designer, I believe in iterating. This is how we approach product design here at Fluxx. Prototype, test, analyze, and refine. In many ways iteration is also one of the fundamental principles behind impact measurement: When you know what works and what doesn’t, you can course correct to improve your approach.

I would wager that grantmakers don’t restrict their funding strategies to only those programs that guarantee 100 percent success. You don’t forgo important advancements in malaria reduction because you’re not sure you can eradicate the disease entirely in one funding cycle. In your own way, you prototype, test, analyze, and refine.

The same should be true for impact measurement. It’s okay if your approach isn’t perfect right out of the gate. It might not be. But you’ll learn, and iterate. And you’ll begin to see results.

It’s early days, still. But that’s what makes this so exciting. It’s not that often that you have the opportunity to claim a stake in defining the future of the sector. But none of this is possible if we don’t take the first step.