Paul Penley
Peter Singer’s 2013 TED talk about effective altruism argues for the moral necessity of supporting effective charities. Since most of us would prefer to support charities we know, working on issues we care about, rather than only the three charities Givewell considers effective, what do we do?
Can you prove it?
We start by determining if organizations we like monitor and evaluate programs well enough to make a conclusion either way. In my doctoral research, it all started with a hypothesis. I had to tentatively make a claim and set out to test it. The one requirement my dissertation mentor gave me: your hypothesis must be falsifiable. You must make a claim that can be proved either true or false, based on the evidence.
Most charities claim to make a difference but don’t measure performance with data that can prove or disprove such a claim. If a charity claims to change the future of kids or animals or communities but does not track key performance measures into that future, those claims drift into the world of hopes and dreams. Reality requires quantitative and qualitative feedback to claim effectiveness.
That’s why worthwhile charity evaluators (like Intelligent Philanthropy) collect basic information about how charities measure impact. Knowing how charities measure impact does not provide conclusive proof or independent verification of effectiveness, but it does determine if enough pieces are in place to actually solve that riddle.
5 questions non-profits must answer
Five questions determine if you can know a charity’s effectiveness:
Does a charity…
- Track outcomes years after serving people, or only annual accomplishments?
- Measure program outcomes against relevant benchmarks, averages or control groups?
- Complete independent evaluations of program effectiveness?
- Survey beneficiaries about program quality and impact?
- Pursue specific and measurable organizational goals?
If a charity is not taking all five steps to track outcomes properly, then you don’t know if their work is effective or not. I fear the majority of charities could not answer all five questions affirmatively (and that is what Intelligent Philanthropy is finding during the first year collecting this data). That does not mean those charities are failing to create lasting improvements, but it does mean we will never know one way or the other.
So how do your favourite charities fare when asked the five questions?
Paul Penley is director of research at the philanthropic advisory firm Excellence in Giving and creator of IntelligentPhilanthropy.com
Comments (2)
This is somewhat misleading. Those questions are all process questions. They pertain to whether we *can know* a charity's results. They don't tell us anything about what the charity's results actually are. A philosopher would say that they're about epistemology (=what we know) rather than ontology (=what is the case). So while I agree that they're good process questions, people shouldn't mistake them for showing whether a charity is effective. This is because a charity might be able to answer 'yes' to all of them, and still have terrible results. For example, it may have an independent evaluation which shows that it's results are rubbish. So the epistemology is good (we know) but the ontology is bad (the facts are bad). This is important because we often observe donors thinking that a good process for measuring results is equivalent to having good results. As a charity CEO, I was often asked 'how do you measure your results' (=epistemology) but barely ever asked 'what are those results' (=ontology). It's the latter which we all care about.