Is most impact measurement in this sector actually theatre?
a lot of it is, yes, and i say that as the m&e person. when the funder designs the indicators, the org optimises for the indicators, and the indicators stop measuring anything real. goodhart’s law with a logo on it.
22 Likes
I want to resist the cynicism a little, because it is comfortable and it lets us off a hook we should stay on. Yes, much of the reporting is theatre. But the underlying question, ‘is what we are doing actually helping’, is the most important question in this sector and most of us cannot answer it honestly for our own work. The theatre is bad. The alternative that many orgs are quietly proposing, which is ‘trust us, we know it works’, is worse. I have seen what that produces.
25 Likes
i do not have a rebuttal to this and i was hoping i would.
12 Likes
the honest middle is: measure a little, for yourself, seriously, and treat the funder’s framework as a separate compliance chore. We conflate the two and then hate both.
10 Likes
the theatre also has a cost the metro orgs do not feel. our field staff now spend a day a week filling forms for three different funders with three different formats, and that is a day not spent with families. that is not neutral. that is harm.
14 Likes
From the funder side, I will admit our part in this. We ask for indicators we do not read, because our own compliance requires us to have asked. It is a genuinely stupid loop and I do not know how to break it from inside.
16 Likes