Making social funding dependent on proving outcomes is a compelling concept – but one that is full of fishhooks.
In Aotearoa New Zealand, the Social Bond pilot (also known as Social Impact Bonds) and the Ministry of Social Development’s Community Investment Strategy to Invest in Services for Outcomes are examples of this seemingly simple and sensible concept. And with limited funds, why would we put money into anything that can’t be shown to work effectively?
However, while it is great to see innovative new funding approaches, measuring social outcomes is anything but simple and is sometimes not very meaningful. And making funding dependent on this can, on occasion, even be counter-productive. Here are a few of the fishhooks in funding for outcomes:
- Who gets the credit? If Sienna stops getting in trouble with the police, is the youth programme responsible for her improved behaviour, or as it the alternative education provider? Or perhaps it was the touch rugby team she recently joined. Or her supportive uncle and his family. Or maybe she just did some thinking and changed herself. In all likelihood, the behaviour change is a complex interplay of all of the above. Who then, deserves the funding?
- The rise of success theatre and vanity metrics: These are terms from the world of business start-ups that are also applicable in the community sector. If an organisation has to show positive outcomes to get funding, successes are likely to be exaggerated, failures to be glossed over and metrics carefully selected to make the organisation look good.
- Perverse incentive to take the easy cases. Worryingly, there may be a disincentive to work with the people who most need support. Take two young offenders – one with good family support, the other with multiple challenges – who is less likely to reoffend? But if funding is only received for successful results, an organisation may need to choose between their own financial viability or taking on the people who need their support most.
- Human behaviour is not widget production. At what point is a youth offender no longer likely to reoffend, or someone battling depression fully recovered? People are not widgets on a production line, either compete or incomplete, functioning or not. Recovery and personal change is an up and down process whose length varies from person to person, may never truly end and is difficult to definitively measure.
- Objectifying the people we work with. Somewhat related to the point above, under this model, an organisation’s survival requires people to be in difficulties, needs those people to require “expert help” rather than turn to family and friends, and needs them to “recover” in the designated time period. There is a danger is that people become objectified, a “service user” and a checkbox on someone’s reporting framework.
- Can small, grass-roots organisations provide the required level of evaluation? The expertise and infrastructure required for attempting to prove outcomes is unlikely to be available to small, local organisations like community development projects and support groups. Yet these organisations, where people help themselves and each other, are often both low-cost and empowering. Will only the large organisations thrive?
Don’t get me wrong – it is very important for all organisations (including, of course, government and nongovernment funders) to try to understand their impact, to innovate, to learn, and to constantly improve. But when these processes are imposed externally and funding becomes dependent on them, there are some very major challenges to overcome.
Thank you Kate for a very well reasoned look at outcomes based funding.
Not to throw the baby out with the bath water I think having an outcomes based approach within a social service programme can be a useful discipline in terms of focus and mission.
But the idea of funders buying outcomes is, I agree, fraught with difficulties.
So how do we address the question some funders are asking, “is this money making any difference?” or more specifically ” is the money making the difference I want it to make in the world?”
Thanks Kate, hitting the nail on the head.
A counter movement may well be on its way. Read what Darren Walker head of the Ford Foundation is now saying http://www.fordfoundation.org/newsroom/news-from-ford/969
Catalytic and venture philanthropy, social investment and the like have brought energy and new funding into the civil society sector. Cast your eyes around, however, to look for examples of large-scale social change and humanitarian relief. You are likely to find established philanthropic foundations and NGOs are more prominent than ‘new’ philanthropy.
Thank you Kate – we’ve boosted this piece on the Community Research Facebook page. It’s very timely, and it’s getting some good comments. Hope all is well in your world. Drop in for a cup of tea some time – I’ve something may interest you. 🙂
Interesting thoughts Kate and all are important for those setting policy for outcomes to consider.
Like all things in this world, nothing is perfect; everything is in a process of evolution and transition. So be it with the delivery of social services.
With the exception of a handful of people who believe money grows on trees, the is an increasing expectation that funds provided for social services will be provided to those able to demonstrate the difference they will make with that money.
It is not really that important whether a person makes a personal decision to change their behaviour, whether the change was bought about by an intervention of a social provider or whether it was a combination of these or several other factors. Almost everyone understands that the majority of social issues are impacted upon by multiple factors.
What is important is that a provider is able to show what they did with the money and then collect from those that benefited the story of how their life changed for the better. What we have had to date is providers saying we used the $$ to admit XX people. This doesnt provide any indication of impact. Now funders are asking for the provider to say we used $$ to admit XX people and these are their stories.
Sensible contracting terms by funders (is that an oxymoron?) should be able to counter many of the potential issues you have outlined.
The reality is that despite considerable investment in social services over the past four decades, coupled with exponential growth in the number of social service providers, there is damned little evidence that this investment has actually made life better for those in need in NZ. Do we continue with this model or do we try something different.
I personally believe Funding for Outcomes is doomed to failure by mediocrity. Not because the idea is bad but because the majority of nonprofit service providers in NZ are too small, have inadequate capacity and capabilities, are poorly governed and have a mindset that says they should be funded for no reason other than that they exist!
Funding for Outcomes can only be successful where there are less providers and where providers engage in truely collaborative service delivery. Regrettably, despite the rhetoric, there is only a handful of examples of that taking place.
Kate,
With the greatest respect, your article has missed the point of outcome based contracts. When outcomes are not specified, which is true for most so-called “outcome based contracts” in MSD, then all of the problems you describe occur anyway. To go through your list in order:
– Organisations take credit for action that are irrelevant to the outcome (and find it easier to avoid blame because what they should achieve has not been specified)
– Anone who worked under MSD’s previous chief executive knows a great deal about “theatre and vanity metrics”. Again, because outcomes were not pre-specified it was very easy for him to claim success because he could pick and choose what was meant to have happened after implementation
– The whole welfare system already has an incentive not to work with difficult cases, because when there is no outcome metric and a case worker has a choice of working with easy or difficult clients, guess who she/he chooses?
– Agreed, but if no outcome is defined, what is to stop case workers “succeeding” by perpetually working with those who no longer need help?
– Again, agreed, but the most bureaucratic of bureaucrats are those who have no objectives specified because “success” is continually ticking boxes. Why make life difficult by addressing the real problems when ticking a box is an “outcome”
– Yes, yes, yes. The mistake is to believe large bureacracies, whether inside or outside government, are better at measuring success. It’s the bureacracies that can take comfort in process, not the people in small organisations who live to see improvement in the people they work with.
In my experience the problem is that bureacracies (including academia) loath measuring outcomes because those at the top of the hierarchy like processes, not outcomes. When you don’t specify outcomes, you are featherbedding hierarchies, not helping the people who need help.
Thank you for the thoughtful feedback and ideas raised by everyone responding – some great points. Like many things it is easier to point out the flaws (as I have done) than envisage the perfect system. Paul Schmitz puts it nicely in a Huffington Post article “To solve our social problems in our communities, the solution must be to build stronger communities not just stronger programs and services.” In an ideal world we would have communities sufficiently strong that there was little required in the way of services and funding. But then building stronger communities isn’t that easy either….
Great post Kate and thanks for article link John Godfrey. One of the main concerns, having just been through the Outcomes trial with MSD as a provider, is that purchasing for outcomes doesn’t address the ‘build stronger communities’ issue as Darren Walker’s article talks to – I just read this article from Stanford and again, spot on in regards to “funders having unrealistic expectations of what it costs to run an NGO” and the impact that has on delivering to outcomes
http://ssir.org/articles/entry/the_nonprofit_starvation_cycle/
We have good systems for outcomes measures already and an RBA focus as the best reason for this is to enable our practitioners to see the change they are affecting and strengthen their practice as the relationship and support provided is key to real change – quantifying that into ‘dollars attached for the change effected’, and when results of work with children may not be seen until later in life seems a difficult task if we do not embed a ‘full cost model’ (no private company would do the service for any less – just ask Serco!)
The more with less mantra is already impacting and a number of small to med social sector providers have closed over the last year – a very worrying trend when we don’t have a deep enough understanding of the community supports needed to build local resilience. Purchasing for outcomes is very ‘individual cohort’ focused so will not necessarily address the wider community needs or integrate across other services.