Understanding risk and success in social impact bonds

ThinkForward’s Social Impact Bond was one of the first in the world to successfully complete. Thanks to this investment we not only transforming the lives of over a thousand of the most disengaged young people in London but investors in ThinkForward’s programme also reaped a financial return.

Since completing the bond we have identified some key lessons highlighted in Philanthropy Impact’s December 2016 issue. To view the full article click here or read the below.

In the UK and continental Europe, SIBs are being used primarily to experiment with new interventions, such as the programmes commissioned by the UK’s Department for Work and Pensions Innovation Fund to support disadvantaged young people. In the US, however, ‘pay for success’ initiatives are generally used to scale existing evidence-based programmes. Following the launch of the first SIB in the UK in 2010, much was written to describe the mechanics, structures and challenges of implementing new deals. Over 40 SIBs have been launched since then – in the UK, US, Australia, Canada, Netherlands, Belgium, Portugal and Germany. As investors crowd-in and the first round of SIBs have been completed, the buzz is less about how to design them and more about whether they have worked. What are the lessons from the first SIB deals? What should investors and advisors know about measuring success and risks within SIBs? As investors crowd-in and the first round of SIBs have been completed, the buzz is less about how to design them and more about whether they have worked. Based on our experience developing SIBs, we have identified some key lessons. Firstly, investors should be aware of the risk that SIBs may not reach completion as planned. The first SIB in the world was launched in the UK in 2010 with the One Service in Peterborough Prison. It was designed to fill a gap in the UK criminal justice system by supporting short-term prisoners and preventing their reoffending. Social investors invested £5m in a project meant to run for seven years. But in 2014 the Ministry of Justice announced a decision to restructure the provision of probation services nationally and the Peterborough pilot was brought to an early close.

The New York Times (Wall St. Money Meets Social Policy at Rikers Island) reported a similar outcome for the first SIB in the US, launched in 2012 to support high-risk adolescents detained at Rikers Island, the New York City jail. In this case, it wasn’t the change of heart of policymakers but implementation issues: the control group fell apart as wardens ‘could not separate teenagers who were to participate in a course of cognitive behavioural therapy from those who were not supposed to attend’, and then there was the failure to recruit the teachers required to carry out the therapeutic programme after the city’s Education Department pulled out. Was this a total failure? Investors will certainly be disappointed: Goldman Sachs had invested $9.6m with a 75% guarantee from Bloomberg Philanthropies. But from the perspective of the City of New York, it wasn’t a disaster: the city had tested a new intervention at scale with the benefit of a rigorous evaluation and then walked away without spending any taxpayer money. So another key lesson for investors is the importance of focusing part of their due diligence around their partner’s record of, and ability to, recruit beneficiaries and frontline workers. At ThinkForward SIB, our programme in London helps more than 1,000 at risk young people aged 14 to 19 years to complete their education and transition into work. The challenge of recruiting beneficiaries was partially solved by working inside schools. ThinkForward SIB carefully selected partner schools where teachers were committed to refer, and give access, to students. And when it came to staff, ThinkForward investors took a conscious decision to pay a competitive salary to frontline workers in order to secure the most talented applicants. When SIBs reach maturity and investors receive payments linked to outcomes, a further set of questions arises: how to compare results across SIB deals? What is the relationship between financial return and social impact? The promise of SIBs (and impact investing in general) is to align financial return with social impact, but one does not always imply the other.

Last year the New York Times (Success Metrics Questioned in School Program Funded by Goldman) reported how early-education experts questioned the validity of the impact results associated with a Utah pre-school programme designed to help 109 ‘atrisk’ kindergarten children avoid special education. Goldman Sachs reported a $260,000 payout from the SIB but, according to the NYT, critics argued that the programme’s unusually high success rate (99%) was based on a ‘faulty assumption that many of the children in the programme would have needed special education without the preschool, despite there being little evidence or previous research to indicate that this was the case’.This issue arises when validation of outcomes is not dependent on a control group or counterfactual assessment and there is the possibility of ‘creaming’, that is, enrolling participants who are easiest to work with and to achieve outcomes. As not all SIBs will run a control group – it is not always practical, ethical or cost-effective – impact investors must appreciate the limitations of linking the achievement of an outcome (say avoiding special education) to impact (the degree to which the outcome resulted from the intervention). In the case of the ThinkForward SIB, while the payments were not linked to a counterfactual, we worked with the Education Endowment Foundation to carry out a pilot randomised controlled trial (RCT) with randomisation at school and pupil level. This evaluation, which is separate from the SIB work, aims to identify quantitative evidence that the ThinkForward intervention is having a positive impact on education attainment and other outcomes. A clear relationship between financial return and impact is also complicated by some pricing strategies.

Sometimes commissioners link payments to a set of outcomes. In the case of the DWP Innovation Fund, for example, its rate card set a price for ten outcomes per participant. DWP would pay for improved attendance at school (up to £1,400 per participant), entry into employment (up to £3,500 per participant) and for other outcomes linked to improved employability. The outcomes in the rate card are important, but not all are equally relevant. A sophisticated impact investor will want to know if the financial return is driven by less important outcomes, say passing one GCSE at school (DWP pays up to £1,100 per participant), or is the result of achieving more socially meaningfully outcomes such as securing employment for 26 weeks (DWP pays up to £2,000 per participant). Another challenge with some payment structures is that they pay for the same outcomes, even if the programmes or the people the programmes target are very different. For example, a 12-week employability programme may be effective with young people who are work ready, but those further from the labour market may require longer interventions to achieve the same employability outcomes. The pricing structures don’t always recognise these nuances. Over time we hope that commissioners’ pricing will be better targeted, reflecting different levels of support needed to get different groups of people to the same outcome. Until then, investors who want to support the hardest to reach may need to accept lower financial returns. Social impact bonds have opened up government funding streams to delivery partners that would otherwise not have been able to access payment by outcomes financing, and nudged up expectations of managing to impact. However, the challenges reviewed above suggest that SIBs should not be seen as the perfect tool for measuring social impact. Ultimately, investors, commissioners and delivery organisations must define and measure success, and hold themselves accountable to the objectives laid in their own organisational missions.

Give the next generation a chance: it might be easier than you think

Young people are either unable or taking longer to secure a first foot on the job ladder.  In England alone, around 450,000 NEETs have yet to make the move from learning into work, outside casual or holiday jobs.  There’s evidence that more employers expect applicants to be ‘job ready’ leading to a catch 22: no skills, no job; no job, no skills.

For the sake of all our futures, we need to close the growing gulf our young people face between leaving school and finding employment.  In the current economic climate that’s a big ask, but employers unable to offer young people jobs or apprenticeships can still have a role.


At a recent Work Foundation event, Dr Anthony Mann, Director of Policy and Research at the Education and Employers Taskforce, pointed to a relationship between the extent of employer engagement arranged during school years (14-19) and the successful labour market progression of young adults (19-24).  He found that each employer contact improved the odds of being non-NEET by 29 per cent.


The provision of work shadowing, mentoring, workplace visits or talks in schools all make a difference.


And, despite the current controversy, in interviews carried out as part of a recent Work Foundation report, one important area where long-term young unemployed people felt they had not received the support they needed was around access to opportunities like work experience.  It’s just got to be offered in the right way, not exploitative but opening up new networks and avenues which could lead to a job.


There’s plenty of scope. For example last year, research by Demos found that ‘shockingly’ there is currently little to no engagement from local employers or businesses in school ‘career fairs’.


We all remember the person who gave us our first break.  If you’re an employer, could you give a young person a helping hand – a connection to the world of work?  As one young person quoted by The Work Foundation said “I love working, I love being busy…If someone could just give me a chance.”


Shaks Ghosh, Chief Executive, Private Equity Foundation