Barriers to the Reporting of Soft Outcomes

Barriers to reporting soft outcomes to funders

The short-term nature of funding streams, and the general funding environment presents barriers to soft outcome (or qualitative) data being reported to funders.

Delivery Organisations: Short-term funding streams leave funding recipients desperate to secure funding from whichever sources they can. This can result in an organisation losing focus and delivering projects they do not have expertise in. Expertise in the target group or activity is required for understanding of which indicators demonstrate greatest impact of project work.

Short-term funding streams can also result in a certain lack of interest in the impact that project work actually has. It can take long periods of intervention to make, recognise, and demonstrate impact. By this time projects may already be more interested in securing new sources of funding than demonstrating this work effectively.

The competitive nature of project work means that delivery organisations often fail to build in administrative and evaluative costs into their projects; they are sometimes unable to report data to funders. They are much less likely to negotiate with funders over the kinds of data that represent value for money and impact. Funding organisations can therefore have a lack of confidence in the information that they are presented with for monitoring or other purposes.

Funding organisations: Funding organisations such as the ESF often pay lip-service to soft outcomes, yet accept only quantitative information for monitoring or audit purposes. This gives the impression that soft outcomes are not important to delivery organisations, despite the fact that they usually (at least with hard-to-reach groups) represent maximum impact of project work.

Funders need to accept and encourage delivery organisations to build in administrative and evaluative costs. There should be some level of dialogue over the kinds of data that represent best value for money in terms of project impact.

The often arduous monitoring requirements that many funders have, and the short term nature of funding streams make innovating and experimenting with new forms of impact measurement much less likely. Funders should concentrate on keeping the monitoring information that they require to a minimum. This would allow delivery organisations to report more accurately, and make the reporting of soft outcome data more managable.

Links to Soft Outcome Tools and Resources

An interim list of some of the sources listed below can also be found in the Soft Outcome & Soft Skills Report.

The Outcomes Approach

• Geared towards homelessness related organisations, but applicable to other kinds of organisation too - a practically orientated guide to developing outcome measurement systems: Burns S & Cupitt S – Managing Outcomes: A Guide for Homeless Organisations; Charities Evaluation Service (2003)
• A guide specifically tailored to bidding for money from the Big Lottery Fund, but also useful for considering an outcomes approach before a project has begun and how you may report on outcomes: Burns S & MacKeith J – Explaining the difference your project makes: A BIG guide to using an outcomes approach; Big Lottery Fund (2006)
• This publication describes an outcomes approach from an organisational and infrastructural support perspective:Burns S & Turton D - Distance Travelled: The experience of outcome monitoring within NCVO's Sustainable Funding Project; National Council for Voluntary Organisations (2006)
• Outline of a speech about barriers that funding agencies and funding recipients may face to implementing an outcomes approach: Carrington D – Outcomes Funding: What is it and what are the consequences; Wales Funders Forum (2003)
• Useful briefing paper about why and how an outcomes approach can be implemented within the voluntary and community sector: Charities Evaluation Service – Using an outcomes approach in the voluntary & community sector: A briefing for funders, commissioners and policy makers on the National Outcome Programme (2006)
• A practical guide for development of an 'outcomes approach' to project work; details how to think and work with and towards both hard and soft outcomes: Cuppis S & Ellis J – Your project and its outcomes; Charities Evaluation Service & Big Lottery Fund (2004)
• The Charities Evaluation Service Outcomes Online Website contains more useful resources.

Defining and Measuring Soft Outcomes

• A useful review of the literature: Butcher B & Marsden L - Measuring Soft Outcomes: A Review of the Literature; The Research Centre (2004)
• Oft quoted article with details about measuring distance travelled: Dewson S, Eccles J, Tackey ND & Jackson A – Guide to Measuring Soft Outcomes and Distance Travelled; Institute for Employment Studies (2000)
  
• An overview document to accompany the Welsh European Funding Office article, about surveys regarding soft outcome measurement tools and systems: Lloyd R & O’Sullivan F - Measuring Soft Outcomes and Distance Travelled: a Methodology for Developing a Guidance Document; DWP (2004)
• Report about soft outcome measurement within the East Wales Objective 3 Programme and the Equal initiative: NIACE Dysgu Cymru – Measuring Soft Outcomes: East-Wales Objective 3 Programme Research Report; (2006)
• Simon Simpson argues for a common standard/definition of "soft skills": Simpson S – The Measurement and Recognition of Soft Skills: Developing a Common Standard; (2006)
• One of the first reports about the development of Soft Outcome reporting systems commissioned by the Welsh European Funding Office: Welsh European Funding Office - A Practical Guide to Measuring Soft Outcomes and Distance Travelled; (2004)
• Welsh analysis of the Measuring Soft Outcomes: NIACE Dysgu Cymru - Measuring Soft Outcomes: Research Report; (2006)

Indicators and Outcomes Tools for Specific Target Groups

• Alcohol misusers: Burns S – Alcohol Outcomes Spire: Scales and Guidance Notes; Alcohol Concern (2005)
• Homelessness: Burns S, Graham K, MacKeith J – The Outcomes Star; The London Housing Federation (2006)
• Young People (Schools) - useful sections on impact: Impact Measurement Pilot Peer Mentoring Report 2005-2006; Mentoring and Befriending Foundation (2006)
• Young People: Kirby P (w. Bryson S) - Measuring the Magic? Evaluating and Researching Young People's Participation in Public Decision Making; Carnegie United Kingdom Trust (2002)
• Adult Learners (with Mental Health Needs): James K & Nightingale C – Self-Esteem, Confidence and Adult Learning; NIACE (2005)
• Adult Learners: Westell T – Measuring Non Academic Outcomes in Adult Literacy Programs: A Literature Review; National Adult Literacy Database (2005)
• Adult Learners: Eldred J, Ward J, Dutton Y - Catching Confidence (Final Report); NIACE (2004)
• Refugees: Robertson V – Fresh Start: From Refugee to Employment; North Glasgow College (2006)
• Young People: Spirit Level CD ROM based Quality of Life Tool; Fairbridge
• Housing/Homelessness: St Mungo's - The Outcomes Measurement Star; St Mungo's
• Communities & Neighbourhoods (Neighbourhood Renewal): Walker P, Lewis J, Lingayah S & Sommer F – Prove It! Measuring the effect of neighbourhood renewal on local people; New Economics Foundation (2000)
• Communities & Neighbourhoods (Quality of Life Indicators): Higginson S, Sommer F, Terry A - Using Quality of Life Indicators in Local Governance; New Economics Foundation (2003)

Non-Target Specific Frameworks and Tools

• A comprehensive evaluation guide, but very much applicable to soft outcomes in many sections: The Evaluation Toolkit; Befriending Network (Scotland) (2007)
• Short report about the development of the Soft Outcome Universal Learning tool that has been developed for multiple target groups and is being rolled out through a series of workshops: Anderson C, Foster H & McKibben J – The SOUL Record: A New Tool for Measuring Soft Outcomes; The Research Centre (2006)
• A report about the development of a soft skills measurement tool for employment projects in Hackey; it took a multi agency research route towards the development of this tool: Balgobin E, Hutton C, Rees G, Weinstock W – Unqualified Success: A Report on the Development of a Soft Skills Measurement Tool for Employment Projects; Hackney Strategic Partnership (2004)
• The full, 116 page report on development of the Soft Outcome Universal Learning tool that has been developed for multiple target groups and is being rolled out through a series of workshops: Butcher B & Marsden L – Soft Outcomes Universal Learning; The Research Centre (2005)
• Website about a hands-on assessment and evaluation tool, the Rickter Scale: The Rickter Scale

Please email any further links that may be relevant or of interest to the Jane Zacharzewski.

Soft Outcomes: A Shared Approach

Soft Outcomes: A Shared Approach is a guide to help projects capture soft outcomes, and demonstrate them to funders with confidence. It is a result of research work within the programme that can be downloaded here.

Two versions of the Shared Approach are available plus a resources pack developed by Jim Simpson to accompany the eight steps of the Shared Approach.

The full version features colour photographs and text boxes throughout; it is suitable for colour printers. The Accessible version is almost identical in content, but prints on black and white printers and should be more suitable for those with accessibility needs.

Please note that you will need Adobe Acrobat to download the publications and resources on this page.

• Soft Outcomes: A Shared Approach - Full Version
• Soft Outcomes: A Shared Approach - Accessible Version
• Managing Soft Outcomes - A Resource Package to Assist Organisations to Develop an Evidence Based Approach to Managing Outcomes

The InDesign source material is available upon request under the Creative Commons Attribution-Non-Commercial-Share Alike 2.0 UK: England & Wales Licence.

Copies of the guide have also been printed in full colour - please contact Equal Admin if you would like to be posted copies.

Soft Skills and Outcomes Workshop

A workshop titled Soft Skills and Outcomes in and Beyond Equal Brighton and Hove - An interactive workshop took place on the 27th September 2006 at the Jubilee Library.

The aim of this event was to create a common set of working definitions and understandings. The eventual aim of this work is that development of skills and practices facilitated by EBH will last beyond the end of the programme and increase the capacity of partner organisations to deliver effectively and acquire funding.

Those taking part included:

• Simon Simpson (Simon Simpson Consulting) who spoke about a common standard for soft skills
• Harriet Cookson (Broadway London) who spoke about development of the ICAN package
• Marc Wood (Friends Centre, Brighton) who spoke about his work taking up the ICAN package on a local level

We received substantial feedback at the end of the workshop, which is helping us develop work based upon the stated needs of partner organisations.

Please find available for download the full workshop pack. Please note that the report distributed at the event has been updated and removed from this page. Aspects of this report are available in the three papers that are being produced.