Brighter Futures
The Millfields Trust 'HQ'
Unit 29
237 Union Street
Plymouth
Devon
PL1 3HQ
Tel: 01752 294044
Mob: 07779 349557

Evaluation

Brighter Futures South West, in partnership with Tamar View Community Complex Ltd successfully secured funding through the Neighbourhood Renewal Fund (NRF) to deliver the pilot program in the three NRF designated areas of Barne Barton, North Prospect and Stonehouse. The program is funded until March 2008.

As part of the conditions of funding, the program is being externally evaluated by SBD Associates. This external and independent evaluation has given authenticity and credibility to the program model and the effectiveness of the intervention program as a whole. For those that choose to embrace the opportunity of real change, the program is successful in empowering young people to manage their feelings and emotions, improve their behaviour, mental health and wellbeing and take a more responsible position in their communities and society as whole with improved future prospects.

The evaluation information so far is based on a relatively small cohort of young people who have completed the program. Therefore, the information is limited at this time but substantiates that on the whole, the program is achieving what was originally predicted by Brighter Futures who wrote the program, based on evidence through our practical experience of this work with adults, although the program was redeveloped for for young people by Danny Biscombe of Brighter Futures South West.

We are happy to share the results of the evaluation of our program.

Brighter Futures: First Interim Evaluation

1.       Summary and Conclusion

a.         The Project is underway as planned and has met/ is meeting its milestones, though as with every project, there is a small degree of slippage with some of them. This should not affect the outcome of the Project.

b.         The Targets are being met with the exception of Individuals counselled. There is a good reason for this and we are recommending that the Targets be changed to formalise the new situation.

c.         The reason is that the referring agencies are asking the Project to help with groups of students who do not need the full 12 hour counselling sessions. These are students who are excluded from school and are in great danger of permanent exclusion. Three two hour group sessions per rotational group, are being put on to help them. 5 groups totalling 20 students have taken place so far.

d.         As the Project is a pilot project and demand led by the referring agencies we think it is entirely appropriate that the emphasis should change in this way, especially as the numbers helped will rise from the original 50 to 90 or so.

e.         Evaluation of the outcomes is as rigorous as it can be at this stage with the individuals having their behaviour marked both before and after; and the groups by whether they stay at school. Any impact on the Floor Targets is unlikely to be adversely affected by the change though demonstrating the link between the effect of the counselling and the Floor Targets is extremely difficult.

f.          The Project is well regarded by the referring and other agencies in the field as evidenced by the demand to attend the Professional Workshop where 22 have attended already (Target for project – 16) and a waiting list drawn up for another.

g.         A good start to a project that seems both to be meeting the needs of the subscribers and having a positive effect. Demand seems to be outstripping resources even ignoring that from outside the NRF areas. Sustainability could be an issue with many referring organisations stating they do not have the resources to pay for the service.

2.       Background

The Project

1.         The project is to set up a Community Intervention Programme in the three priority neighbourhoods of Barne Barton, North Prospect and Stonehouse to help reduce particularly youth related crime by modifying young people’s behaviour. It is hosted by the Tamar View Community Complex and is being delivered by Brighter Futures, who designed the program for young people. It was meant to be a two year project but in effect is an eighteen month one running to end March 2008, with a total cost including matched funding of £

Objectives

2.         The Bid document describes the Objectives in Section 2.4 as set out below. These have not been changed and seem unlikely to be.

Ø      One to one specialist directive counselling for perpetrators of Crime, Violent Crime, bullying, neighbourhood nuisance, abusive, aggressive and violent behaviour, anti-social behaviour and those excluded from school, to provide opportunities to manage their emotions and modify their offending behaviour. The service will be available to young people from the age of eight to twenty one years old,  both male and female through a referral system with existing service providers and agencies, including schools

Ø      Providing Anger Management Workshops for parents with children who have behavioural issues and provide training for professionals/workers, in the priority areas through partnership working with the referring agencies and organisations, taking a collaborative approach to reduce crime.

Ø      Publicising the service in existing community publications and using the press and media to maximise local knowledge and acknowledge the investment by funders

Ø      Report to existing steering groups of partners and referring agencies to ensure the aims and objectives are being met.

Ø      Work in a holistic and collaborative manner in partnership to find the most appropriate support and positive outcome for victims and perpetrators of crime, including attendance at multi-agency meetings and referring to other agencies where appropriate

Ø      Monitor and evaluate the effectiveness of the service and provide monthly and quarterly reports and an evaluation of the program at the end of the funding period

Ø      The Community Intervention Program will work safely within an ethical framework (RESPECT and BACP Principles), and receive regular professional external supervision.

The Evaluation

3.         The Bid document built in an ongoing evaluation element from almost the very start and with specific points in February, May, September and December 2007 and March 2008, which is also the timing of the Final Report though that may slip due to the need to receive the information on the final cohort going through the process.

4.         This first Interim Report is very much process focussed as there are few outputs to report. It is to reassure that the project is underway and on track and being carried out and managed in an appropriate manner.

5.         Interviews with referring agencies etc will not form part of this report as it is too early in the process as the first cohort of individuals at least should have completed the programme before that happens. Interviews with Pat Patel at TVCC and Danny Biscombe of Brighter Futures have taken place.

6.         An evaluation framework and methodology has been agreed with the project principles mainly based on “before” and “after” questionnaires. This is explained more fully in the Outcomes section below.

7.         We are conscious that this is very much a pilot project based on the best practice in this field, taking into account past experience of this work with other specialist practitioners.

3.       Outputs

8.         The programme has a basic set of Direct outputs which it is expected to achieve and a subset to be achieved by March 2007 which we examine in this Interim report. They relate to those affected by the project and are quite straightforward to measure. Because of the late start the end March 2007 targets are not fully met. We examine each of them in detail below.

Young People Individually Counselled

9.         The first cohort of 14 is underway with two having completed at the time of this Interim report. This is below the planned March 07 target. The overall target is 50 individual one to one 12 hour counselling sessions. This is unlikely to be met. The Bid Document (Para 2.5) however did anticipate that that format might not be appropriate for all.

Program Format Flexibility

We would not wish to restrict the format of the work to the above as being flexible to demand is critical to the success of the project.  For example, we already know that this project may be oversubscribed, from the feedback that we have received. Not all clients may require all twelve counselling sessions, therefore we could utilise the counselling hours to enable additional clients to access the service. We would therefore, utilise the time as effectively as possible to deliver either more counselling sessions or more workshops and/or training, where-ever there is greater priority. 

10.       What has become apparent is that the referring agencies are also seeking to use the service to help those who are not so extreme as to need the full 12 hour sessions. Many of these can be handled in group sessions of 3 or 6 hours. The project has therefore responded to this by increasing its group activities. Some individuals may be identified for further one-on-one assistance but given the limited resources the overall numbers seem likely to fall to 25.

11.       It is thought that around 10 Group sessions will be included in the project with an average of 6 members in each group. Thus the total number of individuals impacted by the project will rise from 50 to 90 but they will all not receive the individual 12 hour sessions. Five groups with 20 attendees have already taken place.

12.       Given that this is a demand led pilot we feel it entirely reasonable that the format of the project should change to reflect what the professionals on the ground in the referring agencies are saying is needed. In terms of sustainability if the project demonstrates that it can respond to that demand it is much more likely to be supported by the agencies after this funding ends.

We therefore recommend that the formal targets are modified to reflect this change in working.

13.       What we have started to discuss with the project however is how the effectiveness of the group sessions will be measured, as everyone is conscious that any change of tack within the project must not be detrimental to its impact on the Floor Targets. Our first thoughts in this area are set out in the next Section.

14.       The Agencies that have referred young people to date are –

·        Social Services

·        CAST Team – Plymouth City Council

·        Young Runaways Team – PCC

·        Youth Offending Team – PCC

·        Youth Inclusion and Support Program – PCC

·        Youth Inclusion Program – Youth Offending Team – PCC

·        Secondary Improvement Program – PCC

·        Schools (Primary and Secondary through Excellence Cluster, SENCOs and EWOs)

·        Parent Support Advisors – PCC

·        Positive Futures – PCC

·        Frederick Street Centre – Young People’s Services – PCC

·        Connexions

·        Harbour Young People’s Service

Workshops for Parents

15.       Two workshops have taken place with 15 parents attending. While the number of workshops target has not been met the number attending almost has. We are relaxed about the numbers as again these are voluntary and demand led. What is probable is that the overall total will not reach the target by March 2008 as if the number of individuals counselled falls then the number of parents is likely to follow suit – though some parents of those in group sessions might be added.

16.       There are feedback forms for the workshops. In our next report when the numbers are sufficiently robust we will analyse those responses to see how effective the parents thought the sessions were.

Workshops for Professionals and Workers

17.       This is very much a success story. One workshop has been completed to date with 22 attendees. This more than meets the project target. More people and agencies have expressed an interest and the project is confident that this target will be well oversubscribed.

18.       The organisations/agencies that attended the first seminar were also asked to feedback their views and again we will deal with those responses in a later report. The attending agencies/organisations for the first workshop were -

Public Health – NHS (2)

Positive Futures – Plymouth City Council (3)

YISP – Youth Intensive Support Program (4)

YIP – Youth Inclusion Program (1)

Plymouth City Council - Trainers (2)

Parent Support Advisors Project (2)

White Gold Project (1)

Starting Point Project (1)

ISSP – Intensive Support & Supervision Program (Youth Offending Team) (1)

SALT – Sexual Abuse Counselling Agency (1)

Money Advice Plymouth (2)

Behaviour Support Team (2)

19.       The feedback forms were all very positive with everyone stating that the workshop was of benefit to them and all asking for information on further or similar workshops.

Full Time Staff Employed

20.       This target has been met with the appointment of Danny Biscombe as the Outreach Worker and Neil Thacker in administration.

4.       Outcomes

a.         Impact on the Individuals

21.       The Outputs from the programme are set out above but it was also felt appropriate to try to measure the effect on the individuals taking part in the programme as it is their behaviour that will affect the contribution to the Floor targets – examined below. We are therefore dividing the 50 young people into cohorts depending on when they go through the programme and measuring its impact through a Criteria Assessment Tool (CAT) which has been especially devised for the programme by its Outreach Counsellor Danny Biscombe. This is not however an untried tool as it is heavily based on the Ahimsa Model already being used in Plymouth.

22.       As can be seen from Table 2 below it consists of a series of physical and emotional activities with an assessment of how prone the individual is to them. The intention is to score them at the beginning of the counselling; immediately after it finishes and then again after three months to see if the hoped for improvements have lasted. As evaluators we do not have direct access to the young people and are therefore dependent on the referring agencies and the cooperation of the young people to make this work. We do believe however it could be a very powerful measurement tool.

23.       We also think it will be instructive to see the effect on the cohort as a whole as well as the individuals and therefore Table 2 contains the numbers for each individual and Table 3 the consolidated figures for each of the actions for each cohort. This Interim report only deals with the first cohort of 14 and as all have not finished cannot capture the later data.