'Arts and Minds', 'Confident to Earn' and 'Learning Teams' Evaluation - Executive Summary

Methodology

3.      The evaluation was conducted using a range of methods. For each project, we adopted an identical methodology:

• Initial contact with Director of Education (or equivalent) to identify the key relevant personnel for the project

• Scoping meeting with identified contact personnel for background briefing and to identify the potential research sample

• Review of documentation, and ‘set-up’ of qualitative work

• Qualitative research (focus groups, interviews, classroom observations)

• Questionnaire survey – conducted in tandem with the qualitative work

• Analysis of qualitative and quantitative findings, and presentation of emerging themes to local authority seminar

4.      Overall we conducted 28 focus groups, 73 face-to-face interviews, and 10 classroom observations. In all, we spoke to 101 pupils, 13 parents, 54 teachers, 54 school managers, 11 policy & support staff, and 11 partners. 

Findings

Learning Teams

5.      The Learning Teams project ran in East Lothian from February 05 to June 07. During the project, two cohorts of primary school teachers (one cohort of 28, the other of 30), were trained and supported to implement formative assessment approaches using a Learning Teams methodology . One day of training was given, followed by two follow-up days to reflect on progress. An educational consultant was employed to support and guide the project.

6.      The project aimed to explore the potential of Learning Teams within the East Lothian education community. Teachers were encouraged to become action researchers and reflective practitioners. The project aimed to increase learner attainment and achievement by enabling teachers to move from an ‘activity based’ to a ‘learning based’ culture.

7.      The Learning Teams project was well organised and well delivered, and represented good value for money. The project has been sustained. It had a positive impact on teaching practice, and facilitated professional development particularly in relation to the implementation of formative assessment. Pupils also felt the benefits of an approach which was more child-focused.

Confidence to Earn

8.      The Confidence to Earn project in East Renfrewshire ran from January 2005 to December 2006. It had two strands. The primary school strand, Confident to Earn, was based on research undertaken by an educational consultant, with a ‘writing group’ of teachers producing a teaching pack on risk-taking primarily for P6 and P7. The secondary school strand, Thinking Adventures, was led by another educational consultant, and involved the production of a teaching pack using guided Socratic discussion to develop critical thinking and citizenship, primarily for use with S1.

9.      The project aimed to create curriculum materials that united the areas of citizenship, creativity and enterprise. It anticipated developing higher academic performance, more creative pupils, a more enterprising culture, and better citizens.

10.      The Confidence to Earn project was implemented effectively, despite objectives that were not well defined. It has been sustained through staff capacity building and securing further local authority investment. Its benefits were experienced particularly by staff and pupils at the primary school level, where the project has become embedded in mainstream school activity. Overall it represented reasonable value for money.

Arts and Minds

11.      The Arts and Minds project ran from January 2005 to December 2006. It was based in the St Mungo’s Learning Community in Glasgow. The project built on a range of school and community music initiatives which had been running under the auspices of the Bridgeton Music Project since the 1990s. It was composed of three inter-related strands:

• Specialist music input for nurseries and P1 - 2, based on the ideas of Kodaly. A systematic programme was developed across all five primary schools (P1 - P2) and all three nurseries, using specialist music tutors from the Royal Scottish Academy of Music and Drama (RSAMD) supported by students on placement

• A dedicated Listening Well Programme at Glasgow Listening Centre based at Sacred Heart Primary School for children within the St Mungo’s Learning Community with learning, emotional and behavioural difficulties. The programme is based on the work of Tomatis.

• A ‘Room 13’ project (based on a model developed in Caol primary school in Fort William) for pupils in P5 - 7 at Sacred Heart Primary School, with a musician in residence, supported by placement teams from RSAMD.

12.      The project aimed to establish programmes to demonstrate the scope of music to improve listening, language and literacy skills, to promote students’ creativity and shared enterprise, and to raise and sustain aspirations for achievement and attainment within a Learning Community..

13.      The Arts and Minds project was a challenging project to manage, with objectives that were not well specified. The three strands of the project experienced significantly varying levels of success. Key achievements were the implementation of an early years musical education programme across the Learning Community, and a constructive partnership with staff and students from the RSAMD.  The project has been partially sustained. Investment in staff capacity building would have increased the project’s value for money

Conclusions

14.      Our evaluation has shown that Government investment can be used successfully to support innovation at a local level. However, in order for this investment to represent good value for money, the ‘rules of engagement’ between the commissioning body and the innovation project need to be clearly defined. Project objectives should be well defined, realistic, and capable of evaluation; baseline data should be available in a suitable format so that progress and attainment can be measured; and project management arrangements should be in place so that progress can be monitored and adjustments made as the project develops.

15.      We conclude that innovation is most likely to succeed where the innovation ‘goes with the grain’ of national and local policy. This ‘fit’ with policy direction will give the innovation credibility, and will allow all participants to understand its relevance.

16.      Those seeking funding for innovation should be clear about the research evidence (or lack thereof) which supports their proposal.  Whilst by definition innovation projects are not fully adopted within the mainstream, and the evidence to support them may therefore be patchy, those applying for funds must demonstrate their understanding of the state of knowledge of the evidence base for the innovation, and must show how their project will build on that evidence. Funders should ensure that their criteria for awarding funds include an assessment of the extent to which applicants demonstrate understanding of the evidence base for their proposal.

17.      For an innovation project to succeed and be sustainable, both the idea underpinning the innovation and the implementation of the innovation need to be sound. This requires effective groundwork to understand how the innovation will work in the school context, as well as effective project management. The three tiers of support – from the local authority, the school, and peers – need to be in place. Project coordinators should be focusing on sustainability from the outset, and should ensure that the achievement of project objectives are regularly reviewed so that adjustments can be made on an ongoing basis.  There should be transparency and clarity in financial management.

18.      Classroom teachers can be effective ‘agents of change’, and approaches which have been designed and tested by teachers can have particular credibility. In some cases, outside experts can provide an important impetus to catalyse change or to assist with an innovation. In this latter case, attention should focus from the outset on how the expertise of the ‘expert’ can be transferred into, and sustained within, the school itself.

19.      Innovation requires effective Continuing Professional Development to support it. Moreover, innovation is most likely to work where there is clear evidence of positive impact on pupils, and this is visible quickly to teachers. The positive response of pupils can provide a strong motivation for teachers to change their own professional practice, thereby creating a ‘virtuous circle’ of reinforcement and improvement.  

20.      Developing a positive view of risk taking in the context of teaching and learning has been a consequence of the focus on innovation. This has been important both for children and teachers. It can be seen to support enterprise and citizenship within the curriculum, and to assist with the professional development of teachers.

21.      Finally, an important ‘by product’ of innovation has been the creating of a ‘space’ for reflection for both teachers and pupils. This an essential pre-requisite to changes in professional practice and pupil learning. It is particularly important within the constraints of a busy classroom, if future learning and teaching is to be improved for the benefit of pupils.

Recommendations

22.      The recommendations listed below draw out the main messages for commissioners of innovation projects, for those applying for innovation funds, and for those who have strategic oversight of innovation investments.

23.      For commissioners, applicants, and managers we make the following recommendations

Recommendation 1

All innovation projects should identify SMART (specific, measurable, achievable, realistic, timed) objectives; should demonstrate understanding and knowledge of the relevant evidence base underpinning the innovation; should specify how baseline data is to be collected, and the type of evaluation method to be used to gauge progress against this; should set out good financial, monitoring, and management arrangements.

Recommendation 2

In order for innovation to lead to embedded practice, projects should indicate the steps which will be pursued to achieve sustainability. They should specify how effective support will be secured from peer practitioners, from school management, and from the local authority. Clear plans for staff capacity building should be outlined. There should be clarity about how any innovation which requires ‘external’ or ‘expert’ support can be sustained in the longer term (including plans for the transfer of skills to internal school staff). They should consider how any assets accumulated during the project will be handled.

Recommendation 3

Innovation projects should show how they are congruent with local and national policy objectives. The training of staff should emphasise this congruence so that staff are able to understand how the innovation contributes to the achievement of wider school objectives.

Recommendation 4

As far as possible, innovation projects should capitalise on the enthusiasm and commitment of staff, and should proceed by ‘voluntarism’. This is consistent with the spirit of innovation funding which seeks to go beyond the normal requirements of the curriculum. It also recognises the capacity of teachers to become ‘agents of change’ within their own school environment.

Recommendation 5

Applications should not only present the arguments for the innovation idea, but should also describe in detail how this innovation can be implemented within the busy school curriculum. Thinking this through in advance will enable any potential difficulties to be anticipated early, and measures developed to minimise their impact. The Project Coordinator has a particular role in this regard, and needs to be involved in the project throughout its development, implementation, and embedding.

Recommendation 6

Innovation projects should recognise that the nature of innovation requires the taking of calculated risks. The uncertainty inherent within the innovation project underscores the need for projects to build in space for reflection, through which the value (or otherwise) of the risks taken can be assessed.

PDF file: Evaluation of 'Arts and Minds', 'Confident to Earn' and 'Learning Teams' project report (702 KB)

• Email this page
• Print friendly format
• Digg this
• Save to del.icio.us