Transform Ageing: What we’ve learnt about learning

Transform Ageing is a pioneering programme taking a community and design led approach to improve people’s experience of ageing. It brings together people in later life, their friends, family and carers, social entrepreneurs and public sector leaders to define, develop and deliver new solutions that better support the needs and aspirations of our ageing communities.

In Transform Ageing, we wanted an evaluation approach which not only showed us what worked and what didn’t, but also created space for learning and changing along the way. To do this we brought together theory-based and developmental evaluation. This was a new approach for us and we learnt a lot as we went. As more organisations in the sector respond to complexity with innovative and agile programmes we thought this learning would be helpful to share.

So far, we have identified five key lessons from this process that we would consider before setting out on a similar path again:

1. What scope is there to make changes to the evaluation during the programme and who would need to be involved? We updated some of our original indicators to reflect routes to market for social entrepreneurs that our original framework missed. Insights were brought together from public sector leaders, people in later life, social entrepreneurs and programme staff, to drive the changes that were made.
2. What does using an innovative approach mean for the programme’s objectives, reporting and timelines? Embracing the possibilities provided by innovation in responding to complexity also means being flexible about where the programme is going. Being able to review objectives and reporting had the support of our funder (The National Lottery Fund) from the beginning, which meant we could engage in that process with confidence.
3. How will learning be documented and who should it be shared with? The feedback we received from staff was that the process of capturing and sharing learning created two opportunities for reflection in the programme. We brought people together across different teams and shared learning back with them, as well as with the programme’s governance.
4. How can learning activities be integrated with other programme activities? Learning can feel like a luxury among the demands of delivery. Wherever possible we have run learning alongside other activity in the programme (e.g. team meetings), so it feels integrated to the programme rather than an extra task.
5. How could evaluation get in the way of innovation and what can we put in place to avoid this? We have tried to design an approach that meets the needs of the programme, is deliverable with the resources available, and that has the buy-in of the people involved. If these elements are missing, it is just as possible that the evaluation would be an obstacle to innovation.

Anyone who has been involved in the third sector in the last 20 years is likely to be familiar with theory-based evaluation; it is the organisation of evaluation activity around a theory of change. This helps to articulate how an intervention contributes to an overarching goal and what is expected to change along the way. It presents a pathway which neatly links activities to results and is useful for looking back over a programme to understand what worked and what changed along the way.

Theory-based evaluation is less useful for capturing unexpected outcomes or when objectives can only emerge as the programme progresses. When we started Transform Ageing, there was a lot we didn’t know about how it was going to work. The programme brought together groups of people in later life, their friends, families and carers, social entrepreneurs and public sector leaders to create six design briefs. We needed an evaluation which could adapt to the results of this process and the unique challenges and opportunities in the different areas it was being delivered in.

Beyond that, we also wanted to be able to capture what was working and what might change as the programme developed, and to feed this back to the delivery team and governance. Rather than being a way of monitoring progress, this put learning into the hands of people who were empowered to use it and make changes to improve the programme as it progressed.

To meet these twin demands of accountability and learning we have brought together theory-based and developmental evaluation. In practice this means that whilst we have a theory of change, we review it at regular intervals to reflect the learning we are collecting. To organise our learning, we use a learning framework which guides us towards the things we think might emerge.

We capture learning through reflective sessions, feed it back to staff in learning papers, and test and review our theory of change in light of what we find.

Put like that, it all sounds incredibly straightforward. There are however challenges that we encountered on the way. For example, as we realised how much we were learning from the programme, the original plan of an interim and final report didn’t seem to the best fit for what we wanted to say. Being able to update our reporting schedule to include more targeted papers regularly throughout the programme meant we could respond to new outcomes as they emerged.

Bringing together people from different teams in both the reflective sessions and discussions of learning papers helped to develop a shared sense of the work and empowered those involved to engage with change confidently. The feedback from delivery staff has been that the evaluation felt like a tool for growing and improving the programme rather than simply a reporting process.

We have taken this learning and put it together with some of the detail on how we designed and use the evaluation. We hope those considering how to evaluate innovative programmes find it useful.

Download the full research paper below.