Evaluating what matters: supporting early-stage innovation across health systems

Health Innovation East’s real-world evaluation team is regularly asked to support system partners in evaluating early-stage intervention.

In this blog, we reflect on two recent projects, delivered in collaboration with  National Institute for Health and Care Research (NIHR) Applied Research Collaboration East of England (ARC EoE), a key partner to our organisation, where we have provided support to scope, design and conduct evaluations. The projects were at early stages of implementation and varied readiness for evaluation, requiring flexible approaches to support their needs.

We reflect on some of the challenges of evaluating early stage and cross-organisation interventions within complex and changing health systems. Importantly we also reflect on some of the impact and benefits of how we have worked with the integrated care boards (ICBs) to support immediate requirements for evidence and more broadly their requirements to become more research active.

The projects

Women’s Health Network – Bedfordshire, Luton and Milton Keynes (BLMK) ICB

The BLMK Women’s Health Network secured funding to deliver five pilot projects between 2023 and 2025), aligned with the Women’s Health Strategy for England (2022). The programme primarily sought to address service backlog and included new and existing services across primary and community care.

BLMK ICB required an evaluation to understand how the network was functioning and to generate early outcomes to inform a business case for continued funding.

High Intensity User (HIU) Programme – Cambridgeshire and Peterborough (C&P) ICB

Having secured Better Care Fund, this programme identified individuals with 10 or more visits with A&E departments in three hospitals across Cambridgeshire and Peterborough within a 12-month period.

The intervention aimed to reduce the A&E attendance through personalised support from link workers, and connections to alternative services such as housing, substance misused support, and primary care. The evaluation sought to understand the impact of shifting the flow of care from emergency to preventative care for this group of people and provide evidence for further commissioning.

Our approach

Health Innovation East, in collaboration with NIHR ARC EoE, supported these projects by resourcing evaluation expertise and capacity to work alongside clinical and delivery teams.

A key principle across both programmes was flexibility – adapting our approach to meet the needs of each system partners involved and as system needs changed.

For the BLMK Women’s Health Network, we worked closely with a Community and Sexual and Reproductive Health registrar trainee, providing advice and mentoring support to rapidly evaluate the pilots. While we initially set out to collaboratively develop an evaluation framework, adapting to partners capacity meant we went on to develop logic models for each pilot alongside the framework and to recommend and facilitate stakeholder conversations with the project leads. A mixed methods approach was key to getting early-stage insights from pilot leads where data was limited, and to building a better understanding of the data available to develop a realistic evaluation framework to guide on-going evaluation.

Within the HIU programme, we worked in partnership with C&P commissioners, sharing expertise across quantitative, qualitative, and health economics methods. By embedding evaluators within the system, we were able to build strong cross-sector relationships and deliver a robust interim evaluation that has identified early insights to inform programme development and identify data gaps that can be addressed in the final evaluation.

Challenges and what we learned

Evaluating early-stage interventions within complex health systems bring consistent challenges:

Early-stage interventions expectations need to be realistic in what evidence can be evaluated and when is most appropriate to evaluate. Ensuring early engagement is often highlighted as essential for appropriate scoping & design of evaluation. It may be more critical to bring the right support in at the appropriate time – too late and the opportunity to scope and shape data collection may be reduced, too early the intervention may not be sufficiently defined or availability of data limited. Within the BLMK Women’s Health Network and the HIU projects services had already started some data collection, in those cases it is more difficult to influence what data is collected.

“one of the challenges was definitely lack of availability of data and also where to find that data as well […] it was also like inconsistency with how well the data, the forms were actually being filled out […] there was definitely lack of consistency across the pilot in terms of submitting data.”

Systems and providers require a foundational understanding of evaluation and research methods so they know when to bring in support and can scope projects so they are achievable and address evidence gaps appropriately.

Integration and effective communication internally and across service delivery organisations is essential for aligned working across sectors and stakeholders, avoiding duplication, avoiding inappropriate demands and expectations, and to access cross sector data and stakeholders for evaluation. Often there is a key person within organisations that can be the ‘gatekeeper’ and link to data and people.

“For all evaluations when you are working in an NHS remit there’s a lot of time constraints in practitioners to engage with any evaluations…We [evaluators] were dependent on enthusiastic support from key stakeholders – instrumental in helping us to get data and participants for evaluation”

Senior leadership engagement and access is often a challenge yet these are often the people requesting the evaluation. Recognising the limited resource and pressures people are working within, finding ways to ensure information and actions are not stalled and expectations of evaluations and timelines are managed and realistic is important.

Impact

The impact from these projects highlighting benefits of the collaborations to support evaluation and applied research:

Emerging evidence – In the HIU project the interim evaluation has validated the early impact on workforce and patient flow, and the potential impact it could have as the programme reaches a broader application and larger cohort. For the BLMK Women’s Health Network the quantitative and qualitative data were key to helping inform the business case, as the trainee registrar reflected:

“I think it definitely had the impact of providing the independent, neutral analysis of things, taking a step back and actually looking how things are going in an unbiased way. I think we did manage to demonstrate that even though it was very early in the implementation process that there was impact.”

Provision of tools, guidance supporting on-going evaluation and providing transferable & scalable tools for future evaluations:

• A recommended evaluation framework for BLMK Women’s Health Network and the evaluation report used in the business case for further funding
• An interim report for HIU that is informing ongoing evaluation and work to build a cross sector Business Intelligence dashboard
• Mentoring and advisory support has facilitated up-skilling and capacity building for research and evaluation in the ICB and trusts – The trainee registrar at BLMK’s Women’s Health Network reflected on her improved knowledge of logic modelling, qualitative data collection and evaluation methods following our collaborative approach.
• Embedding evaluation expertise and practices in service delivery – the projects helped strengthen the applied research and evaluation capacity within local systems, helping to secure a sustainable research culture and expertise.
• Building strategic direction, planning and links to support applied research – through our networks and relationships across the health and care sector and our collaborations with experts within the ARC EoE themes we facilitated connections and building relationships between project teams and regional experts.

Why this matters

ICBs are expected to be research-active organisations and to evaluate the services they commission, using both quantitative and qualitative data, including feedback from staff, communities and people using services (NHS Strategic Commissioning Framework Nov 2025).

In a rapidly evolving healthcare landscape, evidence-informed decision-making is essential. Early-stage evaluation plays a crucial role in understand what works, for whom, and under what conditions – in rapidly evolving services it is all the more important that early impacts are evaluated.

Delivering robust, high-quality evaluation requires both expertise and considerable resource that Health Innovation East provides.

What’s next?

Each project has laid the foundations for ongoing evaluation and future scaling:

• BLMK has a robust evaluation framework and business case to support programme continuation
• The HIU programme continues to evolve, with ongoing evaluation and development of a cross-sector BI dashboard.

Together, these projects demonstrate how flexible, embedded evaluation can not only generate timely evidence, but also build lasting capability within health and care systems.