Evidence That Counts: NICE ESF

Innovation gets people excited. Demos impress, pilots spark enthusiasm, and small wins generate momentum. But in the NHS, what really determines whether a technology survives beyond its pilot is evidence. Not just any evidence — but evidence that convinces clinicians, procurement teams, commissioners, and regulators that the system is safe, effective, and value for money.

That's where NICE's Evidence Standards Framework (ESF) comes in. It sets the bar for what "good evidence" looks like, depending on the risk of the technology and the claims being made.

The Principle: Risk Dictates the Bar

The framework is simple: the higher the risk, the higher the standard of evidence.

• A tool that helps with clinic scheduling does not need the same level of evidence as one that influences treatment decisions.
• An AI diagnostic algorithm making high-stakes predictions must clear a much higher bar than a basic symptom checker.

This principle seems obvious, but many digital health projects falter here — suppliers underestimate what's required, while Trusts underestimate the resource burden of helping to generate it.

Understanding the Evidence Tiers

NICE divides evidence into three broad levels:

Basic evidence is about demonstrating usability and accuracy in small, controlled settings. Think: usability testing, early clinical input, or checking that calculations are correct.

Intermediate evidence requires broader validation. That might mean multi-site studies, comparator groups, or early health economic analysis. It's about showing the tool works beyond a single ward or Trust.

Advanced evidence is the gold standard: randomised or controlled trials, large datasets across multiple settings, and full health economics. This is what's needed for high-risk diagnostic or therapeutic tools.

The Operational Reality

Here's the difficult truth: evidence collection is hard.

Time — Basic evidence might take a few months. Intermediate can take one to two years. Advanced evidence often requires several years.

Cost — Proper studies, especially with comparators and economic modelling, are expensive. Funding needs to be planned from the start.

Trust burden — Evidence isn't collected in the background. It requires clinician time, patient recruitment, and access to data — all scarce resources in an overstretched NHS.

That's why so many pilots stall. It's not that the technology doesn't work; it's that the pathway to credible evidence is longer and heavier than expected.

Whose Evidence Is It, Anyway?

Different stakeholders rely on different kinds of evidence:

• Clinicians want to know the tool is safe, accurate, and fits their workflow.
• Procurement teams need assurance of reliability and value before signing contracts.
• Commissioners — Integrated Care Boards (ICBs) — look for population-level outcomes and economic value.
• Regulators — the Medicines and Healthcare products Regulatory Agency (MHRA) — set requirements for technologies making diagnostic or therapeutic claims.

If evidence generation doesn't consider all these audiences, adoption slows — even if the technology itself is sound.

Getting Evidence Right

So how do we avoid the trap of pilots that never scale? A few principles help:

• Be honest about timelines — Don't promise a 6-month pathway when the evidence will take years.
• Plan funding early — Decide upfront who pays for studies and evaluations.
• Design studies properly — Match study design to the risk and tier: observational is fine for low-risk, but high-risk systems need stronger designs.
• Seek independence — Supplier-led studies risk bias. Involving independent evaluators or peer review strengthens credibility.
• Frame evidence as protection — Not bureaucracy. Evidence prevents harm, wasted investment, and reputational damage.

Evidence isn't an afterthought. It's the bridge between innovation and adoption.

In our next post, we'll connect this to the critical decision point: is your system a medical device or not? The Fork: Non-Device vs Software as a Medical Device (SaMD).