Non-Inferiority Trials: Rethinking How We Compare Treatments with Net Treatment Benefit

What is a non-inferiority trial?

Non-inferiority trials are randomized clinical trials designed to determine whether a new treatment is not unacceptably worse than an established treatment.

Unlike a superiority trial, which aims to demonstrate that one treatment is better than another, a non-inferiority trial aims to show that any difference between the treatments remains within a predefined non-inferiority margin.

This approach is widely used when a new treatment may offer other advantages, such as reduced toxicity, a shorter treatment duration, easier administration or lower cost, while providing similar efficacy to the established treatment.

But does a non-inferiority trial always provide the right framework for answering that question?

A 2024 The Lancet Oncology Personal View, “The Tyranny of Non-Inferiority Trials”, argues that it may not. The paper was co-authored by Marc Buyse.

Read the full paper in The Lancet Oncology

The problem with conventional non-inferiority trials

The fundamental issue is that non-inferiority trials can place disproportionate emphasis on a single primary outcome.

For example, imagine a new treatment that:

• provides very similar overall survival to the standard treatment
• causes fewer or less severe toxicities
• requires fewer treatment visits
• costs less
• is easier for patients to receive

A conventional non-inferiority trial may still fail if the confidence interval around the primary efficacy outcome crosses the predefined non-inferiority margin.

The result can be a paradox: a treatment that appears very similar in terms of survival, but is substantially better in other outcomes that matter to patients, may not be considered an acceptable alternative.

This is particularly relevant when evaluating less intensive treatment regimens, such as lower doses, less frequent administration or shorter treatment durations.

The Lancet Oncology paper highlights examples where survival curves between treatment groups were almost superimposable, yet non-inferiority was not formally established. The authors argue that the statistical framework can therefore make potentially more patient-centred treatment strategies unnecessarily difficult to adopt.

Non-inferiority versus superiority: does timing determine the question?

There is another issue with the distinction between superiority and non-inferiority trials.

The same comparison between two treatment regimens can effectively be classified differently depending on when the trial takes place.

Before a treatment is approved, a trial comparing a more intensive treatment with a less intensive alternative may be considered a superiority trial if the new treatment is expected to provide greater benefit.

Once the more intensive treatment becomes the approved standard, however, a trial testing a less intensive alternative may be treated as a non-inferiority trial.

The statistical burden can therefore change because the regulatory status of one treatment has changed, rather than because the clinical question has fundamentally changed.

The authors argue that randomized trials should instead be viewed more simply as comparative trials, with the objective of determining which treatment is better for patients.

Why the primary endpoint can be a problem

A typical non-inferiority trial is built around a single primary outcome, such as overall survival or disease-free survival. However, treatment decisions rarely depend on only one outcome. Patients and clinicians may need to consider:

• survival
• disease control
• serious adverse events
• treatment-related toxicity
• quality of life
• treatment duration
• treatment burden
• frequency of hospital visits
• cost and access

A treatment can therefore be worse on one outcome while being better on several others.

The question becomes: how should these competing outcomes prioritized?

This is where Net Treatment Benefit (NTB) can provide a different perspective.

How Net Treatment Benefit can provide an alternative to the traditional non-inferiority framework

Net Treatment Benefit (NTB) is a way to measure treatment effect based on Generalized Pairwise Comparisons, a statistical approach that allows multiple outcomes to be considered according to a predefined hierarchy.

An important feature of NTB is that outcomes are prioritized and pre-specified. This makes it possible to reflect the clinical importance of different outcomes in the statistical analysis.

Rather than asking only whether an alternative treatment remains within a non-inferiority margin for one primary outcome, NTB evaluates the overall balance of benefits and harms across the outcomes that matter.

For example, a hierarchy could prioritize:

1. Overall survival
2. Severe toxicity
3. Quality of life
4. Other clinically relevant outcomes

Patients in the two treatment groups are then compared pair by pair according to this hierarchy.

If one patient survives longer, the comparison is determined by survival. If survival is similar, the comparison can move down the hierarchy to the next clinically relevant outcome, such as severe toxicity.

The result is an overall measure of how frequently one treatment provides a better outcome than the other, taking the full hierarchy into account.

This approach is different from simply testing whether one treatment falls within a predefined non-inferiority margin.

NTB for comparative clinical trials

One2Treat’s NTB methodology can be used to analyze randomized clinical trials with hierarchical composite endpoints, where multiple outcomes contribute to the overall assessment of treatment benefit.

It can be particularly relevant when evaluating:

• less intensive treatment regimens
• dose optimisation strategies
• shorter treatment durations
• alternative treatment schedules
• treatments with different benefit-risk profiles
• trials where toxicity or quality of life are important considerations

Rather than allowing a single primary endpoint to dominate the assessment, NTB provides a structured way to evaluate the totality of clinically relevant outcomes.

Frequently asked questions