In Praise of Non-Inferiority

Dear All (wonkish but intended for everybody to read and absorb):

27 Jan 2021 update: This topic now has its own YouTube video! As the ideas here are VERY confusing, I suggest you make the time to watch the video, read the newsletter, and then watch the video one more time. I have yet to find a short, simple way to explain this but I’ll keep trying!

There’s been a lot of recent discussion about “lessons learned from COVID-19.” Indeed, there are many valuable points: global cooperation, preparedness, supply chains, and so forth. As an example, I note the 17 Sep 2020 communique from the G20 Health Ministers (link): it was heavily focused on COVID-19 but it did note the need to deliver on “previous G20 commitments to tackle antimicrobial resistance.” All good stuff!

Among these important ideas, I would like to draw out today the lesson learned from COVID-19 about the importance of non-inferiority (NI) trials as tools for developing new antimicrobial agents of all types (antibacterial, antiviral, etc.) in advance of widespread need.

Unless otherwise cited, the text below draws from two papers that I co-authored in 2017 and 2019 (link and link, see full cites below my signature). In outline, there are five key ideas:

1. There are two types of trials: Superiority and non-inferiority (NI)
2. Infections can be lethal: Placebo is a bad idea!
3. Antibiotics are best developed before we need them!
4. NI trials are the public good that allows us to avoid COVID-19-like situations.
5. Superiority efficacy is (hopefully) unattainable, but superior utility is possible.

1) There are two kinds of clinical trials: All pharmaceutical products must (a) show which individuals can benefit from the product, (b) demonstrate a way to identify those individuals, and (c) document the benefit received from the product. Clear answers to these questions are required for product approval and acceptance. To achieve this, one of two types of trial designs can be used:

• In a superiority trial, the goal is to show that NEW is measurably superior to EXISTING or to a placebo.
• In a non-inferiority (NI) trial (sometimes formerly referred to as an “equivalence trial”), the goal is to show that NEW has efficacy similar (within the bounds of a pre-specified non-inferiority margin) to that of EXISTING.

2) Infections can be lethal: From a public health perspective, new antimicrobial agents with an improved microbiologic spectrum of activity should be developed before widespread bacterial resistance emerges. This desire leads immediately, however, to a paradox: although it is easy to demonstrate that a novel test agent has an improved spectrum both in vitro and in preclinical animal infection models, rigorous demonstrations of the NEW agent’s superior clinical efficacy over EXISTING drugs both are and should be difficult to implement on a routine basis in trials of human infections:

• Infections move quickly and can be fatal! As inadequately treated acute infections can be rapidly fatal and enrollment into a trial must often be undertaken empirically before culture results are known, it is obviously desirable that the control arm be predicted to be efficacious in all studies, including studies of potentially superior new agents.
• We always seek to give effective therapy to all patients! Indeed, if resistance is known or suspected to the control arm, then the control arm should always be adapted to offer some form of best available therapy that is predicted to be efficacious. In short, it is important from an ethical viewpoint that the trial make every attempt to use an efficacious control and not anticipate showing superiority relative to an ineffective or substandard control.

3) The time to develop new antimicrobial agents is before we need them: When we lack ANY effective therapies, placebo-controlled trials are possible … at least in theory:

• COVID-19 highlights the only exception to the prior rule! The only exception to the above-noted ethical imperative would be if there were either no efficacious options whatsoever for the infecting strain (or if all forms of best available therapy were meaningfully suboptimal). It is of course obvious that such a situation would imply a situation with grim public health implications.
• For (fatal!) infections, study enrollment is made challenging by both (i) the impact of infection control efforts and (ii) the desire to avoid placebo (link): Again COVID-19 has shown us that trials may not run as quickly as you’d expect. If local control measures are able to reduce the spread of the target infection (good for the community!), infected patients can become hard to find (bad for the study!). And the desire to be treated with something (anything!) can overwhelm the ability to enroll even in properly designed and ethical trials.
• And once you definitively have an effective therapy, the rules change! The window of opportunity to reliably design trials to show superiority because of a complete lack of therapeutic options would close with emergence of a new efficacious therapy. Would you be willing to be randomized to placebo if you were in the ICU with COVID-19?

4) NI trials are a public good: By using NI trials to develop new antibiotics in advance of the (next) pandemic, we can save lives and avoid panic. Think about how different the world would be right now if we (as a global community) had substantially completed the development a few years ago of one or more therapies and vaccines for the coronaviridae (SARS, COVID-19, etc.) based on measures of immune response and perhaps even controlled human infection models (CHIMs, link to NEJM editorial on this approach).

5) Superior utility is possible: Although superior efficacy is thus hopefully something we rarely see, superior utility is entirely possible. Oral rather than IV? Shorter course of therapy? Fewer associated side-effects? Better taste in a pediatric formulation? All of these are entirely possible and could be pursued — the only thing that is off the table is superior efficacy. Indeed, the ARLG (Antibiotic Resistance Leadership Group) is pursuing studies of this type based on the Desirability Of Outcome Ranking (DOOR) suggested by Scott Evans and colleagues (link).

—
There we have the lesson of COVID-19: Antibiotics are the #FireExtinguishersOfMedicine and need to be developed, purchased, and installed in our pharmacies around the world before the fire breaks out. NI studies give us the opportunity to make this happen – provided we are willing to pay for them in advance (and that’s a topic for another day).

In praise of non-inferiority!

Best wishes, John

Key papers:

• 26 May 2017 newsletter (link) on Rex JH, Talbot GH, Goldberger MJ, Eisenstein BI, Echols RM, Tomayko JF, Dudley MN, and Dane A. Progress in the fight against multidrug-resistant bacteria 2005-2016: Modern non-inferiority trial designs enable antibiotic development in advance of epidemic bacterial resistance. Clinical Infectious Diseases. 2017;65:141-6.
• 6 Aug 2019 newsletter (link) on Rex JH, Fernandez Lynch H, Cohen IG, Darrow JJ, Outterson K. Designing development programs for non-traditional antibacterial agents. Nature Communications. 2019;10(1):3416.

John H. Rex, MD | Chief Medical Officer, F2G Ltd. | Operating Partner, Advent Life Sciences. Follow me on Twitter: @JohnRex_NewAbx. See past newsletters and subscribe for the future: https://amr.solutions/blog/. All opinions are my own.