The US Food and Drug Administration Commission Stephen Hahn on Monday apologized for overstating the effectiveness of convalescent plasma as a treatment for COVID-19, after the agency faced backlash for Hahn’s claims that the treatment could reduce mortality by as much as 35%, made during an announcement of the FDA emergency approval of the therapy on Sunday.

“”I have been criticized for remarks I made Sunday night about the benefits of convalescent plasma. The criticism is entirely justified,” Hahn tweeted Monday.

Hahn had described the treatment as  “groundbreaking” at a much-trumpeted White House press conference, announcing the FDA authorization for the use of the antibody-rich blood plasma, collected from recovered COVID-19 patients. In his statement, Hahn had claimed that the FDA-reviewed data showed that “out of 100 people with COVID-19, 35 would have been saved because of the administration of plasma”.  US President Donald Trump and Health and Human Services Secretary Alex Azar also both repeat the claims of a supposed 35% reduced mortality risk, with Trump describing it as a “tremendous number.”

Leading scientists called that a “gross misinterpretation” of the data.

The debate centered around the findings from an observational study by researchers at the Mayo Clinic, which looked at survival outcomes for over 35,000 patients in the US who had received the convalescent plasma treatment. The study was published on the preprint server MedRxiv, and is still awaiting peer review.

The study found a 7-day mortality rate of 8.9% in seriously ill patients who received plasma with high levels of SARS-CoV-2 antibodies. In comparison, the 7-day mortality rate in seriously ill patients who received plasma with low levels of antibodies was 13.7%.

That means the so-called 35% reduced risk in the Mayo Clinic study is reallly a “35% change in small numbers,” according to Harvard epidemiologist William Hanage.

“So it’s like, going from 8% of people showing improvement to 13% of people showing improvement,” Hanage told reporters on Tuesday. “And that’s a sort of statistical quirk in the way that you have interpreted the outcomes of these things.”

Additionally, the Mayo Clinic study only compared mortality outcomes in patients who had received different levels of the antibody treatment. It did not include a control group – meaning it was not able to compare mortality outcomes in patients who had received plasma as compared to outcomes in  patients who had not received any plasma at all.

“What I should have said better is that the data show a relative risk reduction not an absolute risk reduction,” Hahn admitted in his tweet.

Experts said even the retraction, however, was still full of holes. “You need to correct the 35 lives saved per 100 sick with COVID-19 so that people understand that was absolutely wrong, Steve…There is no evidence at this juncture to support any survival benefit…We need RCTs,” tweeted Eric Topol, director of the Scripps Research Translational Institute, after Hahn issued his retraction.

Already on Monday, World Health Organization Chief Scientist Soumya Swaminathan cautioned against premature optimism over the convalescent plasma treatment, telling reporters the evidence supporting the treatment was still “inconclusive” and “very low quality.” More randomized control trials, which include well-designed control groups, must be performed.

“We recommend that convalescent plasma is still an experimental therapy. It should be continued to be evaluated in well-designed randomized trials,” Swaminathan told reporters.

She noted a range of challenges for the treatment, including inconsistencies in the level of antibodies that might be delivered by the plasma taken from other COVID-19 patients, as well as technological challenges faced in collecting blood from donors and making transfusions. Other WHO officials noted that the procedure can also lead to significant side effects.

Image Credits: Official White House Photo by Shealah Craighead.