Two leading researchers who have raised questions about the reliability of Sputnik V’s vaccine efficacy ratings across age groups shared their concerns with Health Policy Watch. One called the results “impossible” and “very concerning”.

More than 70 countries have approved the use of Sputnik V, Russia’s COVID-19 vaccine, based on the reported 91.6% efficacy across nearly all age groups in the first part of its Phase III trial published by The Lancet in 2021. 

But a number of scientists have since raised a red flag about the reliability of the published data. 

In exclusive interviews, two of the leading critics told Health Policy Watch that such consistent efficacy is not only nearly impossible but also unmatched by any other vaccines. 

These findings “may have substantial implications for the utility of the vaccine and thus regulatory approval and the ongoing use to prevent COVID-19,” wrote Dr Kyle Sheldrick, author of the most recent report, published in  American Therapeutics.

In a recent interview with Health Policy Watch, Sheldrick said that data on the Sputnik vaccine as reviewed in independent analyses, such as one published by the Mexican Ministry of Health, is “very reassuring that this is a vaccine that works.” 

Another vocal critic, the Italian researcher Enrico M. Bucci, echoed that, telling Health Policy Watch in an interview that Sputnik is essentially a combination of two proven vaccine delivery platforms, the Chinese CanSino vaccine and Johnson & Johnson dose.    

But the question still remains, the researchers said: does the Sputnik vaccine work as well as its developers have claimed? 

Concerns about transparency and homogeneity of results 

Concerns about a lack of transparency with regards to aspects of the clinical trials for Sputnik V, as well as the unusual homogeneity of results among different age groups, have been raised about Sputnik since Russia first approved the jab back in August 2020. At that time, it was the first vaccine to receive a green light from any country’s regulatory agency. 

To date, the World Health Organization, European Medicines Agency and the US Food and Drug Administration have not authorized the vaccine, despite repeated statements by Russian officials and representatives of the Russian Direct Investment Fund (RDIF) that funds and sells it that all necessary data has been submitted to those bodies. 

In Sheldrick’s recent paper, he asserts that “the results contained with the phase-III RCT of Sputnik by Logunov et al showed a distribution [among age groups] inconsistent with what would be expected from genuine experimental data.” 

He was referring to the fact that the reported Sputnik results showed equal efficacy amongst all age groups. 

“It is our opinion that it is not possible for a journal or reader to have confidence in the results, and the article should be thoroughly investigated, including immediate release of anonymized individual patient data to an unbiased statistical expert. If the authors are not willing to do this, the paper should be retracted.”

Sputnik results ‘too perfect’

Specifically, Sheldrick and his team performed a simulation study that assessed the statistical probability that the results for different age subgroups participating would fall in the results ranges reported for the Sputnik vaccine in The Lancet article. They compared their findings for Sputnik with those of other US and Australia-approved vaccines, including AstraZeneca, Johnson and Johnson, Moderna and Pfizer. 

To do this, they ran statistical simulations for all of the vaccines 1,000 and then 50,000 times – so as to yield probability factors for the likelihood that results by age subgroup would line up exactly as they did in the real, reported results of the trials. 

“We used study-wide efficacy and infection rate for all age groups,” the paper explained. “We recorded the observed vaccine efficacies in each age group and summated how many simulations had all observed efficacies fall within the range of efficacies described in the relevant article.

“We calculated how many times, on average, a trial would need to be repeated to obtain results that fell within the range of efficacies from the relevant published article,” the authors reported. 

Their findings: The simulations showed that each of the vaccines except Sputnik has a range of efficacy results, by age subgroup, which is not unexpected, when considering the prevailing infection rates in the country at the time,  number of participants, and reported study-wise efficacy.  

“You would only have to repeat the studies two or three times to get a similar result for these other vaccines,” Sheldrick explained. “For Sputnik, you would have to repeat it 3,800 times to get results as perfect as they claimed.”

For instance, in the 1,000-trial simulation for the AstraZeneca vaccine, in 23.8% of simulated trials, the observed efficacies of all age subgroups fell within the efficacy bounds for age subgroups in the published article, according to the report. For J&J: 44.7%, Moderna 51.1%, Pfizer 30.5%. For Sputnik, the result was 0.0%. 

“In 50,000 simulated trials of the Sputnik vaccine, 0.026% had all age subgroups fall within the limits of the efficacy estimates described by the published article, whereas 99.974% did not,” it said in the report. 

Australian scientist: ‘Very uncommon to raise accusations against Lancet’

Sheldrick said that it is improbable that the Russian researchers “just got very lucky,” a notion that has been seconded by a number of other scientists in the US, Italy, Europe and Australia. 

“It is very uncommon to have accusations such as this raised against papers in journals as big as The Lancet,” Sheldrick told Health Policy Watch, “maybe because concerns are rare or because we don’t have a culture of double-checking. At first I thought I was just being suspicious.”

But after completing his study: The combination of the rapid approval timeline, the lack of availability of the data and the unexpected results “are very concerning to me.” Although he said he still believes that the Russian vaccine is likely to be at least somewhat effective against fighting COVID.

“Just because the data is not genuine, does not mean the vaccine does not work at all,” he stressed. 

Time and political pressures could have led to shortcuts 

Russia, which only offered its citizens Sputnik, is experiencing a slight increase in cases like the rest of the world, but at a much slower pace, according to official data.

The Reuters COVID-19 tracker, for instance, reported only 15 infections per 100K people in the last seven days. That as compared to 385.23 infections and 324.38 infections per one million people in nearby European countries of Estonia and Latvia.

But Sheldrick said that given the lack of transparency seen in other areas, he is unsure whether one can trust the daily cases as reported by the Russian government.  

He also stressed that he and his colleagues had been attempting to raise its concerns about the Russian researcher in private long before the country’s invasion of Ukraine, so the recent paper has nothing to do with the war.

“We raised these concerns with The Lancet before the invasion occurred,” he told Health Policy Watch. Some people like to link the two. I try to stay out of the political side.”

So why does he think that the Russian scientists would have fudged the data, assuming that they did? Time, Sheldrick said. 

“There was internal pressure to produce results before other vaccines,” he said. “Also, the vaccine was already being given to the general public. If you had by random chance found low results in one group, this would have looked particularly bad.”

Not the first to raise concerns

Sheldrick is not the first to raise concerns over Sputnik’s Phase III data. 

Within days of  Sputnik publishing its Phase I/II data in The Lancet, Italian researcher Enrico M. Bucci published a “note of concern” on his website about the data. 

Bucci and his team also were concerned that the homogeneity of results for different age groups, as reported in the Sputnik V trials was improbably high. They estimated that  the probability of results falling within the ranges reported in The Lancet study was less than 0.1%. But they had concerns beyond that as well. 

“We noted several gross inconsistencies, including several data points from different experiments which look identical, as well as inconsistencies in the number of enrolled patients and some more problematic data,” Bucci explained to Health Policy Watch. 

“With 15 other colleagues from several international institutions, we signed a letter asking for access to the full data set documenting the results purportedly obtained in the Phase I/II study,” Bucci recounted to Health Policy Watch. 

“In disregard of our and others’ requests, access to the data set used for the study publication was never granted.”

A version of that letter was ultimately published by The Lancet in September 2020 under the title “Safety and efficacy of the Russian COVID-19 vaccine: more information needed.” 

In it, Bucci stressed that “while the research described in this study is potentially significant, the presentation of the data raises several concerns which require access to the original data to fully investigate.”

Sputnik: All has data been double-checked

Sputnik immediately responded to Bucci’s points, stressing in a formal letter that the data had all been “double-checked.”

“Some data that repeated itself was what elicited the greatest number of questions from Bucci and his colleagues: nine of the volunteers from day 21 to 28 in the vaccination process registered antibody indicators that were completely identical,” Sputnik wrote on its website. “Logunov stated that in small-sized groups of test subjects this possibility ‘cannot be ruled out.’

“‘It is possible that immune system indicators could reach the kind of plateau that we observed during the research,’ the letter states. Logunov also stated that all of the data obtained by scientists during the experiment was double-checked.”

But Bucci said that it was not only the Phase I/II study that raised eyebrows. 

There have now been three Lancet articles on Sputnik, all of which were problematic and subjected to several partial corrections after the errors were flagged, including the most recent Phase III study. 

Errata were published by The Lancet on 22 September 2020; 7 January 7 2021, 9 January 2021, 20 February, 2022 and finally on 11 June, 2022 – the latter in response to an Argentinian paper on Sputnik.  

‘Doubts about the reliability of data

“The published corrections did not address most of the flagged problems,” Bucci contended. “This is why several criticisms were raised in several scientific journals by researchers from all over the world.”

In May 2021, for instance, Vasiliy Vlassov, vice president of the Society for Evidence-Based Medicine in Moscow, published a piece on the BMJ blog in which he stated that “the quality of the reports and secrecy over trial data raise deep concerns about research integrity.”

Another paper titled, “Controversy surrounding the Sputnik V vaccine,” was published in October 2021 by independent researchers in the journal Respiratory Medicine in which they, too, stated there are “doubts about the reliability of the [Phase III] study and that while “Sputnik V could meet the need to provide equitable access to COVID-19 vaccines for people living in low- and middle-income countries …there are still many concerns regarding the use of this vaccine.”

Still… Sputnik V vaccine should work 

Because The Lancet left the paper online and no action was taken, Bucci ultimately put out yet another assessment, this time in March 2022. In this report, he stressed once again his belief that the vaccine works, but that the data was incorrect. 

From the early days of vaccine roll-outs, the Sputnik vaccine has received considerable uptake in many low- as well as middle and even upper-middle income countries, particularly in Latin America – where it was more widely available and affordable than mRNA options. 

“The Russian Sputnik V vaccine should work much like any others,” Bucci wrote. “I am convinced of this because Sputnik V basically can be seen as a combination of a Chinese adenoviral vaccine and a Johnson & Johnson dose. Such vaccines use disabled adenoviruses (cold viruses) to deliver an inactive fragment of the SARS-CoV2 spike protein into the body, prompting an immune reaction.   

“The [Sputnik] production problems are linked precisely to the fact of combining two different vaccines in one, with what follows for the simplicity of the process and the quality control; but this does not mean that the idea, from a scientific point of view, is not valid, or that once the vials are obtained, they should not be useful.”

Bucci also said he had observed instances of sloppiness in data reporting, which nonetheless made it to press. For instance in the original online version of an article on Sputnik’s rollout in Argentina, published in The Lancet on 15 March, 2022, one chart showed that in the 60-69 age group,  49.7% of vaccine recipients were women and 80.7% were men – obviously impossible. The currently available version online shows the data as 49.7% women and 50.3% men -although it acknowledges that a correction version was posted on 9 June. 

The final results of Sputnik’s Phase III trials have not yet been published by the Russian Direct Investment Fund (RDIF), which developed the vaccine together with the Gamaleya Institute – although their completion was announced by the Russian Health Ministry in September 2021. 

At the time, the Health Ministry explained to the Russian media agency Kommersant that the results would not be published at all, since “the results of clinical trials are a trade secret.”

RDIF: ‘Best vaccine in the world’

This has not stopped RDIF from continually stressing how good the vaccine is. 

The Gamaleya Institute told Health Policy Watch in an email that “the efficacy of the Sputnik V vaccine has been documented  in more than 50 real-world and scientific studies in 10 countries, involving more than 12 million people.” 

It highlighted studies in the United Arab Emirates, Bahrain, San Marino, Argentina, Iran, Belarus and Paraguay. 

 “A computer simulation is not the way to check the quality of clinical trials, as opposed to the multiple independent studies in different countries,” the Gamaleya Institute added in its reply to Health Policy Watch. 

“In international science, computer modeling is not a recognized method to cast doubt over outcomes of clinical trials and real-world studies whose peer reviewed results were published in respected medical journals.” 

In his own interviews, RDIF CEO Kirill Dmitriev has stated boldly that “Sputnik is the best vaccine in the world, we have proven it.” He said any challenges were “bureaucratic obstacles.”

In November 2021, RDIF also disclosed real world data of the Health Ministry of the Republic of San Marino on the Russian Sputnik V vaccine – which it said demonstrated that Sputnik V was 80% effective against COVID-19 infection up to eight months after receiving the second dose – a much higher percentage of efficacy more months after receipt than Pfizer or Moderna.

“Sputnik team believes that adenoviral vaccines provide for longer efficacy than mRNA vaccines due to longer antibody and T-cell response,” a release by Sputnik stated.

Ultimately, the vaccine received approval for use in 71 countries, RDIF reported, and more than 100 million people outside of Russia have received a Sputnik vaccine.

As of 31 January, “over 400 million Sputnik vaccine doses have been supplied worldwide including Russia to date to fight COVID-19 pandemic,” Sputnik tweeted less than a month before the war. “Over 800,000 doses of 1-shot Sputnik Light, standalone vaccine & universal booster effective against #Omicron & other mutations arrived in Turkmenistan.”

‘Science is based on trust’

Bucci said that the most important point at this stage is to address the potential shortcomings in  proper data checking and editorial controls by The Lancet, as well as its failure to press harder for more detailed data clarifications from the authors of the Sputnik studies when questions arose.  

“Science is based on trust stemming from the possibility of checking and reproducing published results,” he told Health Policy Watch. “When the scientific community is denied access to the data needed to reproduce the findings from a specific research group, we are no longer dealing with science but with unsubstantiated claims.”

In response to queries from Health Policy Watch, The Lancet Group said that it “takes issues relating to scientific misconduct extremely seriously and follows best practice guidelines set by the Committee on Publication Ethics (COPE)”. 

“We acknowledge the questions raised about the validity of Sputnik vaccine data published in The Lancet and we have invited the authors of the Lancet paper to respond to these latest questions,” it added.

Image Credits: RDIF, Sputnikvaccine/Twitter, The Lancet.