A new paper was released on medRxiv posting numbers that seem to indicate that the Moderna vaccine is holding up a bit better than the Pfizer vaccine against COVID-19. Frantic headlines scream that the Pfizer mRNA vaccine is only 49% effective against the delta variant. Some thoughts
This paper has not gone through the peer-review process. I am very interested in how the experts view this study. Journalists reporting on the study do not have the expertise to analyze the data.
This was an observational study. In these studies, the healthcare provider (Mayo Clinic Health System) collected records of patients who came in with COVID-19 symptoms and analyzed their vaccine status (Moderna, Pfizer, or unvaccinated), and compared this to outcomes (infected, symptoms, hospital admission, ICR admission, death). The good news is that no one of the over 25,000 analyzed in each group who was vaccinated, died during the study. Also, the risk of hospitalization was extremely small with only 6 (Moderna) and 11 (Pfizer) being hospitalized vs 82 in the unvaccinated group. In all cases, this is a very small risk when compared to the entire cohort of 25,869
Flaws
In observational studies, you enroll those who come to you, which can lead to selection bias. This bias can greatly sway the data. For example, if people who are unvaccinated are less likely to seek treatment or testing, then you will underestimate the number of individuals getting sick in the unvaccinated cohort.
Over 60% of the individuals in the study were over the age of 55. In comparison, there are about equal numbers of individuals in each 10-year cohort in the US. Thus the data skews much older than the general population. Older individuals have weaker immune systems and are more prone to a poor response to vaccination. This will lead to more breakthrough cases. more hospitalization, and more ICU admission.
In comparing Moderna to BioNTech they are comparing very small numbers, (6 out of 25,869 to 11 out of 25,869). While the risk difference is statistically significant, it's not practically significant. If you are vaccinated with Moderna you have a 99.99977% chance of not ending up in the hospital. If you are vaccinated with Pfizer, you have a 99.99958% chance.
The delta comparison is frankly kind of ridiculous. A total of 72 positive COVID-19 tests were found in the Pfizer cohort. No data is given as to the number of positive tests per month, so if we assume equal numbers, that means there were 10 in July. Because of this small sample set, their graph shows a 95% confidence interval of 15 to 62%. This data should not even be reported.
Conclusion
I think the paper does show a difference in the effectiveness of the vaccines. It is something to watch as we move forward. Booster shots of reformulations may have to take place, but the bottom line is the vaccines are still very effective.