Introduction

Although there is broad consensus that boosters are effective against hospitalization and severe COVID-19 illness1, clinicians and the public face unanswered questions as to the comparative effectiveness of mRNA boosters by product type, over time, and within high-risk populations2,3,4. Left unanswered, these issues have the potential to undermine public confidence and feed narratives reinforcing vaccine hesitancy. As of October 2022, comparative effectiveness studies of primary vaccination with mRNA-1273 or BNT162b2 demonstrated a small absolute benefit favoring the mRNA-1273 vaccine, which increased over a 24-week period without detailed sub-group analyses of high-risk populations such as those with immunocompromising conditions2,3,4. As a result, public health messaging has focused on the substantial protection against COVID-19 hospitalization, regardless of vaccine product and membership in an at-risk population4.

Vaccination and boosting have shifted the spectrum of clinical disease toward mild COVID-19 illness5, even as protective immunity wanes6,7. As a result, many patients who are hospitalized for reasons other than COVID-19 (e.g., fracture, alcohol withdrawal, asymptomatic but screened positive) but have a positive test for COVID-19 are classified as having a COVID-related illness, a commonly studied outcome in the COVID-19 literature8. In contrast, studies with highly specific outcomes such as COVID-19 pneumonia are lacking in the literature but can provide high-quality evidence to demonstrate ongoing effectiveness of vaccines. Extending our understanding of booster effectiveness against highly specific outcomes will guide the decision-making of individuals who live in an era of abundant vaccines and concerns as to whether a booster product will change their risk.

In this work, a national cohort of Veterans was used in a target trial emulation design to compare mRNA booster effectiveness at preventing breakthrough COVID-19 and death or hospitalization due to COVID-19 pneumonia following vaccination and booster with three doses of BNT162b2 or mRNA-1273 vaccine.

Results

From an initial group of 6,286,624 participants, the analytic cohort consisted of 1,703,189 participants who received an initial COVID-19 vaccination series, followed by a third dose (Supplementary Fig. 1). Among these participants, 1,205,701 (70.8%) were age ≥65 years, and 1,566,828 (92.0%) were male; 1,188,470 (69.8%) had high-risk co-morbid conditions, and 189,057(11.1%) had immunocompromising conditions (Table 1, Supplementary Table 1). The largest amount of missingness (6.4%) occurred with the race variable.

Table 1 Characteristics of the boosted cohort by vaccination seriesa
Full size table

Of 1,703,189 participants, 917,954 (53.9%) received three doses of mRNA-1273 vaccine and 785,235 (46.1%) received three doses of BNT162b2 vaccine. Between 1 July and 30 November 2021 (Delta predominant variant period), 1,099,808 were boosted (64.6%); the remaining 603,381 participants (35.4%) were boosted after 1 December 2021 (Omicron predominant variant period). Among high-risk populations, 13.5% were age ≥65 years, 74.6% had high-risk co-morbid conditions, and 11.8% had immunocompromised conditions.

Main outcomes

Over a 32-week follow-up period, 22,848 of 1,703,189 developed breakthrough COVID-19 (149.3 events per 10,000 persons; 95% CI: 147.3, 152.1), and 1,649 died or were hospitalized with COVID-19 pneumonia (10.9 events per 10,000 persons; 95% CI: 10.2, 11.8) (Table 2, Supplementary Table 2).

Table 2 32-week incidence of and comparative mRNA booster effectiveness against breakthrough COVID-19 and death or hospitalization with COVID-19 pneumonia following vaccination and booster in adults
Full size table

Comparing participants in the overall cohort who received three doses of BNT162b2 with three doses of mRNA-1273 (BNT162b2 minus mRNA-1273), risk differences were estimated as 23.9 events of breakthrough COVID-19 per 10,000 persons (95% CI: 19.8, 29.9) and as 2.1 events of death or hospitalization with COVID-19 pneumonia per 10,000 persons (95% CI: 0.6, 3.3). These risk differences translated into large numbers needed to vaccinate: 415.5 for breakthrough COVID-19 (95% CI: 334.7, 505.3), and 4,936.6 for death or hospitalization with COVID-19 pneumonia (95% CI: 3,028.8, 16,670.3) (Table 2). Comparative effects of the third mRNA vaccine dose (BNT162b2 versus mRNA-1273) were sustained over eras of Delta and Omicron predominant variants (Supplementary Table 3).

Average-risk sub-group

Among average-risk participants who received three doses of BNT162b2 compared with those who received three doses of mRNA-1273 (BNT162b2 minus mRNA-1273), there was no difference in absolute risk of breakthrough COVID-19 or of death or hospitalization with COVID-19 pneumonia (Table 2). When considering relative risks, however, the cumulative incidence of breakthrough COVID-19 was higher among those who received three doses of BNT162b2 (Cumulative incidence ratio [CIR]: 1.1; 95% CI: 1.0, 1.3) (Table 4). There was no relationship between cumulative incidence of hospitalization and booster product.

High-risk sub-groups (non-overlapping)

Among all high-risk participants (age ≥65 years, high-risk co-morbid conditions, immunocompromising conditions), participants who received three doses of BNT162b2 were at higher cumulative incidence and absolute risk of breakthrough COVID-19 (CIR: 1.2; 95% CI: 1.2, 1.3; RD: 24.5; 95% CI: 18.7, 29.4) and of death or hospitalization with pneumonia (CIR: 1.3, 95% CI: 1.2, 1.4; RD: 2.2; 95% CI: 0.9, 3.6) than those who received three doses of mRNA-1273 (Tables 2 and 4). Comparing the high-risk participants with average-risk participants, additive interaction was present for death or hospitalization with COVID-19 pneumonia (RERI: 1.6) (Table 2).

Within high-risk populations, the magnitudes of the cumulative incidence ratio of death or hospitalization with COVID-19 pneumonia were similar across sub-groups while the magnitudes of the risk differences varied with an increased absolute risk of death or hospitalization with COVID-19 pneumonia in the sub-group of high-risk co-morbid conditions (RD: 2.6; 95% CI: 0.5, 4.4), and immunocompromising conditions (RD: 6.6; 95% CI: −1.2, 13.5) compared with the sub-group of age ≥65 years (RD: −0.3; 95% CI: −3.2, 2.1), (Table 3, Table 4).

Table 3 32-week incidence of and comparative effectiveness against breakthrough COVID-19 and death or hospitalization with COVID-19 pneumonia following vaccination and booster in high-risk sub-groups (non-overlapping)
Full size table
Table 4 Cumulative incidence ratios of breakthrough COVID-19 and hospitalization with COVID-19 pneumonia or death comparing Veterans who received mRNA-1273 ×3 with those who received BNT–162b2, overall and by sub-group
Full size table

Comparing the sub-groups of high-risk co-morbid or immunocompromising conditions with lower risk sub-groups, additive interaction was present for death or hospitalization with COVID-19 pneumonia (RERI: 1.4), but multiplicative interaction was not present (p = 0.50) (Table 3). Further interaction analyses focusing on the sub-group of immunocompromising conditions against other sub-groups showed that the presence of additive interaction (RERI: 1.1).

To prevent one case of hospitalization with COVID-19 pneumonia or death, the estimated number needed-to-vaccinate (NNV) with three doses of mRNA-1273 compared with three doses of BNT162b2 during a 32-week follow-up period would be 1,411.7 for those with immunocompromising conditions. This NNV of 1,411.7 was 2.7 times lower than the NNV of 3,869.6 for the high-risk co-morbid condition sub-group (Table 3).

Cumulative incidence curves

In the overall cohort, the cumulative incidence of death or hospitalization due to COVID-19 pneumonia by booster product was similar until week 12 at which time the cumulative incidence curves diverged after 12 weeks as shown in Fig. 1. Compared with the overall cohort, these differences were smaller among those with age ≥65 years (Fig. 2), similar among those with high-risk co-morbid conditions (not immunocompromised) (Fig. 3), and greater among those with immunocompromising conditions (Fig. 4).

Fig. 1: Overall cohort.
figure 1

32-week cumulative incidence of death or hospitalization with COVID-19 pneumonia following vaccination and booster with BNT162b2 ×3 and mRNA-1273 ×3 in overall cohort.

Full size image
Fig. 2: Those with age >65 with no high-risk conditions.
figure 2

32-week cumulative incidence of death or hospitalization with COVID-19 pneumonia following vaccination and booster with BNT162b2 ×3 and mRNA-1273 ×3 restricted to those with age >65 with no high-risk conditions.

Full size image
Fig. 3: Those with high-risk co-morbid conditions (not immunocompromising conditions).
figure 3

32-week cumulative incidence of death or hospitalization with COVID-19 pneumonia following vaccination and booster with BNT162b2 ×3 and mRNA-1273 ×3 restricted to those with high-risk co-morbid conditions (not immunocompromising conditions).

Full size image
Fig. 4: Those with immunocompromising conditions.
figure 4

32-week cumulative incidence of death or hospitalization with COVID-19 pneumonia following vaccination and booster with BNT162b2 ×3 and mRNA-1273 ×3 restricted to those with immunocompromising conditions.

Full size image

Discussion

In this national US cohort of adults receiving care at VHA facilities, 32-week relative and absolute risk of hospitalization or death due to COVID-19 pneumonia was statistically higher among those who received three doses of BNT162b2 vaccine than those who received three doses of mRNA-1273 vaccine. The cumulative incidence diverged 12 weeks after the third dose. In analysis by sub-group however, there was a divergence in the pattern of findings. Although relative risks remained consistent, differences in absolute risk were increased in the high-risk population and driven by sub-group’s risk profile (high-risk co-morbid or immunocompromising conditions >> average-risk or age ≥65 years), suggesting that the specific populations of patients with high-risk co-morbid or immunocompromising conditions will gain from receiving three doses of mRNA-1273 vaccine instead of three doses of BNT162b2 vaccine.

Studies from this and other research groups have highlighted how those with immunocompromising conditions are at higher-risk of COVID-19 hospitalization than other at-risk groups such as those with high-risk co-morbid conditions8,9, but have not compared the effect of booster products on COVID-19 hospitalization by immunocompetent and immunocompromised populations6. Further, studies comparing two doses of mRNA-1273 and BNT162b2 vaccine demonstrated similar magnitudes of relative risk across age groups and comorbidities as well as small absolute risk differences in the overall cohorts2,3, but also did not include a group with immunocompromising conditions, which would have allowed for a direct comparison of absolute risk and NNV against other sub-groups. In our study, the immunocompromised sub-group had the largest magnitude of the risk difference of death or hospitalization with COVID-19 pneumonia; although non-significant and under-powered, this important finding meant that the NNV of 1411.7 among the immunocompromising conditions group was 2.7 times lower than the NNV of 3,869.6 among the high-risk co-morbid conditions group, suggesting that there may be a larger public health benefit among the immunocompromised group.

Even though the NNV can be used to inform tailored public health decisions, many other studies have focused on relative risks because of their value in estimating vaccine efficacy or effectiveness, ease of interpretation by the public10, higher concordance across baseline risks11, and the large numbers of NNVs associated with COVID-19 vaccines. One of the first major COVID-19 vaccine surveillance studies to be published reported a risk difference of 0.22 per 1,000 persons, which translated to a NNV of 4545 to prevent one COVID-19 hospitalization among unvaccinated individuals, were these persons to have been vaccinated12. Compared with an estimated vaccine effectiveness of 87% against COVID-19 hospitalization, such a high NNV may have been less appealing to clinicians and the public; however, the presentation of multiple effect estimates, inclusive of NNVs, can answer critical questions concerning a particular vaccine’s differing public health benefits across various at-risk sub-groups13. In a later comparative effectiveness study of two doses each of mRNA-1273 vaccine and BNT162b2 vaccine, Dickerman, et al. reported the estimated NNV with mRNA-1273 instead of BNT162b2 would be 1818 (95% CI: 1,205.0, 2,778.0) to prevent one case of COVID-19 hospitalization2. The NNVs in our study were roughly similar to other studies comparing mRNA-1273 and BNT162b2 vaccines, except that our study examined a more specific outcome (death or hospitalization with COVID-19 pneumonia). Further, the high-risk population had relatively lower NNVs than the average-risk population. The non-significant absolute risk differences of COVID-19 hospitalization among average-risk population and the age >65 sub-group in our study should provide reassurance as to the adequate protection from severe COVID-19 illness conferred by either vaccine, consistent with public health messaging of other studies.

This study has several limitations. First, potential confounders such as COVID-19 exposure behaviors could not be measured. Second, our definition of immunocompromised status may have missed some of the relatively smaller sub-populations such as those with intrinsic immune deficits or others who do not take immunosuppressive agents or have a history of cancer; our analyses of this group considered timing of 3rd dose (excluding those who obtained their third dose within 2 months) but not size of dose. Third, sub-groups such as nursing home residents were excluded so that the findings could have greater generalizability to community-dwelling individuals living in the U.S. However, the boosted study population comprised predominantly older, white men with high-risk comorbidities, so despite the substantial absolute numbers of participants with female sex, younger age, African American race, Latino ethnicity, and no comorbidities, inferences to these sub-populations (e.g., age, sex, race) should be approached with caution. Fourth, it is unclear whether these findings would apply to 4 or more doses of vaccine, updated booster vaccine products (e.g., bivalent Omicron booster), and other viral variants. Fifth, VHA-distributed mRNA vaccine types were based on facility-specific factors, and it is unknown if the distribution may have affected outcomes. A facility variable was included in the propensity score model to account for potential geographic differences in allocation. Finally, the use of laboratory testing may have decreased over time, which may have led to underestimates of risk within each group but were unlikely to be different between arms.

Comparative mRNA booster effectiveness varied by population and risk group. During a 32-week risk period, a small benefit was found among the high-risk population who received three doses of mRNA-1273 vaccine to prevent hospitalization with COVID-19 pneumonia or death; no benefit was found among the average-risk population and the age >65 sub-group. In an era of vaccination when individuals can choose their booster, it is important to re-evaluate the populations that may benefit from mRNA-specific booster guidelines because of the public health significance and changing times.

Methods

Ethics statement

This study complies with all relevant ethical regulations. The institutional review board of the University of California, San Francisco, approved this study and waived requirement for patient consent as it involved no more than minimal risk to participants.

Study design, setting, and data sources

We used Veterans Health Administration (VHA) Corporate Data Warehouse (CDW)14 and COVID-19 Shared Data Resource15 to construct the adult cohort for this target trial emulation. The VHA operates a national health system with co-localized services, including COVID-19 vaccination, laboratory testing, outpatient, and inpatient services. The VHA began administering the initial series of COVID-19 vaccinations after December 11, 2020. The health system provided facilities with the initial series of either mRNA-1273 or BNT162b2 vaccines based on local access to cold chain capacity and geography. The VHA allocated to its facilities either mRNA-1273 or BNT162b2 vaccine for the initial series, so patient preference had no role in the administration of either mRNA vaccine, ultimately resulting in almost all (> 90%) Veterans receiving homologous vaccine doses. Booster doses became available on 1 July 2021 and at the time, higher-risk groups, particularly immunocompromised individuals, were among the first to receive a third dose.

Participants

Adults receiving care at VHA facilities were eligible for inclusion if they had documented receipt of all U.S. Food and Drug Administration (FDA) authorized doses of the initial vaccination series of an mRNA vaccine and subsequently had documented receipt of a third dose between 1 July 2021 and 29 April 2022. The cohort was restricted to participants who had a primary care visit at a VHA facility in 2019 or 2020 to ensure adequate baseline data on health status and capture of clinical outcomes. Those participants living in the community without a record of vaccination who delayed the second dose beyond 6 weeks, and did not receive three doses of either mRNA-1273 or BNT162b2 vaccines were excluded1. Participants with non-immunocompromising conditions who obtained their third dose within 5 months and those with immunocompromising conditions who obtained their third dose within 2 months were also excluded1. Further inclusion and exclusion criteria are detailed in Supplementary Fig. 1.

The index date (or date of cohort entry) was the date on which the subject received his or her third dose (first booster vaccine). For each person, follow-up started on the day of booster dose and ended on the day of outcome of interest, death, 224 days (32 weeks) after baseline, fourth dose, or the end of the study period (May 30, 2022), whichever happened first. The observation period included predominance of both Delta and Omicron SARS-CoV-2 variants in the US16.

Measurements

Exposure was receipt of three doses of mRNA-1273 vaccine or three doses of BNT162b2 vaccine, as extracted from the CDW. The primary outcomes were: (1) breakthrough COVID-19 (symptomatic infection); and (2) the combined endpoint of all-cause death within 30 days after breakthrough infection or hospitalization with a diagnosis of COVID-19 pneumonia.

Breakthrough COVID-19 was defined as a post-booster, laboratory-confirmed, symptomatic COVID-19 diagnosis, using the VA’s COVID-19 National Surveillance Tool17,18 and VA’s COVID-19 Shared Data Resource19. Hospitalization with COVID-19 pneumonia was defined as a diagnosis of COVID-19 pneumonia using ICD-10 code J12.84 in the CDW20, or documented by the clinical care team in the electronic medical record during hospitalization.

The outcome of COVID-19 pneumonia was verified through a combination of text processing-assisted chart review and ICD-10 codes (see Supplement). Study staff reviewed 10% of charts in duplicate (95% agreement) and flagged unclear diagnoses of pneumonia (e.g., emergency room note was only reference to pneumonia during hospitalization) for adjudication by three clinicians (JDK, SK, DMB). A list of covariates considered to be cohort characteristics or potential confounders were extracted from CDW and are described in the Supplement.

Statistical analyses

Baseline covariates of the boosted cohort were compared between participants who received three doses of mRNA-1273 and three doses of BNT162b2. Standardized differences were used to quantify differences in covariates between exposure groups. Among the boosted cohort, the propensity score for receipt of three doses of mRNA-1273 versus BNT162b2 was estimated using logistic regression (see Supplement). Inverse probability of treatment weights (IPTW) were computed as Z/PS + (1–Z)/(1–PS), where Z indicates treatment status (Z = 1 for three doses of BNT162b2; Z = 0 for three doses of mRNA-1273) and PS indicates the estimated propensity score. The use of these weights allowed us to estimate the average treatment effect (ATE): the effect of moving all subjects from BNT162b to mRNA-1273.

The weighted cumulative incidence function was used to estimate the risk of the outcome from the day of booster dose until end of follow-up and the risk difference (absolute risk reduction). Participants who died prior to developing breakthrough COVID-19 were treated as a competing risk on date of death. Weighted (adjusted) Fine-Gray models were used to estimate cumulative incidence ratios (risk ratios). Robust variance estimators accounted for the within-person homogeneity induced by weighting. The proportionality assumption was tested with a log-log plot and was met. The risk difference was used to estimate the number needed to vaccinate (NNV). The association of booster product with breakthrough COVID-19 and death or hospitalization with COVID-19 pneumonia was examined. Findings were considered statistically significant if the confidence interval did not cross the null value or if the two-sided p value was <0.05.

These analyses were repeated within sub-groups. The overall cohort was divided into average-risk and high-risk sub-groups, with the high-risk population further sub-divided into three non-overlapping sub-groups: (1) age ≥65 years with no high-risk conditions, (2) high-risk co-morbid conditions (excluding immunocompromised or with cancer), and (3) immunocompromising conditions (including cancer). These high-risk sub-groups were pre-specified and based on risk stratification from COVID-19 vaccine guidance set forth by the U.S. Centers for Disease Control and Prevention1.

Another set of analyses were designed to assess the interaction of booster product by sub-group and by Delta and Omicron predominant variant eras. Interaction was evaluated on the multiplicative and additive scales. The presence of additive interaction was defined as the relative excess risk due to interaction (RERI)21. This study evaluated additive interaction because of its relevance to public health and focus on risk differences and NNV. The variant era was based on the predominant variant circulating during a calendar period. The Delta-predominant era was between 1 July 2021 and 30 November 2021. The Omicron-predominant era was between 15 December 2021 and 30 May 2022. An interaction p value cutoff of 0.20 or non-zero RERI determined statistical significance.

In all analyses, standardized differences of each covariate were all <0·01 (Supplementary Table 4). Missing data were handled as a ‘missing’ category for the variable. Analyses were conducted in RStudio version 1·2·5019, including the survival package.

Reporting summary

Further information on research design is available in the Nature Portfolio Reporting Summary linked to this article.