Development and validation of a new algorithm for improved cardiovascular risk prediction

18 Apr 2024

Nature

Share on XTweet

Development and validation of a new algorithm for improved cardiovascular
risk prediction

The QR4 algorithm for prediction of 10-year cardiovascular disease risk, developed, tested and externally validated in datasets comprising 16.8 million people from the United Kingdom, improves upon the QRISK3 algorithm that is in current use by incorporat

Study population

There were 9,976,306 people aged 18–84 years in the QResearch English derivation cohort, 3,246,602 in the QResearch English validation cohort and 3,542,007 in the CPRD validation cohort from the other three UK nations (that is, Scotland, Wales and Northern Ireland). Extended Data Table 1 shows the flow of patients and the relevant exclusions. Extended Data Table 2 shows the new predictors under consideration.

The baseline characteristics of each cohort and the completeness of the recording of data for predictors with missing data are shown in Table 1. The cohorts were broadly similar, except that both English cohorts contained more complete data for ethnicity, smoking, cholesterol and body mass index (BMI) than the CPRD validation cohort from the other three UK nations. Supplementary Table 4 shows the characteristics of participants with complete versus missing data in the QResearch derivation cohort: those with complete data tended to be older, and more likely to be female and to have clinical conditions.

Table 1 Baseline characteristics of participants

Full size table

There were 202,424 incident CVD cases (based on primary outcome definition) from 49.1 million person-years in the derivation cohort. Extended Data Table 1 shows the types of CVD events in each cohort for each of the three outcome definitions.

The crude CVD incidence rates for the primary CVD outcome by age, sex, ethnicity and calendar year in the English derivation cohort and CPRD validation cohort are shown in Extended Data Table 3. CVD rates using linked data were higher in the English cohort, which was largely explained by the additional data linkage to hospital and mortality data. Extended Data Fig. 1 shows both CVD incidence rates and non-CVD mortality rates by calendar year and month for the whole study period. CVD rates per 1,000 person-years were lower in 2020, the first year of the COVID-19 pandemic, when the overall rate was 4.03 (95% CI, 3.97–4.08) but returned to pre-pandemic levels in 2021 (4.31; 95% CI, 4.25–4.37). Non-CVD mortality rates increased from 3.45 (95% CI, 3.40–3.50) in 2019 to 3.84 (95% CI, 3.79–3.89) in 2020 and remained elevated in 2021.

Factors associated with increased risk of CVD

The adjusted hazard ratios for CVD incidence in the final cause-specific models in men and women (evaluated at the mean age of 39 years for variables with age interactions) are shown in Fig. 1. Extended Data Fig. 2 shows the adjusted hazard ratios for the fractional polynomial terms for CVD risk for continuous variables and the predictor variables with significant age interactions for both men and women. Supplementary Figs. 1 and 2 show the corresponding results for non-CVD death.

There were seven new CVD predictors in men and women (brain cancer, lung cancer, Down syndrome, blood cancer, COPD, oral cancer and learning disability) and two additional predictors in women (pre-eclampsia and postnatal depression).

We found no association between the following variables and CVD risk in men or women: asthma, hyperthyroidism, hypothyroidism, antiphospholipid antibody syndrome, benign intracranial hypertension, HIV or AIDS, and the remaining cancers. In women, there were no associations with in vitro fertilization, endometriosis, polycystic ovarian syndrome, gestational diabetes, miscarriage, termination or placental abruption. No violations of the proportional hazard assumptions were detected graphically. The values for the heuristic shrinkage16 were all very close to one (0.99), indicating no evidence of overfitting.

New CVD predictors in women

The adjusted hazard ratios (95% CI) for the nine new independent predictors of CVD risk in women (evaluated at the mean age of 39 years for variables with age interactions) are as follows: brain cancer, 4.52 (2.49–8.21); lung cancer, 3.50 (1.31–9.38); Down syndrome, 3.18 (2.40–4.22); blood cancer, 2.13 (1.71–2.67); COPD, 1.85 (1.50–2.29); oral cancer, 1.55 (1.27–1.89); learning disability, 1.45 (1.29–1.64); pre-eclampsia, 1.56 (1.36–1.78); and postnatal depression, 1.18 (1.11–1.26).

The adjusted hazard ratios for several of these predictors were higher at younger ages (for example, under 35 years), except for lung cancer in women, for which adjusted hazard ratios were highest for those around age 40 years and then declined gradually with increasing age (Extended Data Fig. 2). The adjusted hazard ratios (95% CI) at age 69 were as follows: brain cancer, 2.18 (1.29–3.71); lung cancer, 1.97 (1.64–2.37); blood cancer, 1.39 (1.28–1.50); COPD, 1.38 (1.32–1.44); and pre-eclampsia, 1.12 (1.01–1.24).

The magnitude and direction for many of the adjusted hazard ratios for the competing outcome of non-CVD death in women were similar to those for CVD except for large adjusted hazard ratios (evaluated at age 39 years) for Down syndrome (18.32; 95% CI, 16.24–20.66), lung cancer (49.94; 95% CI, 40.61–61.43) and brain cancer (33.35; 95% CI, 26.17–42.49). The adjusted hazard ratios for non-CVD death for family history of coronary heart disease, pre-eclampsia and migraine were significantly less than one (Supplementary Fig. 1).

New CVD predictors in men

The adjusted hazard ratios for the seven new independent predictors of CVD risk in men (evaluated at age 39 years are shown in Fig. 1), and the adjusted hazard ratios (95% CI) for these predictors are as follows: brain cancer, 5.45 (3.49–8.50); Down syndrome, 2.35 (1.84–2.99); blood cancer, 2.06 (1.78–2.39); lung cancer, 1.66 (1.45–1.92); oral cancer, 1.49 (1.30–1.70); COPD, 1.37 (1.32–1.41); and learning disability, 1.17 (1.07–1.29). The adjusted hazard ratios in men for brain cancer and blood cancer declined with age; for example, at age 69 years, the adjusted hazard ratio (95% CI) was 2.12 (1.25–3.61) for brain cancer and 1.23 (1.15–1.31) for blood cancer.

The adjusted hazard ratios for the additional models were similar to our final main models in men and women (Supplementary Figs. 3–7). Model A includes the original QRISK3 predictor variables but without competing risks. Model B is similar to our final model, but the follow-up time ended on 29 February 2020, before the COVID-19 pandemic. Model C shows that the adjusted hazard ratios for CVD risk were similar across periods of time after diagnosis with one of the four cancers (except for oral cancer in women), although the adjusted hazard ratios for non-CVD deaths varied with the highest values for more recently diagnosed cancers.

Predicted risks

The way in which each of the new risk factors affects the predicted 10-year CVD risk for specific individuals is shown in Fig. 2. In this illustration, which is presented for both men and women across ages 18 to 84 years, CVD risk was compared between individuals with a new risk factor and reference individuals with no adverse clinical indicators (a cholesterol/HDL ratio of 4.0, an SBP of 125 mm Hg and a BMI of 25 kg m−2). These risk calculations show the impact of the new risk predictors, which mainly resulted in increased predicted risks compared with the reference individuals at younger ages and decreased predicted risks at older ages as competing risks become more pronounced. Using a reference group of individuals with various conventional risk factors (light smokers with a cholesterol/HDL ratio of 6.0, an SBP of 170 mm Hg and a BMI of 35 kg m−2), a similar pattern was observed, albeit with higher overall predicted risks (Supplementary Fig. 8).

Discrimination

The performance statistics (C statistic, calibration slope and calibration intercept) for QR4 and QRISK3 for the validation cohorts in England, Scotland, Wales and Northern Ireland are shown in Table 2. The C statistic for QR4 was marginally higher than that for QRISK3 in both validation cohorts. For example, the C statistics were 0.835 (95% CI, 0.833–0.837) and 0.831 (95% CI, 0.829–0.832) for QR4 and QRISK3, respectively, in women in the devolved administrations (Scotland, Wales and Northern Ireland). The corresponding values in women in England were 0.864 (95% CI, 0.862–0.866) for QR4 and 0.862 (95% CI, 0.860–0.864) for QRISK3. The C statistic values were generally higher in England than in the other three nations, although all values remained within an excellent range (>0.8). The results for men were similar, though the values were slightly lower.

Table 2 Evaluation of discrimination and calibration of QR4 compared with QRISK3

Full size table

The overall discrimination results and discrimination results delineated by ethnic group for QR4, SCORE and ASCVD are shown in Extended Data Table 4; these results were restricted to those aged 40 years and older in the validation cohort in England. For women, overall discrimination was highest with QR4 (0.781; 95% CI, 0.778–0.784), followed by ASCVD (0.767; 95% CI, 0.764–0.770) and SCORE2 (0.767; 95% CI, 0.764–0.770). There was a similar pattern for men.

The C statistics, calibration slopes and calibration intercepts (overall and by ethnic group) for QRISK3 and QR4 in men and women aged 18–84 years in the English validation cohort are shown in Extended Data Table 5. These results show that discrimination varied by ethnic group in England: the C statistic for QR4 was highest for Chinese men (0.923; 95% CI, 0.906–0.939) and lowest for Caribbean men (0.825; 95% CI, 0.801–0.841).

The definitions of the CVD outcomes used for sensitivity analyses are shown in Supplementary Table 1. Supplementary Table 2 shows the performance statistics for QR4, SCORE2 and ASCVD for each outcome measure for each of the four UK nations among those aged 40 years and older. The C statistic values for all scores (QR4, SCORE2 and ASCVD) were highest for the tertiary outcome measure, and for all outcome measures discrimination values were higher for QR4 than those for SCORE2 and ASCVD, which yielded values similar to one another.

Decision curve analysis

The decision curves in Fig. 3 indicate a slightly larger net benefit with QR4 compared with QRISK3 and Model A, but differences in net benefit are more marked in the devolved administrations than in England.

The decision analysis curves for QR4, SCORE2 and ASCVD for the primary outcome in England are shown in Extended Data Fig. 3. Supplementary Figs. 9 and 10 show corresponding results for the secondary and tertiary CVD outcomes.

Calibration

QR4 was well-calibrated in England, showing a close correspondence between predicted and observed risks, whereas QRISK3 overpredicted risk in the higher centiles of predicted risk (Fig. 4). Table 2 shows the calibration slope and intercept values for QRISK3 and QR4 by country. There was a degree of miscalibration for QRISK3 and QR4 in each of the devolved administrations (Supplementary Fig. 11) on the basis of general practitioner (GP) data only.

The calibration results for ASCVD and SCORE2 in the English validation cohort, which are based on our primary outcome definition, are shown in Extended Data Fig. 4. Supplementary Table 2 and Supplementary Figs. 12 and 13 show the corresponding results for the secondary and tertiary outcomes. Overall, there was a degree of overprediction for ASCVD and a degree of underprediction for SCORE2, which were improved when comparisons were made with the more specific secondary and tertiary outcomes.

Reclassification

The characteristics of the 84,700 (2.6%) participants in the English validation cohort reclassified using QR4 instead of QRISK3 at the 10% risk threshold are shown in Extended Data Table 6. Of the 3,554 people reclassified from low risk to high risk using QR4, 1,168 (32.9%) had COPD, 57 (1.6%) had a learning disability, 72 (2.0%) had Down syndrome, 72 (2.0%) had a history of pre-eclampsia, 125 (3.5%) had a history of postnatal depression, 90 (2.5%) had oral cancer, 54 (1.5%) had brain cancer, 92 (2.6%) had lung cancer and 322 (9.1%) had blood cancer. Those reclassified as high risk using QR4 tended to be younger (mean age of 52.4 years) than the 81,146 people reclassified as low risk (mean age of 60.5 years). Supplementary Table 3 shows the corresponding analyses for the 4,068 participants reclassified as high risk using QR4 compared with Model A as well as the 12,791 participants reclassified as low risk; the pattern was similar, although the total number of participants that were reclassified was much smaller (16,859, 0.52%).