Development and Validation of An Interpretable Risk Prediction Model for In-Hospital Mortality in Diabetic Patients with Congestive Heart Failure in Intensive Care Unit
This study presents a machine learning model that predicts in-hospital mortality in diabetic patients with CHF, achieving an ROC-AUC of 0.8663 in the test set.
Where it sits
this study against the rest of the vasopressin corpusSummary and findings
This study developed a machine learning model to predict in-hospital mortality in diabetic patients with congestive heart failure (CHF). The model was validated using data from two cohorts: MIMIC-IV (n=7,063) and eICU-CRD (n=4,532). The logistic model achieved an ROC-AUC of 0.8663 in the test set and 0.751 in the external validation set.
Abstract
<title>Abstract</title> <p> <bold>Background and Objective:</bold> Diabetic patients with congestive heart failure (CHF) face a high risk of in-hospital mortality, yet existing prediction models lack specificity for this comorbidity and often suffer from poor interpretability. This study aimed to develop and validate an interpretable machine learning model for predicting in-hospital mortality in diabetic patients with CHF. <bold>Methods:</bold> Data were derived from the MIMIC-IV (derivation cohort, n=7,063) and eICU-CRD (external validation cohort, n=4,532) databases. Four feature selection methods (LASSO, Boruta, recursive feature elimination, and univariate logistic regression) identified shared predictors. Ten machine learning algorithms were benchmarked, and logistic regression was optimized. Model performance was assessed using ROC-AUC, PR-AUC, calibration, and decision curve analysis. SHAP provided global and local interpretability. An R Shiny web application was developed. <bold>Results:</bold> Thirteen independent predictors were selected: APS III, age, norepinephrine, vasopressin, anion gap, temperature, respiratory rate, intubation status, SOFA, mean corpuscular volume, phenylephrine, and dopamine. The logistic model achieved an ROC-AUC of 0.8663 (test set) and 0.751 (external set), with good calibration (Brier score 0.073–0.096) and net clinical benefit. SHAP revealed APS III as the most influential feature. The web-based tool enables real-time risk prediction with individualized explanations. <bold>Conclusion:</bold> This interpretable logistic model accurately predicts in-hospital mortality in diabetic patients with CHF, and the accompanying web application offers a transparent, user-friendly tool to support bedside risk stratification and clinical decision-making. </p>
Background
Diabetic patients with congestive heart failure (CHF) experience a significant risk of in-hospital mortality, yet current prediction models are often not tailored for this specific group. Previous models have been criticized for their lack of specificity and interpretability. This study addresses the need for a more accurate and understandable risk prediction tool for this vulnerable population.
Methods
Data were sourced from the MIMIC-IV database for the derivation cohort (n=7,063) and the eICU-CRD for the external validation cohort (n=4,532). The study employed four feature selection methods and benchmarked ten machine learning algorithms, ultimately optimizing a logistic regression model. Performance was evaluated using ROC-AUC, PR-AUC, calibration metrics, and decision curve analysis.
Results
The logistic regression model achieved an ROC-AUC of 0.8663 in the test set and 0.751 in the external validation set. The model demonstrated good calibration with a Brier score between 0.073 and 0.096. Thirteen independent predictors were identified, with APS III being the most influential according to SHAP analysis.
Interpretation
The ROC-AUC values indicate a strong predictive ability of the model in the test cohort, although the external validation ROC-AUC suggests a decrease in performance. While the statistical significance of the findings is clear, the clinical relevance may be limited by the model's reliance on retrospective data and the potential for unmeasured confounding factors. The study's findings should be interpreted with caution, particularly in clinical settings.
Key findings
- ROC-AUC of 0.8663 (test set), n=7063.
- ROC-AUC of 0.751 (external validation set), n=4532.
- Brier score ranged from 0.073 to 0.096.
- Thirteen independent predictors identified.
Limitations
- Retrospective data from existing databases.
- External validation cohort may not represent the general population.
- Potential for unmeasured confounding factors.