Mortality assessment in intensive care units via adverse events using artificial neural networks

https://doi.org/10.1016/j.artmed.2005.07.006Get rights and content

Summary

Objective

This work presents a novel approach for the prediction of mortality in intensive care units (ICUs) based on the use of adverse events, which are defined from four bedside alarms, and artificial neural networks (ANNs). This approach is compared with two logistic regression (LR) models: the prognostic model used in most of the European ICUs, based on the simplified acute physiology score (SAPS II), and a LR that uses the same input variables of the ANN model.

Materials and methods

A large dataset was considered, encompassing forty two ICUs of nine European countries. The recorded features of each patient include the final outcome, the case mix (e.g. age) and the intermediate outcomes, defined as the daily averages of the out of range values of four biometrics (e.g. heart rate). The SAPS II score requires 17 static variables (e.g. serum sodium), which are collected within the first day of the patient’s admission. A nonlinear least squares method was used to calibrate the LR models while the ANNs are made up of multilayer perceptrons trained by the RPROP algorithm. A total of 13,164 adult patients were randomly divided into training (66%) and test (33%) sets. The two methods were evaluated in terms of receiver operator characteristic (ROC) curves.

Results

The event based models predicted the outcome more accurately than the currently used SAPS II model (P<0.05), with ROC areas within the ranges 83.9–87.1% (ANN) and 82.6–85.2% (LR) versus 80% (LR SAPS II). When using the same inputs, the ANNs outperform the LR (improvement of 1.3–2%).

Conclusion

Better prognostic models can be achieved by adopting low cost and real-time intermediate outcomes rather than static data.

Introduction

In the last decades, there has been an increasing development in intensive care medicine, where the goal is to provide the best outcome for critically ill patients. Indeed, a worldwide expansion occurred in the number of intensive care units (ICUs) [1]. Moreover, scoring the severity of illness has become a daily practice, with several metrics available, such as the acute physiology and chronic health evaluation system (APACHE II), the simplified acute physiology score (SAPS II) or mortality probability model (MPM), just to name a few [2].

The intensive care improvement comes with a price, being ICUs responsible for an increasing percentage of the health care budget. Resource availability limitations force them to make sure that intensive care is applied only to those who are likely to benefit from it. Critical decisions include interrupting life-support treatments and writing do-not-resuscitate orders when intensive care is considered futile. Under this context, mortality assessment is a crucial task, being used not only to predict the final clinical outcome but also to evaluate the ICU effectiveness. The prevalent prognostic models are built using a logistic regression over a static score (e.g. SAPS II); i.e., computed with data collected only within the first 24 h of the patient’s admission. This limits the impact of clinical decision making, since the scores are usually not updated during the patients’ length of stay.

On the other hand, the use of data mining in medicine is a rapidly growing field, which aims at discovering some structure in large clinical heterogeneous data [3]. This interest arose due to the rapid emergence of electronic data management methods, holding valuable and complex information. Human experts are limited and may overlook important details, while automated discovery tools can analyze the raw data and extract high level information for the decision-maker [4].

The artificial neural networks (ANNs) are one of the most successful data mining techniques, denoting a set of connectionist models inspired by the behavior of the human brain and presenting useful capabilities for medicine such as nonlinear learning, multi-dimensional mapping and noise tolerance [5]. The interest in ANNs was stimulated by the advent of the backpropagation algorithm in 1986. Since then, the number of ANN publications in Medicine has spawned from 2 in 1990 to 500 in 1998 [6] and the search term “neural network computer” in the MEDLINE database displays more than two thousand articles from 1999 to 2004.

In the past, there has been work comparing ANNs and logistic regression models for ICU mortality prediction, reporting either better [7], [8] or similar [9], [10], [11] performances. Yet, in all these studies, the ANNs were trained with the static variables used by the APACHE II score. This work follows an alternative direction, the use of data collected after the first 24 h of a patients’ admission. A similar approach has been proposed by Kayaalp et al. [12], [13] where they adopted a time series prediction point of view, using twenty three temporal fields, such as the daily sequential organ failure assessment (SOFA) score, which takes time and costs to be obtained. However, in previous work [14], it has been shown that the SOFA can be replaced by real-time and less costly outcomes, known as events, which are automatically measured as out of range values of four commonly bedside monitored physiological parameters. Hence, this article presents a novel approach for ICU mortality prediction, based on the use of daily intermediate events.

A final remark will be given to the relation between prognostic models and treatments. A prognostic scoring system should be independent of treatment, providing a means of measuring disease or health status of a patient, where usually a higher score corresponds to greater severity. Different elements contribute to this total score, including physiological variables. Although therapeutical approaches may influence the final patient outcome, their effect will be reflected upon the average number of events per day, the alarms signs. By augmenting a prognostic model with utility assessments of potential outcomes and indicating particular variables for decision support, optimal decisions for a group or individual patients can be determined [15].

The paper is organized as follows: first, the ICU clinical data is presented and the prognostic models are introduced (Section 2); next, a description of the performed experiments is given, being the results analyzed and discussed (Section 3); finally, closing conclusions are drawn (Section 4).

Section snippets

Clinical data

This work adopted part of data collected during the EURICUS II project [16], which involved 42 ICUs of nine European Union countries, from November 1998 to August 1999. The patient’s data was manually collected and registered by the nursing staff. In every hour, the monitored bedside parameters were introduced into daily patient records. The whole data was gathered at the Health Services Research Unit of the Groningen University Hospital, The Netherlands. The final database presented one entry

Training setup

All experiments reported in this work related to the ANNs, including the RPROP algorithm, were conducted using an object oriented programming environment developed in JAVA[32]. On the other hand, the logistic regression models and the statistics, including the Acc, Spe, Sen, AUC and t-Student’s tests; were computed using the R statistical environment [33].

The commonly used 2/3 and 1/3 partitions were adopted for the definition of the training and test set sizes [28]. A model is said to overfit

Conclusion

The surge of data mining techniques, such as artificial neural networks (ANN), has created new exciting possibilities in medicine. In this work, these techniques were applied for the prediction of death in intensive care units (ICUs) by using daily intermediate outcomes, which are defined from four commonly bedside monitored variables. In contrast, the current ICU prognostic models, based on a logistic regression (LR), use data collected only in the first day of the patient’s admission. Both

Acknowledgements

We thank FRICE and the BIOMED project BMH4-CT96-0817 for the provision of part of the EURICUS II data and support for this study, which is integrated in a Ph.D. program, developed at Instituto de Ciências Biomédicas Abel-Salazar from University of Porto and the Departments of Computer Science/Information Systems from the University of Minho. We also would like to thanks the anonymous reviewers for their helpful comments.

References (38)

  • L. Wong et al.

    A comparison of ICU mortality prediction using the APACHE II scoring system and artificial neural networks

    Anaesthesia

    (1999)
  • G. Clermount et al.

    Predicting hospital mortality for patients in the intensive care unit: a comparison of artificial neural networks with logistic regression models

    Crit Care Med

    (2001)
  • Kayaalp M, Cooper G, Clermont G. Predicting ICU mortality: a comparison of stationary and nonstationary temporal...
  • Kayaalp M, Cooper G, Clermont G. Predicting with variables constructed from temporal sequences. In: Proceedings of the...
  • Silva. Á. et al.

    Multiple organ failure diagnosis using adverse events and neural networks

  • A. Abu-Hanna et al.

    Prognostic models in medicine: AI and statistical approaches

    Methods Inf Med

    (2001)
  • R. Miranda, The effect of harmonising and standardising the nursing tasks on Intensive Care Units (ICU) of the European...
  • Pyle D. Data preparation for data mining. S. Francisco, CA, USA: Morgan Kaufmann;...
  • J. Le Gall et al.

    A new simplified acute physiology score (SAPS II) based on a European/North American multicenter study

    JAMA

    (1993)
  • Cited by (54)

    • The comparison of selected machine learning techniques and correlation matrix in ICU mortality risk prediction

      2022, Informatics in Medicine Unlocked
      Citation Excerpt :

      The data collected in intensive care units has increased and been used in data analysis and data mining fields in the past decades [11]. There is a great deal of data to improve the care of intensive care unit patients who are still unused or need further analysis for various reasons, including the lack of access and lack of human specialists, the preoccupation of doctors or novice specialists, busy doctors and paramedics, and lack of equipment and enough knowledge for data maintenance have ignored essential details; however, automated discovery tools based on different prediction models can analyze raw data by extracting information from machine learning algorithms for decision-makers; it is possible to make better decisions [12,13]. In the models of mortality prediction in the intensive care unit, we are always looking for answers to critical questions about how to reduce the risk of mortality and related factors for patients in the intensive care unit, identify and minimize mortality rates in intensive care units based on available data [12].

    • Early hospital mortality prediction of intensive care unit patients using an ensemble learning approach

      2017, International Journal of Medical Informatics
      Citation Excerpt :

      For this and other reasons, the evidence base for critical care practice is less well developed than for some other acute specialties. The unique combination of rich data sources from monitoring, and a complex, heterogeneous patient population, makes the ICU setting particularly well suited for the implementation of an assistant data-driven system which analyzes large amounts of raw data that could be overlooked by human experts [12]. The use of ICU data in early prediction of mortality is an attractive open area for investigation, both for reasons of quality and cost.

    View all citing articles on Scopus
    View full text