Back to List of Publications
Enrico Coiera(1)
Hewlett Packard Laboratories
Filton Rd., Stoke Gifford, Bristol, BS12 6QZ, UK
This paper appeared in the Proceedings of the International Conference on Medical Physics and Biomedical Engineering - MPBE '94, 5-7 May 1994, University of Cyprus, Nicosia, Cyprus, pp 130-142.
Abstract - The conceptualisation of user needs that drives much of the technological development in decision support for patient monitoring is inadequate or flawed. In an attempt to develop a clearer picture of such user needs, preliminary results of a study into the behaviour of clinicians performing real-time monitoring are discussed. The study informally suggests that a critical reassessment needs to be made of the type of decision support that should be provided in monitoring situations. More generally, the study results are used to argue that such clinician studies are generally lacking in the design of all clinical decision support systems.
It seems almost a truism to say that the role of a clinical decision support system is to assist health care professionals in carrying out their clinical duties. Our purpose in building them is also apparently straightforward - we wish either to provide assistance for tasks that are subject to human error, or to provide tools that augment human performance. The measured results of our efforts should either be reduced patient morbidity and mortality, or a reduction in the costs of providing care.
For example, decision support has long been advocated in patient monitoring, based upon the perception that monitoring constitutes a complex set of tasks that are prone to human error. Further, it is believed that such enhancements to monitors can lead to the earlier detection of significant clinical events, and that this will reduce morbidity, mortality and costs.
Unfortunately, while there have been many attempts at developing computer systems for decision support with patient monitors, they have failed to make any significant impact upon clinical practice. The current state of the art in monitoring reflects improvements in measurement fidelity, and the development of new measurements, rather than any advance in the way users interact with the monitoring system. In this paper, the motivations that underlie much of the technological development for decision support in patient monitoring are critiqued. Difficulties with the present conceptualisation of user needs are presented that may in part explain the current state of affairs. In response to these, preliminary results of clinician studies are presented that suggest a critical reassessment needs to be made of the type of decision support that should be provided in monitoring situations. More generally, it will also be argued that such clinician studies are generally lacking and that much clinical information system design needs to be reappraised in this light.
In many ways, work to date in the design of decision support systems for patient monitoring has been fragmented. On the one hand, there has been a steady effort concentrated on the technical aspects of diagnosis, monitoring and control of physiological signals [2]. On the other hand, some work has taken place exploring clinical errors in patient monitoring, for example in the delivery of anaesthesia [5] [15]. There is also some literature on human factors in the clinical workplace, identifying causes of suboptimal performance and suggesting areas in which decision support might be appropriate (e.g. [8]). Further removed from clinical medicine, there is a large literature on the cognitive aspects of process control [16]. These latter bodies of work help to characterise clinical tasks, and should provide the motivational impetus and focus for the technological development of intelligent patient monitoring systems.
Unfortunately, there seems to have been little connection between the technical developments of decision support systems, and the work on the cognitive aspects of clinical decision making that should motivate these developments. Yet it is the cognitive motivations that should identify which aspects of clinical tasks should be supported by a decision support system.
Much of the work done in monitoring decision support has paid lip service to the notion of task understanding, and has operated on a set of largely untested assumptions about clinical need. For example, a strong paradigm for the design of decision support systems has been the provision of diagnostic information in real time situations. This view was possibly motivated by early research in decision analysis which suggested that human judgements were prone to decision biases [12], and the corollary which stated that such flawed judgements could be normalised by relying on sounder, more formally based, diagnostic systems. There is now growing evidence that questions such an assessment. Based upon studies of individuals in the field rather than in controlled laboratory situations, evidence now suggests that the primary effort for decision makers is not at the moment of choice, but rather in situation assessment [13]. In other words, it may well be the case that the majority of clinicians do not have difficulty in making a diagnosis, but rather in establishing a clear picture of the state of the world, and clarifying their goals and assumptions, prior to attempting to make a diagnosis. If it is the case that situation assessment is the key bottleneck in decision making, this has implications for the design of decision support tools for monitoring and control. Systems that assist clinicians in making an assessment of monitored data may be of more utility than systems that attempt to manufacture a diagnosis.
With this background, we undertook to re-examine our understanding of clinical behaviour during patient monitoring situations, in the hope that the insights such a study might provide would translate into requirement specifications for a clinical decision support system.
How does one gain an understanding of the needs of clinical practice? There are a number of alternatives, which can be roughly grouped into the following four categories:
Anecdotal or individual experience: At the most superficial level one can draw upon individual experience, or rely on anecdotal evidence from other colleagues, to help pinpoint the tasks clinicians need help with. This method usually operates within a framework of the existing assumptions about user needs.
Asking clinicians: If one wants to be a little more methodical, one might conduct a survey, and ask clinicians about their needs. While this may seem attractive, it has its problems. The biggest is that it is unlikely that clinicians really know what they want or need - "Although users are expert at what they do, they have difficulty predicting what they would like" [8].
More specifically, both these approaches suffer from a number of methodological difficulties. The problem of attitudinal bias has long been understood in the social psychology literature - people's actions differ from their verbalised responses [18]. We don't necessarily do as we say. Secondly, these methods require a degree of self-reporting. When asked why they made a certain judgement, or how they solved a particular problem, people are capable of providing apparently plausible reports of their mental events. However, asking people to introspect about their behaviour is a contentious investigative method in psychology because verbal reports of influences on behaviour may not be valid. Since there is no method of independently checking the validity of self reports, the details of cognition remain private to the individual [15].
Non-participatory observation: The next stage is not to ask clinicians what they want, but to study the way they work in the field. There are many approaches that one could take, including techniques derived from ethnography [8], as well as the more traditional software design methods of data-flow and task analysis. Here researchers try to understand the demands made upon individuals through detailed observations of them engaged in routine tasks. By making such observations, one is in a position to identify needs that are not apparent to the individuals being studied.
Formal psychological experiments: Finally, at the most detailed and scientifically rigorous level, one might have a set of hypotheses about specific aspects of clinical behaviour, and set up controlled studies to test them out. Some of the classic work in clinical decision making has been of this type [6].
While it may seem that the ideal method of proceeding is to invest in formal psychological studies of clinical decision making, this is not necessarily the case. Laboratory experiments run the risk of treating decision making as an abstract process, independent of the vagaries and interrupts of working life. By their nature, such studies are conducted in laboratory conditions which factor out the interruptions and pressures of the real clinical workplace. In the real world, time pressures mean that short cuts are taken, and that people and events interrupt clinical activities.
What we need to do is to characterise medical decision processes as much as possible in the way they really occur, as opposed to the way they should occur. This will give us the best chance of designing systems for use in such environments, as opposed to designing what would be needed if everything else was ideal. It may well be the case for example, that in an ideal consultation a doctor may want to consult a computer to help with the diagnostic process. However, with the pressures of the real workplace, the bottlenecks to increased performance may be much more mundane and less glamorous, and may be more closely associated with easy and timely access to information, or better communications between professionals, or even simply reducing the amount of paperwork in the clinical workplace.
This argument strongly favours pursuing non-participatory methods in the field, such as the ethnographic approach. These are sufficiently formal to allow robust statements to be made about results, and have the advantage of being grounded in the realities of the clinical workplace. Formal experiments will only answer narrow questions and are relatively expensive to conduct - one must therefore have a broad understanding of the domain to make a value judgement about the most likely avenues for investigation prior to making the investment in experimentation.
Most analyses of monitoring practice have sought to focus on the causes of errors that have an impact on patient care [4][5]. This approach is limited in several ways. Firstly, categorising and tabulating errors says nothing about the behavioural context within which they occurred, specifically the sequence of behaviours that led up to them. Secondly, while errors are clinically important, they account for a very small portion of the total behaviour of a clinician [14]. Thirdly, the emphasis on removing causes of errors is one sided. We are unlikely to ever be in a position where all causes of error have been eliminated. We should thus acknowledge that errors and mishaps are inevitable and rather than solely focusing on the design of practices and support systems to eliminate error, we should aim to design ones that are explicitly intended to cope with error [1].
An alternate strategy is to examine the routine behaviour which occupies the bulk of clinical practice. This approach has two advantages. Firstly, by focusing on the elements of normal behaviour that are responsible for sub-optimal clinical performance, we may be in a better position to suggest useful forms of behavioural support. The emphasis here is on optimising behaviour which clinicians spend the majority of their time engaged in, rather than supporting critical but rare failures. Secondly, major errors usually arise in the context of a sequence of minor errors [10][4]. By analysing the dynamic context within which minor errors occur, as opposed to simply identifying and categorising them, we may obtain a clearer idea of how to prevent them. An important consequence may be that potentially critical errors can be prevented by optimising "normal" clinical practice. An analysis of normal behaviour should also assist in exploring the ways in which clinicians routinely successfully recover from incidents, since in the main, most significant events are successfully dealt with and do not result in negative outcome for the patient [9]. Equally, we are interested in identifying the circumstances under which such strategies fail.
In an attempt to understand the types of decision problem facing clinicians in a real-time monitoring environment, a study of anaesthetists during cardiac bypass surgery was undertaken at two large teaching hospitals in the United Kingdom. While the full results of this study will be reported elsewhere, a brief description of the type of analysis performed, and the type of information that it supplies will be presented here.
The study aimed to collect qualitative data about the types of behaviour anaesthetists exhibit when monitoring patient data, and to generate a cognitive model of the monitoring strategies that they employed. The eventual aim of this work is to design monitoring decision support systems that naturally blend in with the anaesthetist's work practices, and support the tasks with which they experience difficulty.
The study involved 12 clinicians with a range of expertise from junior registrar to senior consultant anaesthetist. Each of the subjects was followed though one surgical procedure of between 2 and 4 hours in duration. The data collection commenced with a preoperative interview with the subject. The interview tried to capture the subject's understanding of the patient, the intended surgical procedure, and the intended anaesthetic. During the operation a variety of data were recorded, including video and audio records of the anaesthetist interacting with the monitoring and anaesthetic machines. A second shadow clinician who was not responsible for patient care acted as an observer, and recorded all therapeutic interventions and clinically significant events. This event record served as an annotation to the real-time data that was gathered from the monitoring systems attached by the subject clinician to the patient. Subsequent to the operation, a short interview obtained the clinician's reactions to the operation, attempting to gather data on how the clinicians's preoperative expectations were matched by the actual outcome of the operation.
Within a week of the operation, each subject was re-interviewed for about 2 hours, during which their performance was explored with reference to two or more clinically interesting portions of the operation that had been recorded. The data from the operation were available to act as prompts for the subjects' memory.
Once the entire data gathering exercise was completed, the data were extensively analysed, and episodes that demonstrated errors or potential errors were examined, along with periods of good performance. The result of the analysis was a description language of anaesthetic behaviour, and a number of hypotheses about the way anaesthetists work in the monitoring environment. These hypotheses form a cognitive model which will allow us to further explore anaesthetic behaviour, and develop advanced decision support systems for monitoring tasks.
Several events were culled from each of the operations studied, and used as the subject matter for each of the 2 hour post-operative interviews. These events ranged in duration from a few minutes to periods that were 30 to 40 minutes long. The subject was interviewed in an attempt to gain an understanding of the intention behind observed clinical actions, and to obtain as complete a description as possible of the data used by the subject in framing each clinical decision.
In the following example, the behaviour of an anaesthetist is presented as he deals with events during an operation involving a coronary artery bypass graft. The event is reconstructed both from the data gathered during the surgery, as well as from the post-operative interviews.
The subject anaesthetist in this example was an experienced clinician who had been practising for several years. He had substantial experience with the type of cardiovascular surgery for which he was providing anaesthesia.
The patient was a 47 year old gentleman who had a fairly large myocardial infarction several months prior to surgery. Following the myocardial infarction he had had recurrent episodes of left ventricular failure and subsequently developed a left ventricular aneurism. The patient was admitted to have the aneurism excised.
The problems noted by the subject anaesthetist prior to surgery were recurrent bouts of ventricular failure with episodes of pulmonary oedema that had not responded to conventional medical therapy. The patient's relatively poor left ventricular contractility with dyskinesia from the aneurism was considered a problem by the clinician, and he suggested that the patient could require fairly intensive haemodynamic support when coming off cardiac bypass.
The event demonstrates a gradually declining SaO2 level post bypass, probably caused by pulmonary oedema (Figure 1.). The downward trend started off with an SaO2 at 100 immediately after bypass, and dropped down to a low of about 87. The clinician did not detect the trend, despite it reaching clinically significant levels, until he took a routine blood gas sample about 30 minutes after the drop had commenced. As usual with most clinicians we observed, the alarm capabilities of the monitoring device had been suspended - many clinicians perceive that patient monitors have a high rate of false alarms. Consequently, no alarm sounded. In some sense, however the presence or absence of alarms was irrelevant, since the downward trend would have been visible for quite some time before the SaO2 dropped into the alarm region.
The goal of event analysis is to uncover what the clinician was doing during this period, understand what his objectives were, and what factors may have contributed to his failure to notice the clinically significant fall in oxygen saturation.
The chronology is as follows. The subject normally had his patients coming off cardiac bypass with an infusion of an inotropic agent running to support cardiac contractility. In this particular case, the provision of inotropic support was not the surgeon's normal practice. The surgeon preferred instead to have the patient on an infusion of glyceryl trinitrate (GTN). Thus, counter to the subject's normal plan, an intravenous infusion of GTN was running at the end of the cardiac bypass.
The subject was concerned about the patient's cardiac performance, as demonstrated in the preoperative interview. In the absence of his preferred strategy, he gave the patient repeated doses of Calcium Chloride (CaCl) (Figure 2.) This was given for its inotropic action - rather than giving a more potent inotrope which would run counter to the surgeon's expressed practice. The clinician had a rule that by 20 minutes after bypass the various pressures should have returned to normal range. Since this had not happened, he finally felt justified to depart from the surgeon's practice and commence treatment with standard inotropes, and so administered adrenaline at 15:56.
At 16:12, he started to take a routine blood gas. In so doing noted that the SaO2 value was lower than acceptable, and immediately reacted by increasing the patient's inspired oxygen (FiO2). He then commenced hand bagging, and increasing the end-expiratory pressure on the ventilator (PEEP), aiming to treat pulmonary oedema. The routine blood gas value taken at 16:12 showed an arterial PO2 of 7.2
There were several possible explanations for the prolonged drop in saturation. Firstly, the lungs may have been partially collapsed after bypass and choosing to hand bag the patient with a bit of positive pressure to try and re-expand them reflects that the clinician entertained this possibility. In view of the patient's previous history, the tightening of the left ventricle following surgery, and the high filling pressures, pulmonary oedema was a more likely explanation. The addition of PEEP and increase in FiO2 were commenced in response to this possibility.
The next stage in our study involved analysing each event, and for this we needed to develop a descriptive language with which to bring some order to the data set. To this end we developed a classification system for clinical actions. Actions were classified as either strategic or reactive, based upon whether the subject had intended to carry the action out in a preplanned way, or whether the action was in response to a clinical event that had not been anticipated. Events were also classified according to whether they were associated with monitoring, diagnosis or control. This simple structure defines a vocabulary of 6 possible classifications for clinical events. (Figure 3.).
Using this vocabulary, the sequence of the subject clinician's actions was labelled for our example event (see Figure 4.). Much of the clinical activity throughout the drop in saturation was seen to be reactive. In other words, the subject was forced to deal with a combination of events for which he had not developed a preplanned response. In particular, he was faced with a poorly performing heart, and pulmonary oedema. However, as a skilled and experienced clinician, these factors on their own would not be expected to trouble him. The third confounding element of the scenario was that the subject was forced to adopt the therapeutic plan favoured by the surgeon. This was not the subject anaesthetist's normal way of working (having to place the patient on a regime of GTN for preload reduction, when he would rather give direct inotropic support).
It appears that this combination of circumstances led to an period of intense concentration, when the subject was preoccupied with the management of the patient's cardiovascular performance. One consequence of this preoccupation was that, despite looking repeatedly at the monitor, the subject did not detect the saturation trend.
The video and physiological data provide sufficient evidence to substantiate the hypothesis that the subject was preoccupied with issues of cardiac performance. From the physiological data, we can see he switches the CVP line to measure LA pressure repeatedly during the period between 16:05 and 16:10 (Figure 5.). The intraoperative video demonstrates the subject repeatedly spending long periods in front of the monitor, dispelling any doubts that the SaO2 value was missed because there had been any failure to use the monitoring equipment.
With this single example, one would not want to draw anything more than preliminary conclusions. A common wisdom states that one of the major problems facing clinicians is data overload (in other words, they have difficulty taking in the large amount of data available to them on their patient monitoring systems). It may at first glance appear that this is a reasonable explanation for this event - that the clinician simply had too much to look at, and so missed a vital piece of clinical information. The evidence however, does not support this. There was ample time available to look at the monitor (in excess of an half hour).
Based upon evidence both from the actual operation and subsequent interviews, it seems reasonable to suggest that he was trying to develop a mental picture of cardiac function over the period in question. This involved the integration of several pieces of indirect evidence (the cardiac pressures) and therefore involved the constructing of some kind of mental model of the evolving cardiac haemodynamic status.
At particular points, there is clear evidence of focus on this activity - especially the switching behaviour around 16:05-16:10. This period of activity reflects the intensity with which the subject was pursuing the goal of assessing myocardial performance.
Cognitive psychological theories of attentional resource allocation suggests that the subject's difficulties in detecting the SaO2 trend may have been due to this reasoning process swamping his available mental resources. So intense was the activity, that there were no mental resources available to attend to even simple monitoring activities like looking at parameters other than the pressures. Fundamental cognitive principles indicate that unless a clinician specifically allocates attentional resource to the perception of something like the SaO2 numeric, that under such a heavy load the only thing the clinician will be able to reliably report is the numeric's colour [11].
We hypothesise that the clinician was not data overloaded; in fact he was suffering from processing overload. In other words, in the absence of direct data, the clinician's attention is being devoted to interpreting the meaning of several pieces of indirect information. This correlates well with the observations made at the beginning of this paper that situational assessment represents the primary effort for a decision maker; that most effort is devoted not to individual decisions such as diagnosis, but to establishing an understanding of the state of the world upon which such decisions can be based.
The notion of data overload has long driven monitor designers to consider that the simplification and abstraction of monitored signals is an ideal to be aimed for. The idea is to reduce the burden placed on clinicians swamped with data by presenting them with information that has been condensed in some useful way.
The notion of processing overload developed from the previous example suggests that monitor displays should be constructed with a quite different aim - to assist clinicians with the cognitive tasks that tax them. This may indeed mean presenting even more information than we expect, if that information assists them in completing the tasks before them.
In our example, the clear presentation of information about cardiac performance may have helped our subject clinician. For example, there may be clear ways of assisting with the visualisation of clinical data that makes estimation of cardiac contractility an easier cognitive task. Even simple grouping of data may be of help.
The concept of attentional load becomes a central one in this discussion, as it becomes clear that a clinician's attention is a finite resource whose use should be optimised. Thus, rather than presenting clinicians with alarms that immediately demand chunks of attention (and indeed may distract attention from important tasks at hand), we should aim to unload the clinician and free up attentional resources. If we recognise that the clinician is the one best able to handle the complexities of decision making in the domain, then we can begin to design systems that get out of the way, and instead help free up the clinician's inherent problem solving abilities.
For applications like patient monitoring, the complexity of the tasks for which the systems are used make it difficult to specify how information should be displayed. Yet it is unusual to find clinical support systems that have their designs based upon a detailed understanding of the way clinicians handle information. Some of this information does exist in the literature, but more specific information about the behaviours and tasks of clinicians is lacking.
In the clinical example presented here, the complexities of the processes involved in understanding how a clinician might come to miss a simple but critical piece of information are evident. If we are to design systems that support such complexity, then we need to invest in studies that make clinical decision processes explicit.
More generally, the development of advanced decision support systems in health care settings is a challenging task that requires a sensitivity to the unique characteristics of that environment. To date, most developments in this area have been motivated by the perceived need for decision support without any clear data pointing to its particular nature. What is now needed is a concerted effort to understand the nature of clinical practice, and in particular the nature of the user tasks that we hope to support in the future. Such studies need not be of an extravagant nature, but they do need to be methodical and focused in their approach. To assist this endeavour, there is a significant body of work in cognitive psychology and human computer interaction that can be harnessed. Our goal, as always, is to develop clinical information systems that have a genuine and positive impact on the delivery of health care, and with the right tools and approaches, this will be a singularly achievable goal.