Thomas Goetz in Wired Magazine highlights Alexandra Carmichael and her decision tree for health decisions, along with 2 other scenarios. Alexandra is the founder of the CureTogether open source health research community. CureTogether is an innovative service that facilitates finding effective ways to address health concerns by active participation by people living with certain conditions, especially those resistant to conventional treatment regimes, such as chronic pain. (She is also an active and inspiring member in the Design for Care community, which is why I noticed and had affinity for her scenario).
Goetz’s article on decision trees (methods of structured decision analysis) suggests that they are an effective tool making better everyday health decisions. The reasoning is essentially based on the assumption that better health is a matter of inputs and outputs, which can be mapped and judged to determine a preferred course of action. Better inputs – food, exercise, lifestyle decisions – lead to better outputs, which are improved health measures and a healthier experience of life. Wired online even provides a decision tree mapping tool you can try.
Alexandra’s story is the most interesting and complex, of course. Here’s the decision tree they worked up for her, a lovely if simplified diagram:
I have to admit I found decision trees too be too neat and simple in my work as a systems consultant (analyst) in the mid-90’s. You learn in AI and knowledge elicitation that cognitive representations can be simplified, but if you over-simplify you lose the human texture of real life. Decision trees are retrospective constructions, not prospective. They are applied in a prospective way in the article, but they were not actually used for these case decisions, so their application is speculative.
When people are in the middle of a life decision, they often go by gut choice (Frank Kozik’s realization that he could lose his teeth if he kept smoking did not require a tree). What we call gut decision is based on dealing with the complexity by sensemaking – working through plausible narratives in the face of a changing situation. Alexandra’s seems like the latter, a sensemaking, not decision making situation.
Here’s why they don’t really work in these situations, and I quote from the article:
“Alexandra Carmichael spent a decade looking for a diagnosis. It took her another two years to determine the best treatment options.”
Decision trees are typically used to express the variables in situations characterized by regularity, such as computer programming and healthcare treatment decisions (both of which call the trees algorithms). Decision trees are applied retrospectively, based on observations from past experiences that are expected to hold with regularity in future applications of the same logic.
So if it took Alexandra two years just to explore treatment options, do you think she would be using a tree at the time to map out those options? Probably not, even if she wanted to. Long-term decisions take a long time because of the time necessary to trial and experiment with different options, to evaluate the trade-offs for each option, and to update that learning experience with all the other life learning that co-occurs and makes the process messier along the way. I would imagine that Alexandra, and the other two cases as well, made their actual life decisions based more on sensemaking.
So while decision trees could be used to describe prospective, anticipatory decisions (such as at least two of those in the article), they are not the best tool for the way people reason about the future. For one, by the time you’d have the information available to make the decision tree, the best path would be obvious because of what you learned over time. Two, important decisions are more complex than input-output. Over time people change their valuation of different options, they do not balance evenly on a simple flow diagram. A process more like dialogue (e.g., sensemaking) is what we often see – individuals talking with others who had similar experiences and judging the variables in personal terms. Three, people are quite poor at reasoning about prospective situations, about unknown futures. Attempts to simplify the process by reducing it to a pathway may help, but I would argue would not actually aid in a significant decision. Because an actual “decision” is not a simple act in time. It will often be a mix of actions and options that are kept open until or if needed. Again, that’s classical sensemaking.
I would like to hear from people about their own decision stories. Here’s my story that occurred while reading this very article:
I have to admit I don’t rush to read Wired or any magazine, but let them pile up for leisurely lunch reads. In fact, I was sitting in a walk-in clinic in Toronto waiting for my first-ever doctor’s appointment after becoming a Canadian resident in March. In fact, I had saved Ontario the burden of covering a cyst removal by having my US dermatologist do it last week when in Dayton. So my decision tree was: rapid infection + known doctor + easy appointment outscores Wait for return to Canada + free + new doctor. Again, in real time I would not use an algorithmic approach. The decisions are made by the necessities of emergence, the quality of certain breakdowns, and the need to fit life’s other goals in place with a mundane travel schedule.
Sensemaking is not just for the everyday health seeking situation. It is even more critical for expert decision making, although this is only now beginning to be recognized, in medical disciplines that are characterized by what they like to call evidence-based decision making. In my research on clinical informatics, I’ve found that residents and trainees use a decision-tree approach to diagnosis and treatment decisions. They have to – they cannot afford to take chances and have to “go by the book.” Senior clinicians, on the other hand, have a huge repertoire of experience and do not need to look up decision models. Their experience is most helpful for the unforeseen and complicated situations that trainees cannot and would not address. And if a situation is challenging to an expert, do you think they will be able to resolve it by using a decision tree? If the problem is not canonical, it needs to be addressed in the unfolding complexity of discovery and iteration. In short, experts rely on sensemaking, which is also called “expert judgment.”
From a health informatics perspective, these observations are in line with Gary Klein and other decision making researchers in cognitive engineering. Essentially, the more expert a decisionmaker is, the more variables they will have to consider, and their model becomes a lot more web-like and dynamic than a decision tree. Because experts have such a large repertoire of prior situations to draw from, their decision processes are more like pattern-matching (what Klein calls Recognition-Primed Decision making). They don’t step through branches of consequence, they just ‘know” from experience. When breakdowns occur and new situations arise, the “making sense of things” that follows leads to what looks like a decision, but should be seen rather as the output of a sensemaking process. I also think people can “make sense together” aided by well-designed resources and ubiquitous access to information. Sensemaking for clinical problem solving is the point of the Interpretive Collaborative Review.