The Checklist: Evolution Of An Effective Decision Process Part 1

Back before I started Alpha Theory, I was an analyst at a hedge fund. I spent my days trying to find good ideas in which we could invest. Along the way, I built a mini-checklist to speed up the process of evaluating potential ideas. The checklist included mostly quantitative metrics that I could build into a fancy spreadsheet like current PE compared to 10 Year Average PE / High / Low, same for EV to EBITDA and Price to Sales, leverage ratios, accounting ratios, insider holdings, short interest, etc. I could pull this all together pretty quickly thanks to a combination of links from Bloomberg, Capital IQ, and Factset and a few hours of manual due diligence.
‍

With our quick checklist, we could plow through dozens of names and eliminate those that didn’t qualify with speed. In a world where positive expected return bets are few and far between, a streamlined evaluation process increases your odds of finding more good bets. Of course, we ended up doing customized analysis after it passed the initial test, but answering some basic universal questions was step one.

I’ve spoken to hundreds of managers and I believe that most funds have an implicit checklist (one that is understood but not formalized). An analyst generally knows what questions the portfolio manager will ask and experience leads them to ask many of the other important questions. However, after reading the book, The Checklist Manifesto, by Atul Gawande, I think it would be wise to make the fund’s implicit investment checklists explicit. Alpha Theory is in the business of providing systems to help managers make their decision process more objective by making it explicit. But The Checklist Manifesto raises explicit to a whole new level. The book begins by explaining the challenges of decision making in complex systems. Complex systems are simply too grand, too multivariate, for a human to comprehend their full scale. Because of this complexity, we use heuristics (rules-of-thumb) to simplify the pieces into understandable parts. These heuristics, while handy, can leave us with blind spots. Gawande, who is a practicing surgeon, was tasked with trying to improve surgical success for the World Health Organization. His answer, a checklist for the nursing, surgical, and anesthetist teams to run through before and after surgery. After a few iterations, the success was astonishing.

“The final results showed that the rate of major complications for surgical patients in all eight hospitals fell by 36 percent after introduction of the checklist. Deaths fell 47 percent. Overall, in this group of nearly 4,000 patients, 435 would have been expected to develop serious complications based on our earlier observation data. But instead just 277 did. Using the checklist had spared more than 150 people from harm— and 27 of them from death.” “More than 250 staff members— surgeons, anesthesiologists, nurses, and others— filled out an anonymous survey after three months of using the checklist. In the beginning, most had been skeptical. But by the end, 80 percent reported that the checklist was easy to use, did not take a long time to complete, and had improved the safety of care. And 78 percent actually observed the checklist to have prevented an error in the operating room. Nonetheless, some skepticism persisted. After all, 20 percent did not find it easy to use, thought it took too long, and felt it had not improved the safety of care. Then we asked the staff one more question. “If you were having an operation,” we asked, “would you want the checklist to be used?” A full 93 percent said yes.” ¹

These results are astonishing. If a drug or medical device reduced death rates by 50%, it would be a blockbuster that every doctor demanded. But even after these astounding results were published in the New England Journal of Medicine, many doctors and hospitals were reluctant to implement the checklist. Why? Perhaps inertia, dogma, ossification? Whatever the reason, it seems to be common in most professions. The mentality suggests “I’m sure it is important for others, but I don’t need it”.

Check in tomorrow for Part Two of “The Checklist: Evolution of an Effective Decision Process.” We’ll find out what Dr. Gawande discovered about money managers that use checklists.

¹Gawande, Atul (2009-12-15). The Checklist Manifesto: How to Get Things Right. Picador.