A checklist for making better decisions

Duke University Professor Jack Soll explains how you can avoid bias and make better decisions.

At the heart of nearly everything in my professional life – including this newsletter, the Choiceology podcast, my research, my book, and my teaching – is a simple goal: to help you make wiser, more far-sighted decisions. This month, I’m breaking slightly from my usual newsletter format to share two new resources in direct service of that goal before turning to a Q&A with Duke University’s Jack Soll, who provides a verbal checklist designed to improve the quality of your choices.

A New Season of Choiceology

We’ve just launched a fresh season of Choiceology! Below is a trailer for our first episode of spring, which is focused on algorithm appreciation. Be sure to listen soon, and let your friends know that we’re back with an all-new slate of episodes arriving every two weeks.

A Glossary of All the Topics We’ve Featured on Choiceology

With the support of two incredible undergraduate research assistants (thank you Vidya Pandiaraju and Katarina Siggelkow!), I’ve created a new resource to help you keep track of all the different topics we’ve covered on past Choiceology episodes. It’s an interactive glossary: you can click the name of any topic listed and you’ll be taken to a short entry providing a definition, where to find the relevant episode, and the expert guest(s) featured. How might you use it? If you’re curious about a particular bias or concept, you can jump right to the definition and find the episode and experts available to teach you more. And if you’re a longtime listener who’s wanted a handy crib sheet summarizing the decision-making topics we’ve covered that you can easily revisit, your wait is over.

Q&A: A Checklist for Making Better Decisions

This month’s Q&A from Choiceology is with Jack Soll, a professor at Duke’s Fuqua School of Business. He offers a “checklist” you can follow to avoid some of the most common and harmful judgment and decision making biases.

Me: I’m hoping we could dive right into some strategies for better decision-making. I’m a big fan of checklists, so I’d actually love to get your list of the best advice in that format.

Jack: All right, a checklist. I have four pieces of advice here, which are probably incomplete. But here’s four.

So one is to be decision ready. So often, we fall back on our intuition and can be biased if we’re distracted, if we’re tired, if we’re hungry, if we’re really rushed, we don’t really have time to deliberate. And we often make better decisions if we actually: (1) know how to make the right decision — we know what principles to apply and (2) have some time to think about and apply those principles. So there are a lot of situations where we’re not decision ready. Maybe we’re angry at somebody, or maybe we’re trying to do multiple things at once and are distracted. And so at those times, it’s best to put off the decision to another time.

Me: That’s a great item number one. What else?

Jack: Yeah, sometimes we miss opportunities because we didn’t think broadly enough. And so a second piece of advice is to broaden the frame. A lot of biases actually could be attributed to thinking too narrowly. Maybe we’re only thinking about the one option that’s in front of us. Should I go to this college? Should I rent this apartment? And when we do that, we only have one objective in mind. We’re not thinking about all the things we care about. So imagine a student choosing a college who wants to be a software engineer. They might focus on which school has the best computer science department, and that might turn out to be the best choice for them. But if they stop and think about it, they might realize, “Well, in the future I might prefer a different career path” or “There’s other things I care about for choosing a college, like social things.”

So, try to think more broadly about what can happen, how might our preferences change, what can happen in the world? At the end of the day, decisions are limited by the option set. A lot of research has shown that people aren’t necessarily great at choosing well, or they might be OK at choosing between A, B, and C, but sometimes none of those options are the right option. Sometimes what you actually ought to do is throw out all those options and invent some new ones. And so that’s what I mean by broadening the frame.

Number three on the checklist is to take advantage of the wisdom of others. Seek advice. And when doing this, make sure to get independent advice. Meaning, don’t tell the other person what you think the answer is because this is going to influence what they tell you. They’ll either think that you’re asking for confirmation (and they’ll tell you what they think you want to hear) or their thinking will be influenced by your thinking.

Me: And what’s the fourth item?

Jack: Fourth: experiment. Try things that you think won’t work. Do this in a low-cost way. The idea here is to try to generate dis-confirmation, try to prove yourself wrong. One way to do this is empirical. So if you think you know what a website should look like, you could do some A/B testing, and you could create variations that are different from what you think are best to kind of test whether or not your idea is correct. But in other situations, experimentation is difficult, and there we have to rely on the wisdom of others. And one way to do this is to have a devil’s advocate, have somebody argue for an alternative point of view.

Me: Yeah, I love that. I often tell my students to try to find somebody who will play the role of devil’s advocate and just argue the opposite, too, when I’m worried that I might be too attached to a given path or argument and not thinking broadly enough.

Jack: Oh, for sure. You want to think about reasons why you might be wrong. We often think about why we’re going to succeed or why our answer is right. But consider why the answer might be much different from what you thought. Sometimes we’re all on our own, so this is a little bit challenging, but ask yourself why you might be wrong.

Another way to seek dis-confirmation is to tackle the same problem but on different occasions. Actually, this is a trick that a lot of professors use when they’re grading papers. They do it twice. And it takes extra time, but often what happens is, at any given moment, there’s some noise in our judgment, or we have different things in mind. And so if we look at the same decision problem twice or are grading the same essay twice, we might come up with different answers. And in those cases, if the answers are different from each other, you could either, if it’s quantitative, you can average the grades together. If it’s more discrete, or if it’s categorical, what you can do is look at it a third time or have somebody else look at it. There’s a lot of work along these lines in medicine where if you ask a radiologist to look at the same x-ray on multiple occasions, they might come up with different answers. And this isn’t really a critique of their expertise. This is just symptomatic of how difficult the task is. But for those tricky ones where they’re giving different answers on different occasions, then maybe a third look or bringing in another expert can be helpful.

Me: I love that, Jack. And P.S., it also makes me feel better about the fact that I’m that dissertation advisor who after six months reads a new draft from my student and says, “I really don’t like the third paragraph of the intro. You should cut it.” And they tell me, “Actually, I added it because you told me to.” Anyway, I’m often inconsistent in my advice to my students, but then I say, “Well, you’re getting the wisdom of the crowd within.”

Jack: Exactly.

Me: It’s like two independent perspectives. Aren’t you lucky? Anyway, that’s my framing trick.

Jack: There’s more than one Katy Milkman.

Me: That’s right. That’s right. Jack, this has been so enlightening and helpful, and I really appreciate you taking the time to talk with me and share these insights. Thank you so much.

This interview has been edited for clarity and length.

To learn more about Jack’s work, listen to the episode of Choiceology where we dig into his research on debiasing or check out our HBR article “Outsmart Your Own Biases“, which we co-authored with John Payne.

That’s all for this month’s newsletter. See you in April!

Katy Milkman, PhD

Professor at Wharton, Host of Choiceology, an original podcast from Charles Schwab, and International Bestselling Author of How to Change

P.S. Join my community of ~100,000 followers on social media, where I share ideas, research, and more: LinkedIn / X / Instagram / BlueSky / Threads