Can AI Do Pre-Mortems for Us?

Recently, I was informed that a large language model (LLM), probably ChatGPT, can do pre-mortems. A pre-mortem is a risk assessment exercise that is typically performed in 20-30 minutes at the end of a kickoff meeting for a new project. The project team, having been fully briefed on the project, is asked to imagine that an infallible crystal ball shows that the project has been a failure, an embarrassing failure. Then the team members are given two minutes to write down all the reasons wjy the project has failed, and the results are collected and displayed. The exercise is more effective than the usual question, "Anyone see any problems?" That approach usually comes up empty, whereas the pre-mortem exercise generates lots of thoughtful issues. Up to this point, pre-mortems have been done in person or via zoom.

But now, an LLM system can do a pre-mortem for us. We just describe your project and ask what can go wrong, and the LLM quickly answers. The output is surprisingly good and comprehensive.

Like many of us, I continue to be impressed by this new wave of LLMs, and now we have AI-based Pre-mortems—no more meetings, no more time spent coordinating with team members. It’s all at the tip of our fingers. Chalk up another victory for AI.

Perhaps. Now let’s step back and look at the downside.

What AI-Generated Pre-Mortems Might Be Missing

One obvious disadvantage is that AI systems miss the details and nuances of our specific projects, and these do make a difference. It misses things like the composition of our team—the abilities, attitudes, and workloads of the team members.

In my own work, I can recall a number of pre-mortem sessions in which the team factor came up. Once, it came up against me! Someone noted that Gary might not be sufficiently open to suggestions from the others on the team. (A fair concern.)

Another time, after a project went poorly in large part because the project leader got distracted and let the project drift without making timely mid-course corrections, we did an after-action review—a post-mortem—and we dug up the original pre-mortem. To our surprise, there it was. One item called out, “William [not his real name] gets distracted.” Exactly what happened. We spotted the problem in advance, but no one followed up on it.

These are the kinds of dynamics an LLM could not be expected to notice.

The Social Benefits of In-Person Pre-Mortems

Even more worrisome is the likely loss of team cohesion when we have LLMs do our pre-mortems. Our in-person pre-mortem sessions do more than identify potential problems. The sessions build a culture of candor, a culture of speaking out—of saying unpopular things and being listened to respectfully.

The sessions help us build respect for our colleagues as we listen to the ideas they offer. Pre-mortems aren’t a beauty contest; they are a competence contest in which we are all competing with each other to nominate clever showstoppers to impress each other. It is a healthy contest, part of team-building, that prepares us to collaborate more effectively. It also provides a framework for the “quiet” participants to have a voice—to be heard. What's more, when the pre-mortem sessions are conducted with the sponsor in the room, they help calibrate hopes and concerns so that the sponsor is more fully on board.

A friend of mine identified another advantage of pre-mortems: They let him gauge the people on the team, and sometimes identify people who should not be taking a decision-making role, people with the potential for disrupting the coordination of the team.

One lesson here is how AI can devolve social tasks and coordination into data tabulations—and we may be poorer for it.