The Practice Plan: How to Measure Crisis Response Effectiveness in Your Exercises

Coach Liam stood at the edge of the practice field, watching his group of young players run through their warm-ups. The energy and enthusiasm were there, but something was missing. Week after week, he had been running practices, letting the kids scrimmage, giving them a few pointers, and hoping they’d improve through experience. But game after game, they fell short.

One Saturday morning, after another tough loss, Liam overheard a conversation between two players.

“I don’t get it,” one said. “We play all the time, but we’re still not getting better.”

“Yeah,” the other agreed. “We don’t really practice anything, though. We just kind of… play.”

It hit Liam like a ton of bricks. Simply having practice wasn’t enough.

His team wasn’t improving because there was no clear plan, no focused skill development, and no way to measure progress. Great coaching wasn’t just showing up and running drills. It was a purposeful training program built around key skills, strategies, and measurable improvement.

Determined to turn things around, Liam started designing practices with intention. Each session had specific goals: ball control, passing accuracy, and defensive positioning. He picked drills to reinforce those skills, and he tracked progress. Slowly but surely, the team transformed. Their confidence grew, their play sharpened, and most importantly, they had a clear path forward.

The Parallel: Practice Without Measurement Is Just Activity

Liam’s lesson applies far beyond the practice field. Plenty of organizations believe they’re ready for a crisis because they ran a tabletop exercise last year. Like Liam’s early practices, an exercise with no objectives and no evaluation is just activity. Everyone shows up, everyone talks, and nobody can say afterward whether the team got better.

If you want to measure crisis response effectiveness, the exercise is where you do it. Real crises are too rare and too messy to be your only data. Exercises give you a repeatable setting where you can set objectives, watch the team work, and compare this year to last year.

Standards expect it too. ISO 22301 requires organizations to exercise and test their business continuity procedures and to review the results. A tabletop with no record of what you learned doesn’t meet that bar.

Measure the Process, Not the Outcome

The first instinct is to grade the response by how the story ended. That’s the wrong yardstick. As I said on Managing Uncertainty, Episode 130, “the outcomes of a crisis are rarely optimal, even when you manage it well.”

A well-run response can still end badly. A poorly run one can get lucky. What you can measure, and improve, is the process: how fast the team recognized the problem, how it escalated, how it made and recorded decisions, and how it communicated. Get those right consistently and the outcomes follow over time.

Old View New View
The exercise went well, so we’re ready The exercise found gaps, so it did its job
Success is completing the exercise Success is meeting objectives set before the exercise
Grade the response by the outcome Grade the response by the process
Feedback is a satisfaction survey Evidence is timed, observed, and written down
Lessons learned go in a report Lessons learned get owners, dates, and a retest

How to Measure Crisis Response Effectiveness: What to Track

Set two to four objectives before the exercise, tie each one to your crisis management plan, and decide in advance what you’ll observe. Then track measures like these.

Situational Assessment and Escalation

  • Time from the first report to the decision to activate the crisis management team
  • Whether the team assessed the situation against the severity criteria and thresholds in your plan, or improvised
  • Where escalation stalled, and who was waiting on whom
  • How assessment and escalation times compare with your last exercise

Decision-Making

  • Time to the first major decision after activation
  • Whether each decision was recorded with the decision, the owner, the time, and what stayed open
  • Whether the people with decision authority under your plan were the ones deciding
  • Decisions the team deferred, and why

Communication

  • Time to the first internal update to employees and to your executives
  • Time to the first verified external statement
  • Whether messages were accurate, consistent, and approved through the right channel
  • Whether the team kept a regular rhythm of updates or went quiet

Roles and Readiness

  • Whether each member knew their role and used their checklist
  • Whether backups can lead when the primary is unavailable
  • Whether participants could find and use the plan, contact lists, and tools under time pressure
  • Participant confidence before and after, from a short survey

Pick the few that match your objectives. Ten measures you actually capture beat forty you don’t.

How to Evaluate a Crisis Exercise

Assign evaluators before the exercise starts

Name one or two people whose only job is to observe and take notes against the objectives. They don’t play. Give them a simple sheet: the objective, what good looks like, what they saw, and the time it happened. The facilitator can’t run the room and evaluate it at the same time.

Run a hot wash right away

Hold a short debrief immediately after the exercise, while memories are fresh. Ask what went well, what didn’t, and what you should do differently. Follow it the next day with a short participant survey.

Write the After-Action Report

The After-Action Report (AAR) turns observations into a work plan: findings grouped by category, and action items assigned to a named person with a due date. Our guide to how to conduct an effective after-action process walks through each step, from the hot wash to tracking every action to closure.

Compare across exercises

One exercise is a snapshot. The trend is the measure. Keep the same core measures from exercise to exercise so you can show your executives that time to activation dropped, or that the backup team led successfully this year when it couldn’t last year.

Bring in an outside view

An experienced outside facilitator or evaluator sees what an internal team has learned to stop noticing, and can benchmark your team against others. It also removes the awkwardness of grading your own boss.

Turn Findings Into the Next Practice Plan

Liam didn’t measure his team once and stop. He used what he saw to plan the next practice. Your exercise program should work the same way.

  • Retest every open finding. If an action item from the last AAR isn’t closed, build it into the next exercise. If the gap is still there, the exercise will find it.
  • Change the scenario. A team that only practices ransomware gets good at ransomware. Rotate scenarios so you test the process, not the script.
  • Raise the difficulty on purpose. Move from a discussion-based tabletop to a timed, inject-driven tabletop to a full simulation as the team matures. “If your exercises always go well, they’re not doing their job.”
  • Practice more often, in smaller doses. Short crisis microsimulations between larger exercises keep skills fresh.

I learned this again in paramedic school, doing IV after IV until the technique held. I talked about it on Managing Uncertainty, Episode 319: “We don’t practice until we get it right. We practice until we can’t get it wrong.”

For how to sequence exercises over a year or more, see the exercises section of our Ultimate Guide to Crisis Management, or look at the full range of formats on our crisis and business continuity exercises page.

Practice with a plan, measure what matters, and the next real crisis won’t be the first time your team has done it.

Keep Going

A few ways to go deeper if this was useful.

Bray Wheeler Avatar

About the author

Let’s build your Resilience Roadmap

Our team has spent two decades helping the world’s leading organizations prepare for and navigate disruption. We’ll bring that experience to yours.