A Practical AI Risk Assessment You Can Run in One Afternoon

Ask most teams to run an AI risk assessment and they land in one of two failure modes. Either it's a checkbox exercise — a form with vague yes/no questions that gets filled out to satisfy a requirement and forgotten immediately — or it's an unfinishable research project, modeled on enterprise risk frameworks built for organizations with dedicated compliance staff, that a small or mid-sized team starts and never actually completes. Neither produces a usable answer to the only question that actually matters: is this specific AI use case safe enough to launch, and if not, what needs to change first. What follows is a method scoped deliberately to fit in one afternoon, thorough enough to catch the failure modes that actually matter, and simple enough that you'll run it again next time rather than treating it as a one-off.

The four questions that do almost all the work

A workable risk assessment for any single AI use case comes down to four questions, scored rather than answered yes or no, because risk is a matter of degree and a binary answer flattens exactly the distinctions that matter most.

Question 1: What happens if the output is wrong? Score this 1 to 5, from "someone notices and fixes it with no real consequence" (a brainstorming tool suggesting a mediocre idea) to "irreversible harm to a customer, employee, or the business" (an automated decision that denies someone a service, a benefit, or a job with no human check before it takes effect).

Question 2: How would you know it's wrong? Score 1 to 5 based on how quickly and reliably an error would surface. A tool whose errors are immediately obvious to any user scores low risk here; a tool whose errors are subtle, plausible-sounding, and likely to go unnoticed for weeks scores high — this is the dimension most risk frameworks skip, and it's often more important than the severity question alone, because a moderately bad outcome that's invisible for months compounds into something much worse than a severe outcome caught in minutes.

Question 3: Who or what is exposed? Score 1 to 5 based on scope and vulnerability — an internal tool used by one experienced team member scores low; a customer-facing tool, a decision affecting employment or credit, or anything touching a vulnerable population (minors, people in financial distress) scores high regardless of how good the underlying model is, because the exposure itself is what drives the risk ceiling.

Question 4: What data does it touch? Score 1 to 5 from fully public, non-sensitive information to regulated categories — health data, financial account details, anything covered by a specific compliance regime your business operates under. This question exists separately from the others because a use case can score low on all three previous questions and still carry serious risk purely through what data it processes and where that data goes.

Turning four scores into one decision

Add the four scores together for a total from 4 to 20, and use three straightforward bands. A total of 4 to 8 is low risk: proceed with basic documentation of what the use case is and who owns it, no formal review required. A total of 9 to 14 is medium risk: proceed with a named accountable owner, a documented human-checkpoint before consequential outputs are acted on, and a plan for how errors will be caught. A total of 15 to 20 is high risk: this use case needs a real review before launch — not necessarily a rejection, but a genuine look at whether the risk can be reduced (adding a human checkpoint, narrowing the scope, restricting the data involved) before it goes live as currently designed.

This banding intentionally mirrors the risk-tiering structure used in broader AI governance frameworks, because a one-afternoon assessment for a single use case should feed directly into your organization's larger governance approach rather than existing as a disconnected exercise — the scoring worksheet is how an individual use case gets sorted into the tier system, not a separate process running in parallel to it.

A worked example, start to finish

Consider a mid-sized insurance brokerage evaluating an AI tool that would draft initial responses to customer claims inquiries, with a human agent reviewing and sending every response. Question 1 (what happens if wrong): a wrong draft gets caught by the human reviewer before it reaches a customer, so this scores a 2 — low, because the human checkpoint contains the failure. Question 2 (how would you know): the reviewer sees every draft before it's sent, so errors surface immediately and reliably — another 2. Question 3 (who's exposed): customers are the eventual audience, even with review in place, which pulls this toward a 3 rather than lower, since a reviewer's attention isn't infallible and repeated exposure to plausible-sounding drafts can erode review rigor over time (a real, documented failure pattern called automation complacency). Question 4 (what data): claims inquiries often reference policy details and sometimes health or accident information, which scores a 4 given the sensitivity of what the tool is processing even though it's not making the final decision.

Total: 2 + 2 + 3 + 4 = 11, landing in the medium-risk band. The practical outcome isn't rejection — it's a specific, achievable set of conditions: name an accountable owner for the tool, formally document the human-review checkpoint as a required step rather than an assumed one, and set up a periodic audit where someone other than the day-to-day reviewer samples a batch of AI drafts against final sent responses, specifically to catch the automation-complacency risk flagged in question 3 before it becomes a real pattern.

Why this fits in an afternoon and enterprise frameworks don't

The scoping choice that makes this achievable is running it per use case rather than attempting to assess "our organization's AI risk" as a single undertaking. A single use case has a bounded set of facts — you can answer all four questions about one specific tool doing one specific job in under an hour, because the answers are concrete rather than organization-wide abstractions. Enterprise risk frameworks stall precisely because they try to answer sweeping questions before anything specific exists to assess; a per-use-case method sidesteps that entirely by never asking a question bigger than "this specific tool, this specific job."

Common mistakes when running this

The most common mistake is scoring optimistically because the person running the assessment is also the one who wants the use case approved — a structural conflict of interest that's worth naming explicitly rather than pretending it doesn't affect judgment. A second common mistake is running the assessment once at launch and never again, even as the use case's scope quietly expands — a tool that started drafting internal notes and later got extended to draft customer-facing messages needs a fresh assessment, not a grandfathered pass based on its original, narrower scope. A third is treating a low total score as permission to skip documentation entirely; even low-risk use cases benefit from a one-line record of what they are and who owns them, because six months later nobody remembers which of the dozen small AI tools in use were ever actually evaluated.

What to do with a high score

A high total isn't necessarily a stop sign — it's a signal to look specifically at which question drove the score and address that dimension directly rather than abandoning the use case. If question 1 (harm if wrong) is what pushed the total high, the fix is often adding or strengthening a human checkpoint. If question 4 (data sensitivity) is the driver, the fix might be restricting what data the tool can actually access rather than changing anything about its output. Treating the four sub-scores as diagnostic, not just as inputs to a total, is what turns this from a pass/fail gate into an actual tool for making a risky use case safer.

Running this across more than one use case at once

Once you've run the assessment on a handful of use cases, a second, higher-level value emerges: comparing scores across use cases tells you where your organization's real risk is concentrated, which is usually not where intuition would guess. A common pattern is that the tool everyone worries about — because it's new, unfamiliar, or uses a buzzworthy technique — scores low, while a quiet, long-running internal tool nobody thinks of as an "AI risk" scores high, usually because it's drifted in scope since its original approval without anyone re-running the assessment. Keeping a running log of every scored use case, not just the ones that felt risky enough to bother checking, is what surfaces this pattern; checking only the use cases that already feel risky is a selection bias that guarantees you'll keep missing the quiet ones.

Adjusting the method for your own context

The four questions and the 1-to-5 scoring scale are a starting point, not a fixed formula — a healthcare or financial services business operating under specific regulatory obligations should weight the data-sensitivity question more heavily, potentially doubling its contribution to the total, since a regulatory violation carries consequences the simple additive scale doesn't fully capture on its own. A business with very little customer-facing AI usage might reasonably decide the exposure question deserves less relative weight than the harm-if-wrong question. The point of keeping the method simple enough to adjust is that a risk framework nobody understands well enough to modify sensibly is a framework that quietly stops getting used the moment it stops fitting the situation in front of it exactly.

The AI Governance Playbook expands this exact scoring method into a full worksheet with additional prompts for each question, plus guidance on how individual use-case scores should feed into an organization-wide risk-tier system over time. The book's AI Risk Assessment Toolkit gives you the ready-to-use scoring template described here, and the broader Risk Management resource covers how to keep this kind of assessment current as a use case's scope changes rather than letting it go stale the moment it's first approved.