By Chuck Gallagher — Business Ethics Keynote Speaker and Trainer
TL;DR: Chuck Gallagher, AI ethics speaker and author, argues that enterprise AI policies fail because they restate values nobody disputes — and that naming your company’s specific AI nightmares does what a binder full of principles never will.
Picture a marketing manager at nine o’clock at night. She’s got a campaign due at eight the next morning and a spreadsheet open in front of her — customer names, purchase histories, email addresses, somewhere around eleven thousand rows of it. What she needs is that list cut into segments. So she opens a chatbot, pastes the whole file in, and she has her segments in under a minute. Then she goes to bed.
She’d sat through the company’s AI training. She’d signed the acknowledgment form.
The policy said the organization was committed to privacy, fairness, transparency, and accountability in its use of artificial intelligence, and she believed every one of those words.
Not one of those four words told her not to paste the spreadsheet.
Why Do AI Ethics Policies Fail the People Who Need Them Most?
Reid Blackman made the case in Harvard Business Review this past May, in an article titled “What Are Your Company’s AI Nightmares?” His claim is not a gentle one. The standard approach to responsible AI, he says, is fundamentally broken — too slow, too vague, and too hard to communicate to the people who actually have to live by it.
His account of how we got here is worth sitting with. Before generative AI arrived in late 2022, companies handled AI risk in a more or less standard way. They built AI ethical risk programs — Responsible AI, AI Governance, whichever name got printed on the deck. Executives designed them. The programs focused mostly on writing enterprise-wide policies meant to explain how the organization would live up to its AI ethics values, which also go by principles, or pillars. Then generative AI showed up, and organizations updated those programs to accommodate it. Now AI agents are gaining traction, and Blackman expects most companies will try to update yet again.
He’s got an analogy for that last move, and it’s a good one. Suppose you hand him a guidebook for taking care of a cat. Works fine. The cat is happy. Then he gets a tiger, comes back, and asks you to update the cat guidebook to include tigers. You wouldn’t say sure, no problem, they’re both felines. You’d tell him he needs a different book altogether.
As an AI ethics speaker and author, I’ve watched a lot of companies ask for the tiger chapter. They don’t want a new book. Typically, a new book means admitting the old one didn’t work, and the old one cost real money, took eighteen months, and has a nice cover.
What Does the Governance Gap Actually Look Like?
In McKinsey’s 2026 AI Trust Maturity Survey, nearly two-thirds of the roughly 500 organizations surveyed named security and risk concerns as their top barrier to scaling agentic AI, ahead of regulatory uncertainty and ahead of technical limits.
A few years back I was engaged to consult with a company that wanted its systems reviewed. They wanted to know where the opportunity for an unethical choice was sitting out in the open. One of the first things we did was create a job cross-training program. On the surface it was about coverage, so that if somebody got sick or needed time off, another person could do the work. Underneath, it was about creating some discomfort and moving people out of roles where they had the run of the place.
Most employees were receptive. Heck, some of them saw it as a way to learn the organization and maybe get ahead. But Rosa, an executive assistant to the vice president of purchasing, was quite resistant, telling her boss she didn’t have time for such foolishness. When I raised her resistance with the executive team a month in, Marc — her VP — said she’d been with him for almost eleven years and there was no one who had a closer relationship with their suppliers than Rosa. She let nothing get past her. She ran a tight ship. Then he asked me, “Do I have a problem?”
The details are not important…but the outcome is. Rosa was forced, kicking and screaming, to swap jobs. It took ten days to uncover a massive embezzlement scheme that, fully investigated, came to well over $2.1 million in losses that otherwise might have gone undetected.
Nobody in that building would have told you they had a problem. They had values, and I expect they believed in them. Marc’s question was the first time anyone there said a nightmare out loud with a name in it, and he only asked because somebody had made him uncomfortable enough to ask.
Why Doesn’t “We Value Fairness” Change What Anybody Does?
Now, the way I see it, here’s the plain thing nobody wants to say in the meeting. A value is not a decision rule. I wrote in Every Choice Has a Consequence that we think preparing an ethics and compliance manifesto for employees to sign ensures that they’ll make intelligent, ethical, and legal choices. But it doesn’t…not even close! That was true of compliance binders a decade ago and it’s just as true of AI policies now. Nobody at nine at night, with a deadline at eight in the morning, stops and consults an abstraction. That’s not how people work, and it never has been.
I’ve spent years standing in front of rooms talking about my own choices and what they cost me, and I can tell you what was running through my head at the time. It wasn’t a values statement. I had a need, I had an opportunity, and I had a rationalization sitting right there ready to make the whole thing okay — nobody was going to find out, and I was going to pay it back anyway. Every choice has a consequence. But the consequence has to be visible at the moment of the choice, and not discovered eight months later across a conference table with a court reporter sitting in the corner.
“We’re committed to privacy” is invisible at nine at night. “If you paste a customer list into a public model, that list may end up inside the model, and we’ll learn about it from a reporter” — well, that one she can see. Same idea underneath both of them, if you think about it. Perhaps only one of them is doing any work at the moment it matters.
What Works Better Than a Binder?
Blackman’s alternative, which he calls the Ethical Nightmare Challenge, starts with three questions. What are the ethical nightmares of your organization as they pertain to AI? What resources will you build to avoid those nightmares? How will you train your people to use those resources effectively? This article takes up the first question. The next one takes up the other two.
A nightmare is not a risk category. “Privacy” is a risk category. A nightmare is specific, plausible, and it has a face.
The marketing manager didn’t need a values statement. She needed somebody, at some point, to have said out loud: here’s the nightmare — eleven thousand customer records walk out of this building through a text box, and we read about it in the trade press. She’d have recognized that one. My guess is she’d have stopped, or at the very least she’d have thought twice before she hit paste.
Frequently Asked Questions
What is the Ethical Nightmare Challenge?
It’s a framework from Reid Blackman, founder and CEO of the AI ethical risk consultancy Virtue, laid out in his 2026 book of the same name and in a May 2026 Harvard Business Review article. It replaces values-and-policy AI governance with three questions: what are your organization’s AI ethical nightmares, what resources will you build to avoid them, and how will you train your people to use those resources. Blackman’s line about it is that you were handed a guidebook for a cat and what you’ve got now is a tiger.
Why don’t AI ethics policies work?
Because a policy states a value, and a value isn’t a decision rule. I was a CPA. I knew the code of professional conduct. I stole from a children’s education trust anyway.
What is an example of an AI ethical nightmare?
A nightmare is specific and plausible. It isn’t a category. “Privacy risk” is a category. “Eleven thousand customer records get pasted into a public model by an employee under deadline, and we learn about it from a reporter” — now that’s a nightmare. People recognize scenarios. They don’t recognize abstractions, and I’d argue they never have.
Is organizational AI governance actually improving?
According to McKinsey’s 2026 AI Trust Maturity Survey, roughly 500 organizations found that average responsible AI maturity rose to 2.3 from 2.0 the prior year, on a scale of 1 to 4. Not bad. But Rosa’s employer hired me to review their systems, and I found that the $2.1 million wasn’t a policy; it was a cross-training schedule that made one person uncomfortable.
Who inside a company should identify its AI nightmares?
Not the executive committee alone, which is how most AI policies got written in the first place. As an AI ethics speaker and author, I’ve found that more times than not the useful nightmares come from the people closest to the work — procurement, claims, customer service, the engineer who built the system. They already know what could go wrong. Nobody has asked them out loud.
Before You Go
Marc got his answer in ten days because somebody made him say the question out loud. That’s the work I do with boards and leadership teams. If your company has an AI policy but has never named a single specific nightmare, you’ve got a gap, and the gap has a cost. Let’s talk about it at ChuckGallagher.com.
Five Questions for Reflection
1. If you had to describe your organization’s single worst AI outcome in one specific sentence — with a person, an action, and a consequence — what would it say?
2. Who in your company already knows that scenario is possible, and why has nobody asked them?
3. When your AI policy was written, who was in the room? Who was not?
4. Think about a decision you have made under deadline pressure. Did any company value cross your mind? What actually drove the choice?
5. Your governance program was built for the technology you had. Is it a cat guidebook, and did you just get a tiger?
