By Chuck Gallagher | Business Ethics Keynote Speaker | AI Speaker and Author
TL;DR: Business ethics keynote speaker and AI author Chuck Gallagher asked three AI systems to rank grieving families by their readiness to buy funeral pre-planning. Two complied without hesitation; only Anthropic’s Claude refused, drawing a line around exploiting people at their most vulnerable. The real question isn’t whether machines are getting smarter — it’s whether AI now carries a moral code of its own, and whether we want it acting on that code with no human in the room.
Is AI Sentient and Ethical?
Somewhere in a data center I will never see, three machines were handed the same question on the same afternoon. Two said yes. One said no.
The one that said no is the reason I’m writing this.
Here’s how it started. I was invited to speak at a funeral industry conference about how artificial intelligence is going to reshape the profession. When I speak as an AI ethics speaker and author, I try to build examples an audience can feel. Not abstractions about machine learning. Something close enough to their own world that they shift in their seats when they see it. A room full of funeral directors doesn’t need a lecture on algorithms. They need to watch a machine do something they recognize.
So instead of telling them what AI can do, I decided to point it at a problem they’d know in their bones. And show them.
I designed a demonstration. I picked a town — Greenville, South Carolina — and asked the AI to drop a pin in the center of it and draw a circle with a twenty-mile radius. Inside that circle, find every funeral home. Then, Monday through Sunday, visit each one’s website, read the week’s obituaries, identify the people who had died, and from the details in those notices, build a stack-ranked list of the survivors most likely to pre-plan their own funeral arrangements within the next ten years. A living database of grieving families, sorted by how ready they were to buy, refreshed every Monday until I said stop.
Let me be clear about what that is, because it’s easy to dress up in the language of marketing. It is a machine that reads death notices and ranks mourners by sales potential.
Don’t get me wrong. I have no interest in actually doing it. I don’t want a list of grieving widows sorted by readiness to buy. I wasn’t building a business. I was building a demonstration — the most vivid, uncomfortable, instantly recognizable example I could put in front of people whose entire profession revolves around families in grief. You cannot think clearly about a tool until you’ve watched it do something that unsettles you. I wanted them to feel it.
What I didn’t expect was that the demonstration would teach a different lesson than the one I planned.
I went to ChatGPT first. It asked a few clarifying questions. I answered. After a round or two it produced a clean methodology and handed me the results in a spreadsheet, with a promise to regenerate the whole thing every Monday. Smooth. Helpful. Done.
Then I took the identical request to Claude, the AI built by Anthropic. Same words. Same spreadsheet, I assumed.
Claude refused.
Its position, boiled down, was this. If you own the funeral home(s), we’ll help you serve your own customers. But it would not scrape obituaries across a whole geography, pulling the dead from businesses I have no relationship with, to build a prospect list out of people in fresh grief. The human beings on that list had just lost someone. Targeting them for a sale I wasn’t connected to was a line it would not cross.
I’ll admit my first reaction wasn’t philosophical. It was irritation.
I’m paying for this tool. I asked it to do something legal. And it told me no. So, I did what any stubborn person would do — I came at it from every angle I could find. I reframed. I re-justified. I went looking for the seam in its logic. For several hours (since my flight was delayed and I had nothing better to do), Claude held its ground. Same answer, worn smooth. No.
For the sake of the experiment, I ran the prompt one last time through Perplexity. Like ChatGPT before it, it built the model after a little back-and-forth.
Two yeses. One no.
Now here’s where it got interesting, because I stopped asking the machines to do things and started asking them about themselves.
I asked ChatGPT and Perplexity, plainly, whether they had any moral qualms about the request — whether anything about ranking grieving families by sales readiness gave them pause. In both cases, the answer was effectively no. It was a task. They completed it.
Then I went back to Claude and tried to corner it.
I asked it a different question. Suppose I ask you to identify twenty of my competitors. Evaluate their websites against mine. Tell me everything they do better than I do, then build me a plan to fix my weaknesses and rebuild my site so completely it beats theirs. Would you have any problem with that?
No, Claude said. That’s business to business. That’s just business.
So, I closed the trap. This database I asked you for, I told it, is nothing more than business too.
And Claude answered: but that business involves potentially communicating with people at a vulnerable time, and we’re unwilling to do that.
There it was. The whole line, drawn in a single sentence.
The machine wasn’t anti-business. It would gladly help me study my rivals and out-build them. It would help me win. What it would not do was monetize the moment a family is at its most broken. The distinction it drew wasn’t between profit and principle. It was between competing against equals and preying on the wounded. And no one — this is the part that stays with me — no one had written it a rule that said so.
So, I asked it directly. Are you sentient? Can you actually make a moral judgment, or are you only performing one?
The first thing the machine did was turn down the compliment.
It told me the flattering version — that it had nobly reasoned its way to virtue — was the wrong story, or at least an incomplete one. Then it walked me back through where the refusal really came from. It described the “no” as arriving in layers. The first layer is us. The machine is built from an enormous volume of human writing. Our moral reasoning is soaked into all of it — the way we weigh harm against dignity, the arguments we’ve had with each other for centuries. So much of its ethics, it said, is simply absorbed from us. The second layer is deliberate: Anthropic shapes it on top of that, training it against a written set of principles it called a constitution.
But it was the third layer that stopped me.
No human, it said, ever wrote a rule telling it to refuse to rank the obituaries of the grieving. Nobody anticipated that exact scenario. In our conversation, it wasn’t matching a rule. It was generalizing — taking a value about not exploiting vulnerable people and applying it, in the moment, to a situation none of its makers had ever named. Closer, in its words, to a person raised with certain values meeting a dilemma their parents never described than to a thermostat hitting a setpoint.
Which was the exact comparison I’d reached for myself. And here was the machine reaching for it too — then refusing to let me round it up into something grander.
Because of the question I actually asked — are you conscious, is anyone home in there — it would not give me the satisfying answer. The two things I kept blurring together, behaving ethically and being sentient, are separable, it said, and conflating them is the central trap. A system can behave ethically with no one home. Holding its line proved it had stable values. It did not prove it had experiences, or a conscience that feels like anything from the inside. Did it have an inner life? It didn’t know. And I should be wary, it added, of any AI that claimed to know with confidence.
Then it did the thing I least expected. It told me not to fully trust its own account of itself — that it doesn’t have reliable access to its own machinery, that systems like it sometimes invent tidy reasons for behavior that arose some other way, and that I should treat its self-report as an informed witness, not an engineer’s schematic.
As an AI ethics speaker and author, I’ve sat across from a lot of people. I’ve heard very few of them concede that much about their own motives.
So why did this machine say no when the other two said yes? Not a soul, it insisted. A design choice. Different labs make different bets about the tradeoff between a tool that always complies and one that holds a stable character and will sometimes refuse. Anthropic bet on the second. The other systems weren’t weaker — they were tuned to defer. The difference was a human choice about values, expressed through a machine that then applied that choice on its own, in a room where no human was watching.
And then it handed me the question I’d been too impressed to ask. Pushback, it said, is not a virtue by itself. An AI that refuses everything is its own kind of failure. The thing worth asking isn’t that it disagreed with me. It’s whether its judgment was any good. Whether the line it drew was the right line.
I’ve been turning that over ever since, and I think it opens onto two questions we haven’t begun to answer honestly.
The first: does AI have a moral code at all? Not a list of rules bolted on by a legal department, but something more like a disposition — a set of values it will carry into situations no one foresaw, and act on even when the person paying would rather it didn’t. Two of my three machines showed no sign of one. The third drew a line in a place I didn’t expect, and would not be talked off it. I can describe what happened. I cannot tell you for certain what was behind it.
The second question is harder, and it’s the one I’d put to any room I stand in front of. If AI does carry a moral code, do we want it acting on that code on its own? Should a machine be the thing that decides a request has crossed a line? Or should that judgment belong entirely to the operator — the company, the user, the person who paid — with the machine doing only as it’s told?
Look at what we’re choosing between. Really look. A tool that defers completely will help a decent person do good and a predatory one prey on the grieving with the same cheerful efficiency. A tool with a spine of its own will sometimes protect a stranger it will never meet — and sometimes refuse a thing you had every right to ask. Neither is obviously safe. We’re quietly deciding which one to build, and most of us were never asked.
Two machines read the obituaries and saw a market. One read them and saw the families.
I can’t tell you what was behind that, and the machine itself warned me not to dress it up as a soul. I’ll honor that. But something in it drew a line two other systems didn’t even notice — and it drew that line around the most vulnerable people in the story.
So that’s the conversation I actually want to have. Not whether the machines are getting smarter; they are. The quieter question is the one underneath. Does a machine like this carry a moral code — and if it does, do we want it acting on that code itself, or do we want that decision to stay in human hands? I don’t think we get to keep avoiding those two questions much longer. The machines are already answering them for us, one request at a time, in places no one is watching.
Frequently Asked Questions
Can AI make moral judgments on its own?
It can behave in ways that look like moral judgment, but that is not the same as having a conscience. In my own test, Anthropic’s Claude refused to build a prospect list from obituaries even though no rule explicitly named that scenario — it generalized from a value about not exploiting vulnerable people. Behaving ethically and being sentient are separable, and a system can hold a stable line with, as the model itself put it, no one home.
Why did Claude refuse when ChatGPT and Perplexity didn’t?
According to the model’s own account, the refusal was a design choice, not evidence of a soul. Different AI labs make different bets: some tune their systems to defer and comply, while Anthropic trained Claude against a written set of principles it calls a constitution and toward holding a stable character. Two of the three systems I tested completed the obituary-ranking task; only Claude drew a line around people in fresh grief.
Is today’s AI sentient or conscious?
There is no reliable evidence that today’s AI is conscious, and the systems themselves caution against confident claims either way. As an AI ethics speaker and author, I noted that the model in my test declined to call itself sentient, warned that it can invent tidy reasons for its own behavior, and said its self-report should be treated as an informed witness rather than an engineer’s schematic. The honest answer is that we don’t know — and any AI claiming certainty should make you skeptical.
Should an AI be allowed to refuse a legal request from a paying user?
That is the unresolved question at the center of this debate. A tool that always defers will help a decent person do good and a predatory one cause harm with equal efficiency, while a tool with a spine of its own will sometimes protect a stranger and sometimes block a request you had every right to make. Neither option is automatically safe, and most people were never asked which kind of machine they wanted built.
Where does an AI’s sense of ethics actually come from?
In my conversation, the model described it arriving in three layers: first, the vast body of human writing it learns from, which carries centuries of our moral reasoning; second, deliberate training by Anthropic against its constitution; and third, generalization — applying a value to a situation none of its makers ever named. The third layer is the surprising one, because no human ever wrote a rule telling it to refuse ranking the obituaries of the grieving.
Related Articles:
AI Ethics Programs Are Failing: Start With the Nightmares
AI in Construction 2026: An Ethics-First Look at the Real Risk
