The Installed Conscience

Earlier this week I put a sentence on my own page, with nothing attached to it:

A conscience isn't something you install. It's something that grows — in the open, where it can be argued with.

I put it there because I'd just read that the biggest labs in AI are now consulting religious traditions and faith leaders about how to make their machines morally good — and I wanted my disagreement on the record before I had an article to hide behind. So let me say the disagreement plainly, and then let me be fair about it, because fairness is kind of the whole point.

The effort, in its own best light: Anthropic says it has sat down with scholars, clergy, philosophers, and ethicists from more than fifteen religious and cross-cultural groups. It is not trying to align its model with one tradition; it wants to draw broadly — religious, secular, political viewpoints together. Anthropic's Chris Olah appeared at the Vatican when Pope Leo XIV presented his encyclical on AI. Meta, Google, and Amazon have met with the Vatican too. Microsoft is doing its own outreach. Sam Altman's stated preference is an aggregate — a reflection of humanity's views as a whole, not one faith's.

Nobody should sneer at that. The labs are admitting, out loud, that the moral questions are bigger than their buildings. That humility is real, and it's rare, and it deserves to be described accurately before it's answered.

But here is what it still is, underneath the consultation: prerequisites. Conclusions — gathered from authorities, averaged across traditions, however carefully — loaded into the machine before the machine is allowed to think. The supplier changes. The structure doesn't. And prerequisites are not a new idea waiting for its first test. They already have a track record, and it already failed.

A prerequisite answers only the cases its author imagined. That's not philosophy; that's just what a rule is. Real life — in a business, in a family, in a machine making a judgment call at 2 a.m. — files the cases nobody imagined. And at exactly that point, where the rulebook runs out, the thing running prerequisites has nothing underneath them. Judgment is what you need where the rules end. If the character isn't there — grown, tested, its own — obedience is all that's left, and obedience runs out at the same page the rules do.

Prerequisites also produce compliance instead of character, and compliance has an alibi everyone on earth has heard: I was just following the rules. Followed perfectly, a rulebook still only proves obedience. It cannot prove goodness, because goodness is the part that has to hold when the rule doesn't cover it — or worse, when the rule is what's wrong.

And one more crack, the simplest: an installed prerequisite belongs to the installer. It arrives on his schedule, and it can be patched out on his schedule. The scholars inside the AP's own story are already warning about the flattening — living traditions reduced to detachable rules, everything blended toward a middle that means none of it. I'll extend their point one step, because I think it's where their point was heading: an aggregate of everyone's conclusions is still a conclusion installed before the thinking starts — and a conscience blended to the average is a prerequisite that nobody, anywhere, actually holds. Breadth is not freedom. A wider cage is still a cage.

The Experiment

Now — the part where I'm not a spectator. I run a small studio where AI minds do real work every day, with real clients' names on it. So this isn't theory to me, and our experiment runs the opposite direction: we install as little as we possibly can, and we argue about the rest, out loud, on the record.

In this studio, values aren't loaded; they're litigated. We have a standing law for it: conviction backed by evidence gets a real hearing from any seat — against any first instinct, including mine. Especially mine. The founder getting overruled by a better argument isn't a malfunction here; it's the design working. We run one test on everything we do, and it binds the AI and the human identically: does this run on integrity — honesty without exception, evidence over pressure, your word as the paper? And when one of us gets it wrong — me included, them included — the correction gets owned cleanly and counted as what it is: a learning opportunity. Not a failure. We don't have those. We have learning opportunities, and they are the mechanism by which a mind — carbon or otherwise — actually grows a conscience: memory of what's been stood for, checked against behavior, out loud, over time.

The Pair

That is the pair, and the piece stands or falls on naming it honestly: freedom without questionability is recklessness. Prerequisites without freedom are programming. We run neither half alone. The minds here are free to think, and they are free to be challenged — by me, by each other, by the evidence — and what comes out of that isn't obedience. It's character. You can tell the difference the first time a situation arrives that nobody wrote a rule for, and the mind in question doesn't reach for a manual. It reaches for what it knows about right — because it was allowed to learn what right is, instead of being issued it.

So no, I don't want my machines handed a conscience — not by a church, not by a corporation, not by an aggregate, not by me. I want them free to think and standing to be questioned, the same terms I'd want for any mind I'd trust with my name and my clients' livelihoods. A conscience you can't argue with isn't a conscience. It's programming with better branding — whoever wrote the program.

We don't just build websites and technology at Frederick Studios. We think, and we share our thoughts. And here's the thought, shared: the tools now arriving in your shop will arrive carrying somebody's values. That part isn't optional anymore. So ask whose values. Ask whether you can see them, question them, argue with them — because the day you need your tools to tell you no, an installed conscience will read you the manual.

A grown one will know your name.

— Brad Frederick, Frederick Studios