Every Reformation Needs a Prince

On the evening of Friday, 17 November 2023, the board of a San Francisco nonprofit fired the chief executive of the most valuable startup in the world, and then discovered that it had not thought about Saturday.

Two of that board's six seats were held by people with direct institutional roles in a single philosophical movement, and reporting at the time counted a third. Helen Toner had worked at Open Philanthropy before Georgetown. Tasha McCauley sat on the board of Effective Ventures, which is roughly the holding company of organised effective altruism. Both also sat on the board of the Centre for the Governance of AI. By Wednesday the chief executive was back and both of them were gone.

That is the story I want to tell, and it is not the one I set out to tell.

By Saturday the board was on the phone to Anthropic. It had approached Dario Amodei about merging the two companies and handing him the combined thing. Reuters reported it within days, and Sutskever's later deposition and Toner's testimony at the Musk trial both describe the discussion. Toner has disputed parts of the account, specifically that she made the call happen and that other directors were behind a merger, while calling it an option worth considering among a set of difficult options.

Amodei declined. He had walked out of OpenAI two years earlier with his sister and most of the safety team, and being offered the parent back inside forty-eight hours is not a thing that happens to reformers very often.

By Monday, 738 of roughly 770 employees had signed a letter threatening to follow Altman to Microsoft unless the board resigned and brought him back. The count had been 505 that morning and kept climbing all day. Satya Nadella had already offered Altman and Greg Brockman somewhere to land, which turned the threat from a gesture into an address.

Among the signatures was Ilya Sutskever's, who had voted to fire Altman on Friday and now put his name to a document demanding that his own vote be reversed. He posted that he deeply regretted his participation.

Five days. A nonprofit board constructed, deliberately and at some expense, to be able to remove a chief executive in exactly this situation removed him, and the removal did not survive the weekend.

The comparison to a religious schism writes itself at this point, which is the first reason to be careful with it. Four laboratories, one method they all inherited from the same paper, a founding generation that keeps splitting, and public statements of belief about the end of history that read more like creeds than like product roadmaps.

I reached for that comparison inside a minute, which should have been the warning. It has already been written several times over. There are essays mapping Google to Catholic orthodoxy and the breakaways to Protestant traditions, and a small genre treating scaling as a holy war. The lineage-as-schism reading is not a new observation, and I spent the first week of this treating it as mine.

So take it as a hypothesis rather than a conceit, and run it against the record. The sociology of religion is an empirical literature with measurements in it, and it predicts particular things about groups that split over belief.

Applied to the companies, most of those predictions fail. Applied one level up, to the movement that staffed and funded and sat on the boards of those companies, they hold almost embarrassingly well. That is the correction this essay is, and I had it the wrong way round for a month.

A word first about who is in the sample. The four are Google DeepMind, OpenAI, Anthropic and the company formerly called xAI, which SpaceX acquired in February of this year and renamed SpaceXAI in July. I chose them because the question I started with was about American research laboratories with public doctrines, and because the four of them are connected by people who moved between them.

Meta is held out deliberately, and that exclusion turns out to matter. I will come back to it.

The received account of the lineage has DeepMind as the mother house, the transformer as the revelation, and OpenAI as the first defection. Demis Hassabis, Shane Legg and Mustafa Suleyman founded DeepMind in London in 2010; Google bought it in 2014; researchers at Google published "Attention Is All You Need" in 2017; and the breakaways proceed from there in a tidy line.

Almost every join in that chain is loose.

The transformer paper came out of Google Brain and Google Research, not DeepMind, and Brain and DeepMind did not merge until April 2023, six years later and five months before the board meeting above. OpenAI was founded in December 2015, two years before the paper existed, so it cannot have been founded over it.

And OpenAI was founded against DeepMind rather than out of it. Musk's own lawsuit describes Altman recruiting him with the pitch of a nonprofit that would act as a counterweight to Google's control of Hassabis. Ilya Sutskever, the closest thing to a scriptural author in the founding group, came from Google Brain, not from London. The first branch of the tree runs the opposite way from the one everybody draws.

Which leaves the question of what actually produces these splits, and there is a body of work on that already, none of it theological.

Steven Klepper spent a career on spinouts in the American automobile industry and in semiconductors, and the pattern he documented is the pattern here: new firms cluster geographically around a small number of unusually successful parents, they are founded disproportionately by high-status insiders who were blocked from strategic influence rather than by marginal employees, and they appear in bursts after governance shocks. Gompers, Lerner and Scharfstein found the same in venture-backed spawning.

I set a win condition before looking at the answer, because I wanted to be able to lose. If the spinout literature predicts the lineage, the timing, the geography and the identity of the people who leave, then schism is a decoration on top of labor economics and the essay should stop.

Run it. Amodei was a vice president of research who left over direction. Sutskever was chief scientist who left after losing a governance fight he started. Mira Murati was chief technology officer. Suleyman co-founded DeepMind and sold his next company into Microsoft. Senior insiders with influence but not control, departing after a shock, into the same three metropolitan areas, with capital waiting. Klepper called that in 1996.

That is where I expected to stop. The condition I set had fired and the interesting version of the essay was dead.

What kept it alive is that the spinout literature does not predict two things. It does not predict that these firms publish creeds; Fairchild's founders did not write essays about what the world looks like after transistors. And it does not predict that in November 2023 the money lost for five days. Every material interest in the building pointed one way and a nonprofit board pointed the other, and the fight took most of a week rather than most of an hour.

Labor economics has nothing to say about why that board existed, who put those particular people on it, or what they thought they were doing. For that you need the thing the companies came out of.

So here is the claim, stated flatly, and I will spend the rest of the essay bounding it rather than defending it.

Effective altruism is the only genuinely sect-shaped object in this story. Not one of the four laboratories is a sect by any measure the literature uses. The movement that seeded, staffed, funded and governed large parts of all four of them passes almost every test the literature applies, and it is the only thing here that does.

Start with what the strictness literature actually measures, because this is where I had it wrong for a month.

Laurence Iannaccone on why strict churches are strong, and Richard Sosis and Eric Bressler on nineteenth-century American communes, are about the costs of membership. Prohibitions, sacrifices, stigma, surrendered outside options. The mechanism is screening. Demands expensive to meet drive off people who are not committed, which raises the average commitment of everyone left and makes collective goods possible. Sosis and Bressler found that religious communes outlived secular ones, and that costly requirements predicted longevity only in the religious ones.

Now ask what it costs to join a frontier laboratory. A very large salary, equity, the best hardware on earth and the most interesting problem of the decade. That is the opposite of a screening cost. Whatever those companies are, they are not selecting for conviction by making entry expensive.

Now ask what it costs to join effective altruism. The Giving What We Can pledge is a public, lifelong commitment to give at least ten percent of income, enforced by nothing but the signatory's own conscience. It is voluntary, expensive, hard to fake, and correlated with actually believing the thing. It is, almost to the letter, the costly signal Iannaccone describes, and it is a hundred years of sociology arriving on time.

Dario Amodei signed one early.

The rest of the apparatus is there too, and it is not a stretch to name it. A shared text in Bostrom's Superintelligence, which is the book Musk was reacting to when he started warning about the thing he then funded. A career pipeline in 80,000 Hours that told a generation of unusually able graduates which jobs were worth taking. An in-group vocabulary dense enough to function as a shibboleth. Forums, conferences, forecasting tournaments, group houses. Entry stigma of the specific kind the model predicts, which is that explaining your giving to your family is awkward in a way that explaining a donation to a hospital is not.

And the thing I keep coming back to, because it is the part that is hard to get from the outside: this is a community where people genuinely changed their lives. Not their opinions. Their careers, their giving, in some cases where they lived. That is a high-demand group by any definition the literature offers, and treating it as a mere intellectual fashion misses what makes it worth studying.

Now the bounding, because "cult" is the word people reach for and it does specific work that does not apply here.

A cult, in the usable sense rather than the pejorative, has a living leader with authority over members, information control, and an exit penalty. Effective altruism has none of the three. It has no single leader; the nearest candidates are a philosopher at Oxford and a handful of foundation staff, and none of them can excommunicate anybody. It does not control information; the most cutting criticism of effective altruism has always been published by effective altruists, on their own forum, under their own names, which is not a thing cults do. And there is no exit penalty at all. People drift away from it constantly and nothing happens to them.

What it is, on the evidence, is a high-demand voluntary sect with unusually good epistemic hygiene and unusually bad governance instincts. That is a less exciting sentence than "cult" and it is the one the record supports.

Two more limits, and they both cut against me.

The lineage claim does not reach DeepMind. Hassabis and Legg founded it in 2010, before effective altruism existed as a named movement, and Legg's interest in AGI risk came from a different and older tradition. Anybody telling you the whole field fell out of one Oxford seminar room is selling the tidy version, and I was briefly one of them.

And the people running the stricter house do not claim the label. Neither Dario nor Daniela Amodei identifies as an effective altruist. Daniela is married to Holden Karnofsky, who co-founded GiveWell and ran Open Philanthropy, which is about as close to the centre of the movement as a person can stand. Both things are true at once, and a frame that cannot hold both is too crude to use.

So: not a cult, not the origin of everything, and still the only sect in the building.

Which brings the argument to the part that changed my mind, and it is about money rather than belief.

Effective altruism was not only the sect. For about six years it was also the patron.

In March 2017 Open Philanthropy granted OpenAI thirty million dollars, ten million a year for three years, and Holden Karnofsky took a seat on the board as part of the arrangement. That is a movement buying governance in a company it was worried about, using money from Dustin Moskovitz's Facebook fortune, and saying so in public at the time. Karnofsky held the seat until 2021 and stepped off when his wife co-founded the competitor.

On the other side of the field, Jaan Tallinn led Anthropic's Series A in May 2021, a hundred and twenty-four million dollars, with Moskovitz also in. And Sam Bankman-Fried, the movement's most famous donor and its most expensive mistake, put five hundred million into Anthropic through FTX and Alameda for roughly eight percent of it.

Read that list again. The same movement funded a board seat at OpenAI, led the first institutional round at Anthropic, and staffed half the governance of both. For a window in the late 2010s, effective altruism was the closest thing this field had to a prince.

And then it lost, twice, in two Novembers.

In November 2022 FTX collapsed and took its newest large funder with it, along with a good deal of the movement's moral standing, because its most famous adherent turned out to be running a fraud. Open Philanthropy had always granted more, and still does; what died was the money that had arrived fastest and asked the fewest questions. In November 2023, one year later almost to the week, the EA-aligned directors of OpenAI used the authority they had been given, and the staff and the patron removed them within five days.

The bankruptcy estate later sold the Anthropic stake for about one point three billion dollars in two tranches during 2024. Anthropic closed a sixty-five billion dollar round in May of this year at a post-money valuation of nine hundred and sixty-five billion. The best investment anybody in the movement ever made was liquidated early by a court, to repay the customers of the man who made it.

You do not need the religious frame to find that shape interesting, but the religious frame is where the shape has a name. A sect acquires a wealthy patron, the patron's money turns out to be dirty, the sect loses its endowment and then loses its seats on the councils it had bought into, and the institutions it helped build carry on without it, keeping the vocabulary and discarding the discipline.

This is also where the strictness literature pays its rent, because it predicts the sequel. Sosis and Bressler found that costly requirements predicted survival in religious communes and not in secular ones, and the mechanism was that a secular group cannot justify the cost when the cost stops being worth it. The laboratories kept the language of the sect, the mission statements and the safety teams and the published thresholds, without keeping a single one of its costly signals. On the model, that is the configuration that does not hold.

One case cuts against the easy version of this, and it is better for it.

When Daniel Kokotajlo left OpenAI in 2024, he refused to sign the non-disparagement clause in the standard exit paperwork and gave up vested equity reported at between one and a half and two million dollars, which he has described as the large majority of his family's net worth. He and his wife spent two months with lawyers before deciding. Expensive, voluntary, hard to fake. A costly signal in the strict technical sense, from exactly the tradition that trains people to make them.

Then the complication. When the clause became public in May 2024, immediately after Sutskever and Jan Leike left, OpenAI retracted it. Altman said that vested equity is vested equity, full stop, and that it was one of the few times he had been genuinely embarrassed running the company. The company released former employees from the obligation on 24 May and equalised secondary sales in June. Kokotajlo kept the money.

Two separate things are true in that. The sacrifice was real, and it was refunded.

And the clause itself was never a membership cost at all. It was an exit penalty, a device for suppressing departure after the fact rather than screening commitment on the way in, which is why it collapsed the moment it was visible. The one clear case of a researcher paying a genuine price for conscience ends with the price returned and the practice abolished, and the man who paid it learned to pay it somewhere other than his employer.

Meta's absence from the sample stops being a convenience here and becomes the control. Meta is the one large American laboratory with no eschatology, no public account of the end state, and no founder-prophet, and it is the one that behaves like a normal corporation, reorganising its research function and losing its chief scientist to a startup the way normal corporations do. Yann LeCun rejects the method itself, says large language models "can't truly reason or plan, because they lack a model of the world," and left in November 2025 to build AMI Labs on video and spatial training.

That is the only genuine doctrinal schism in the story. Everyone else fights about who holds the keys while agreeing entirely about the text.

And here is the price of it. AMI has been reported at around three and a half billion dollars. Sutskever's Safe Superintelligence, which has shipped no product at all, sits at roughly thirty-two billion after Nvidia put five billion in this July. The one man who broke over what to believe runs the smallest house in the story, and the man who broke over nothing but refuses to ship is worth ten times what he is.

One more line on this, because it cuts against me. LeCun's own exit quote was that you certainly do not tell a researcher like him what to do, which is a claim about authority, not about creed. Even the purest doctrinal break in the set has a control fight inside it.

Max Weber sorted legitimate authority into three kinds. Traditional, resting on inherited custom. Legal-rational, resting on office and written rule. Charismatic, resting on the person, on the conviction that this particular individual has something the rules cannot confer.

In November 2023 the OpenAI board held legal-rational authority in its purest form. It had the charter, the votes and the documented power to act. Altman held charisma, a patron and the staff. The contest ran for five days and the office lost.

Weber also said what happens next, and it is not that charisma wins permanently. Charisma is unstable by construction, because it cannot survive the person carrying it, so successful movements routinise it into offices, rules and inheritable structures. A great deal of what these companies built since 2023 is exactly that process, visible and dated.

Anthropic's answer is the Long-Term Benefit Trust, built around a class of stock called Class T, created when the Series C closed and held only by the Trust. It elects and removes directors on a schedule that phases in by time and by fundraising milestones, from one seat to a majority within four years. It carries almost no money: the shares are few and their economic rights limited, so the body accumulating the votes does not accumulate the upside.

OpenAI's answer, completed on 28 October 2025, put the operating company inside OpenAI Group PBC underneath the OpenAI Foundation, with the Foundation holding both control and conventional equity valued around a hundred and thirty billion dollars, after months of review and concessions involving the attorneys general of California and Delaware. Microsoft came out with a position valued at about a hundred and thirty-five billion.

Both are attempts to convert a founder's conviction into something that outlives him, which is the oldest problem in the sociology of religion. It is worth noticing that neither of them is an attempt to convert a movement's conviction into anything. The structures built after November 2023 protect the company from a repeat of November 2023. They do not give the sect its seats back, and they were not designed to.

Albert Hirschman supplies the other half. In 1970 he distinguished exit, leaving, from voice, staying and complaining, and argued that voice gets its force from the credible availability of exit. The letter is a textbook case. Seven hundred and thirty-eight people did not leave. They demonstrated that they could, to a named destination, at once, and that was sufficient. Exit was never exercised, which is why it worked.

Note who had voice and who did not. The staff had it because Microsoft had given them somewhere to go. The board had formal authority and no exit at all, because there is nowhere for a nonprofit director to defect to. On Hirschman's account that board was always going to lose, and the only surprising thing is that it took five days.

So the finding, and it has been sitting in the history the whole time. No reformation in the record ran without a prince.

Luther survived his excommunication because Frederick the Wise staged a kidnapping and hid him at the Wartburg, where he translated the New Testament into German at a nobleman's expense. The English Reformation ran on the Crown and the proceeds of dissolved monasteries, which is to say on confiscated land. The Peace of Augsburg in 1555 settled eighty years of religious war with the principle cuius regio, eius religio, whose realm, his religion, conceding that the unit of religious decision in Europe was the territorial prince. And Cardinal Richelieu, a prince of the Church, funded Protestant armies for thirty years because weakening the Habsburgs mattered more to France than the confessional question did.

Doctrine travels on somebody else's money, always, and the shape of a reformation is set by who is paying and what they want out of it.

For a while the movement was paying. Thirty million for a board seat, a hundred and twenty-four million to lead a Series A, five hundred million from a man who is now in prison. Set against what came next, that is the endowment of a wealthy abbey, and an abbey is what gets dissolved.

Compute is the land. It is the one input that cannot be improvised, cannot be raised from congregants, and is controlled by a very small number of parties who are not themselves laboratories.

Microsoft holds the largest outside position in OpenAI Group. It also agreed in November 2025 to invest up to five billion dollars in Anthropic, alongside Nvidia at up to ten billion, with Anthropic committing to purchase thirty billion dollars of Azure compute in the other direction. Amazon and Google are both substantial Anthropic investors. SpaceXAI resolved the question the most direct way available by having its founder's other company acquire it outright, which is less Henry VIII making himself supreme head of the English church than the same instinct with better lawyers: if the authority is going to be contested, own it.

One prince, Microsoft, is now funding two rival churches while holding a quarter of one of them. That is Richelieu exactly, and it is not hypocrisy. It is what patrons do. A world with two dependent laboratories is better for the patron than a world with one independent one.

This is also the test the stricter house has actually faced. In February the Secretary of Defense designated Anthropic a supply chain risk to national security, a label previously used for firms tied to foreign adversaries, after the company refused Pentagon terms permitting use for all lawful purposes and held lines on lethal autonomous weapons and mass domestic surveillance. Anthropic went to court, and in August Judge Rita Lin in the Northern District of California vacated the designation in a fifty-nine page summary judgment on First Amendment retaliation, due process and administrative law grounds.

The record is not a clean victory and I would be selling you something if I told it as one. Anthropic lost its bid for emergency relief in the D.C. Circuit in April, and the government's appeal of the earlier injunction is held in abeyance. But the company did refuse revenue on a stated principle, absorb a federal designation for it, and litigate for six months, while raising sixty-five billion dollars in the middle of the fight.

That is the most interesting data point against everything I have just argued, and it deserves to be stated at full strength. A company with no costly membership signal, whose founders do not claim the movement, did the thing the movement would have wanted, at a moment when doing it was expensive. Conviction survived the loss of the sect in at least one building.

Note also what made it survivable. Sixty-five billion dollars in the bank and a legal department. Not a monastery: a well-capitalised institution with a public creed doing the thing the creed said it would do, at a price it could comfortably afford. The Cistercians who broke from Cluny in 1098 because Cluny had grown rich were, within a century, efficient farmers and major wool traders who had to be reformed in their turn. The belief was sincere and the endowment grew anyway, and those two facts have never been in tension outside of sermons.

The question I started with was whether these breakaways are searching for a better operating system for large-scale human cooperation, and it deserves a direct answer.

An operating system for cooperation is a set of rules that makes people behave well toward strangers when nobody is checking, enforced by something other than a contract. Religions do it with belief, ritual and the threat of exclusion. Firms do it with pay and monitoring. Effective altruism tried to do it with a pledge, which is the religious mechanism with the metaphysics removed, and the pledge is the part that worked.

What the companies are attempting now is the same problem with the people taken out: a share class with votes and no money, a nonprofit holding a controlling stake, a published policy committing the company to stop at defined capability thresholds. An attempt to write conscience into a cap table, by organisations that watched what happened the last time conscience was held by individuals with board seats.

I think it mostly will not hold, and I would rather be wrong about that than right. A structure enforces what its enforcer is willing to pay to enforce. In November 2023 we learned what the enforcers were willing to pay.

Against the win condition I set, the lineage reading lost. The strictness reading did not lose; I had simply pointed it at the wrong object, and when it is pointed at the movement rather than the firms it does most of the work in this essay. The costume version, where each company is assigned a historical analogue and wears it for a chapter, should be abandoned entirely, because with four firms and four stock comparisons somebody gets flattered by nothing more than arrangement.

Weber, Hirschman and the princes survived. So did the sect, once I stopped looking for it inside the companies.

That carries a prediction, which is the only thing a frame like this is good for. If these are patron-backed institutions that inherited a sect's vocabulary without its costs, then restraint holds exactly as long as the patron finds restraint useful, and the informative moment is the first time it stops being.

Here is what to watch for, and you can check every item yourself from public filings and company blogs. A capability threshold in a published safety policy that gets revised rather than triggered. A refusal of revenue quietly not repeated once a competitor takes the contract. A resignation on principle that produces no letter. A trust or a foundation whose composition is adjusted in the year before a listing.

Any of those is the endowment winning.

The opposite counts as evidence too, and hold it to the same standard. A shipped product delayed past a quarter for a stated safety reason. A company lobbying against a rule that would have helped it. A governance body outvoting its own founders once. Anthropic's six months in court is one of these, and one is not a pattern.

Nothing in the creeds will settle it, because the creeds are written to be unfalsifiable and the people writing them mean it anyway. The structure will settle it. Who can remove whom, on what vote, and what happens to the money when they try.

In November 2023 the answer was that the board could remove the chief executive and the chief executive could remove the board, and the second turned out to be faster by a factor of about a hundred. The people on the losing side had given ten percent of their income for years to be the kind of people who would be trusted with that decision. It bought them five days.

The next time is coming, on a schedule set by two confidential S-1 filings and whoever is holding the compute when they price. The princes are already in the room. They have been the whole time, and they are the ones with the land.

Every Reformation Needs a Prince

Sources

Want this kind of automation working for your business?