Skip to main content

°FAI: AI companies more powerful than governments

Karen Hao asked John Borthwick at CUNY whether we are in an AI bubble, and he said “absolutely.” Borthwick runs Betaworks and has lived through two of these cycles already. He calls this a productive bubble, like the railroads or the fiber buildout: too much money goes in, things break, and the survivors inherit the infrastructure. °FAI: AI companies more powerful than governments

Hao’s answer to that is specific. Railroad tracks lasted a century. “The chips depreciate really, really fast,” she said, and if adoption takes five years to catch up with the buildout, “no one’s going to want to use those data centers anymore.” The scale is hard to picture. In an internal memo that The Information reported last fall, Sam Altman set a target of 250 gigawatts of data center capacity by 2033. At roughly $50 billion per gigawatt, outside analysts put the bill between $10 and $12.5 trillion. New York City draws about five gigawatts on an average day. Hao did the math out loud: “50 New York cities of data centers and supercomputers to ultimately build, like, an AI slop talk.”

The Pentagon deals OpenAI and Anthropic signed are not, as far as she can tell, about weapons. The companies are “working on bureaucracy,” helping officials write emails and reports. The real military AI is small and old. The Israeli Lavender targeting system, according to the +972 investigation, ran on “a linear regression algorithm.” So why court the Pentagon at all? Her answer: “they are also aware that the bubble is going to pop at some point, and so they are trying to integrate themselves as deeply into the state as possible such that they become too big to fail.” Maybe this is all conspiracy theory, but it offers another perspective. At the time she added a caveat: “I could be wrong, but this is my understanding.”

Nine months later, in her Bloomberg interview, Hao described something different. The model she had said wrote emails now, by her own account, picked targets. Mishal Husain asked her straight out whether these companies now hold more power than governments. “They are definitely increasingly becoming that way,” Hao said. “Even the US government, the most powerful country in the world.” Her example was the fight between Anthropic and the Department of War. The department, in her words, “used the nuclear option”: it threatened to declare Anthropic a supply-chain risk, which would have cut the company out of work across the federal government. Anthropic did not fall in line.

Husain pushed back. Didn’t Anthropic just take the moral high ground and pick up a wave of new users from the publicity? Hao agreed, and said that was exactly her point. “Anthropic won in this situation, both PR and in this spat.” A company will now go against the US government because “whatever penalty comes at them from the government might actually be counterbalanced by the other benefits that come directly to the company by profit elsewhere.” The market pays better than the state can punish. That is a new kind of power, and she calls it the governance structure of an empire: one person at the top makes decisions that land on billions of people, and none of them have a formal way to push back.

She refused to make Anthropic the hero of the story. When Husain asked whether it is a more principled company than OpenAI, she said no. Dario Amodei, she pointed out, has said he does not object to fully autonomous weapons and thinks the US should keep up in that area; he objected only to that particular version of Claude doing the work. The military has since used Claude in AI-assisted targeting in Iran, and reportedly in the raid on Maduro in Venezuela. Standing up to the Pentagon and building for the Pentagon turned out to be the same business.

For a shop owner the risk is more ordinary, and I wrote about it in Vibe Coding. If your production depends on a subscription to an everything machine, you are exposed the day the price goes up or the model quietly changes underneath you. Hao says ChatGPT’s accuracy has already slipped as the company leans toward “persuasion and addiction.” She checks every answer against a second source, and so do I. Pair the big model with a local one, and keep your customer data on your own drive.

The technology is valuable. The bubble, the Pentagon play, and the slop are choices the people who wield it make. Hao’s fix is transparency across the whole supply chain, with public say over the data.

Comments

Leave a comment