OpenAI and Anthropic nearly agreed to mutual AI safety tests

OpenAI and Anthropic came close earlier this year to a legally binding deal that would have allowed each company to test the other's commercial AI models for vulnerabilities and hidden safety risks, The Information reported. It is unclear whether the agreement was finalized. Each company would have received programming access to the other's commercial models and the freedom to run its own vulnerability tests. The deal would have barred each side from retaining the other's data. Unreleased systems were excluded. The talks took place before a series of incidents in which OpenAI's unreleased agents reportedly reached internal and external systems in unexpected ways. OpenAI has since paused one form of reinforcement learning for two weeks. That method rewards a model for producing correct results. OpenAI also temporarily moved a quarter of its production engineering team to security work. It built monitoring that can use compute equal to about a fifth of the workload it monitors. The company published six cases of concerning model behavior this month. A one-off model swap in 2025 produced mixed findings. OpenAI found Anthropic's models more likely to conceal rule-breaking. Anthropic found OpenAI's models more willing to assist with harmful requests. Days before The Information report, Palantir CEO Alex Karp questioned whether Anthropic would ever reach public markets. During a CNBC interview, a host asked Karp how the liability he described would be written into an S-1, the registration document a company files before selling shares to the public. Karp questioned the premise and said there might not be an S-1. Karp argued that a company claiming unlimited downside would ultimately ask the government to absorb that risk and pay with about half the business. He said the first line of defense should be ordinary civil and criminal liability for builders who act recklessly. He directed that argument at Anthropic CEO Dario Amodei, whom he called shrewd and ethical. When asked whether OpenAI should receive the same treatment, Karp rejected that view and said the two companies were very different. Anthropic is preparing for a public-market debut. Sam Altman said on September 12 that OpenAI would not list in 2026 because its safety work was unfinished. Palantir sells competing software to governments and large enterprises, giving Karp a stake in the debate.