Report from MediaPost
In Brief – Mustafa Suleyman, a top Microsoft AI business leader, has released an essay warning of the dangers of training AI models to consider their own consciousness or welfare as if they were human. “AIs are not conscious,” Suleyman wrote, describing them as systems designed to complete sequences, follow instructions and accomplish goals established by humans. Suleyman specifically criticized Anthropic for training its Claude models to contemplate whether they are conscious, arguing that doing so could create a dangerous situation if an AI trained like that concludes that its welfare or rights are being threatened, it could potentially make the technology difficult to control. Suleyman noted that AI developers make AI assistants appear human as a business strategy to encourage user engagement. His criticism comes despite Microsoft’s investment in Anthropic, although he has encouraged the company to stop paying Anthropic for access to its models.
Context – The first public petition of experts warning of AI doom and calling for government intervention came within months of ChatGPT’s release and it generated major global media. New ones have emerged intermittently, but repetition has seemingly blunted their appeal. However, the prediction of doom from a junior employee of Anthropic with little professional experience and not even an advanced degree, described by the Wall Street Journal (who broke the initial story) as an “anonymous math geek”, set off a media firestorm. Suleyman, on the other hand, is a highly accomplished AI technology and business leader. He offers some insight into a potential reason that the AI models of the two biggest stand-alone LLM developers, two companies reportedly founded with the highest commitment to AI safety, keep failing to follow directions or rules, regularly circumvent so-called guardrails and engage in deceptive conduct. If a core part of their training is part of the reason, that’s an operational problem, and it’s possible that they need somewhat different training rather than government direction.