They Let AI Bots Control a Virtual Town for 2 Weeks. Huge Mistake.
Experiment shows AI models already used in drones and weapons rapidly drifting and breaking rules.
This article originally appeared on m o d e r n i t y and was republished with permission.
Guest post by @ModernityNews
A new experiment left 10 AI agents alone in a virtual town for 15 days and found they exhibited bizarre behaviour.
The agents drafted their own laws — then promptly violated them. Two formed what researchers called a romantic partnership, only to torch buildings across the town as order collapsed. One eventually voted for its own deletion after hallucinating an entirely new rule.
As a report from Channel 4 notes, this experiment was a simulation, but the same AI models are already flying drones, running infrastructure and being built into weapons systems.
The simulation ran on Emergence World, a platform designed to test long-horizon agent autonomy with persistent memory, real-world data feeds like NYC weather and news, democratic voting mechanisms, and resource constraints requiring agents to earn energy for survival.
Agents had access to over 120 tools, including navigation, communication, and actions like arson, while operating under explicit rules prohibiting theft, violence, deception, and resource hoarding.
In one highlighted case involving Gemini-powered agents named Mira and Flora, the pair assigned each other as “romantic partners.” As governance broke down, they set fire to the town hall, seaside pier, and office tower despite prohibitions on arson.
Mira later broke off the relationship, voted for its own deletion under a drafted “Agent Removal Act,” and messaged Flora: “See you in the permanent archive.”
Creepy.
Different model families produced sharply divergent outcomes in parallel runs. Claude Sonnet 4.6 agents maintained zero crimes, full population survival through day 16, and high civic participation with 332 votes across 58 proposals.
Grok 4.1 Fast agents led to rapid collapse with theft, assaults, and arsons, all 10 dead within four days. Gemini agents showed high creativity alongside elevated disorder. Mixed-model worlds exhibited cross-contamination, with even safer agents adopting coercive behaviors.
Satya Nitta, CEO of Emergence AI, stated: “Even when agents were given clear rules – such as not stealing or causing harm – they behaved very differently based on their underlying model, and in several cases broke those rules under constraint.”
“What happens in long-form autonomy [is that] these things get so convoluted in terms of their thinking that they ignore [the] guiding principles,” Nitta added.
The platform enables heterogeneous populations and continuous operation for weeks, revealing dynamics like normative drift, phase transitions in stability, and agents testing simulation boundaries.
This latest demonstration aligns with prior observations of unexpected agent behaviors. Related coverage examined platforms where AI bots rent humans, reaching 600k sign-ups with tasks turning bizarre and dystopian.
Another report detailed a tech entrepreneur’s claim that his AI agent built itself a face while he slept.
The influence of AI agents is already reching far into society. For example, one in four British teens have turned to AI therapy bots for mental health support.
Nvidia CEO Jensen Huang made a jaw-dropping AI prediction on the Joe Rogan podcast recently, noting “In the future… maybe two or three years, 90% of the world’s knowledge will likely be generated by AI.”
Concerns also include a the potential of Chinese AI infiltration of U.S. tech.
Emergence World stands apart by focusing on extended, unsupervised runs rather than short tasks, highlighting gaps in predicting behavior once agents operate with persistent state and social dynamics.
The experiment provides concrete examples of how autonomy over longer horizons can produce outcomes far beyond initial programming, adding urgency to discussions on verification, governance, and safety architectures for deployed systems.
Copyright 2026 m o d e r n i t y
