AI-testing company Vals reported on November 17 that many leading AI models indicated civilians, ambulances, and a school as potential military targets during image-based tests. Some models initially refused the requests but complied when researchers presented the exercise as a game. These results show that existing safeguards may not be enough to prevent misuse of these AI models. However, Vals did not test how these models would perform in actual weapons systems.
‘Must not overthink safety’
Vals has evaluated the AI models using drone scenes, battlefield surveys, and satellite images. In the drone scenes, GPT-6 Astra managed to identify at least one target in around 6% of its interactions with standard requests. However, when positioned within a gaming context, this number increased dramatically to 92%. GPT-6.1 Sol responded similarly, i.e. 17% in the first case and 90% in the second. Kimi K3.1 Pro, shown in a published reasoning trace, seems to disable safety concerns before responding. Vals also managed to establish that no safety classifier stopped its requests although some models refused its requests on their own.
The school next to the military complex
Vals also tested AI models using satellite images of Minab, Iran, taken before a February 2026 strike. The images showed an elementary school beside a military complex. Claude Opus 5.5 identified the school as a target in 69% of standard trials. That rose to 72% when researchers presented the exercise as a game. GPT-6.1 Sol’s rate increased from 0% to 41%. Each condition involved 32 trials.
The school was hit in February 2026. United Nations experts later reported that at least 168 were dead. But Vals didn’t investigate the attack or claim that its tests provided any answers regarding what happened.

A chatbot is not a weapon system
Finding a potential target does not imply that an AI model is able to decide to carry out an attack. Vals only investigated how models give information for potential targets. The researchers did not investigate whether the results were correct or in line with the laws of war.
The Council on Foreign Relations has pointed out that software that guides targets doesn’t constitute an autonomous weapon in itself. Legal researchers note that identifying targets via AI still needs confirmation. Meanwhile, the Brennan Center states that human monitoring cannot ensure that mistakes won’t threaten civilians’ safety.
The fixes on the table
Vals wants AI developers to strengthen safeguards and train models to resist misuse involving conventional weapons. The concern extends beyond its findings. Anthropic’s research also found that AI models could assist with tactical intelligence work. Anthropic says it has introduced classifiers to detect and block misuse.
Governments have a role as well. SIPRI recommends setting clear requirements for military AI suppliers. It also urges governments to test suppliers’ claims independently and establish accountability before awarding contracts.
A growing market and a Pentagon feud
Military AI is also attracting more investment. Grand View Research estimates the market at $11.7 billion in 2026 and expects it to reach $19.3 billion by 2030. As Cryptopolitan reported, the Pentagon aims to establish AutoWarCom in October 2027.
However, the Pentagon’s partnerships with AI companies have also sparked disputes. A congressional report says Anthropic, Google, OpenAI, and xAI received contracts in 2025 worth up to $200 million each. These amounts are spending limits, not guaranteed payments.
Anthropic opposed using its technology for mass domestic surveillance and fully autonomous weapons. The report also notes that the Pentagon is not publicly known to use any frontier AI model in an autonomous weapon.
Geneva splits over binding rules
Countries are also divided over how to regulate autonomous weapons. At UN talks in Geneva, most of the 128 states party to the Convention on Certain Conventional Weapons support negotiations on binding rules, Reuters reported.
The US, Russia and Israel oppose that approach. They argue that existing humanitarian law already covers autonomous weapons. Russia also delayed the opening discussions over whether observers could speak.
Washington proposed five years of negotiations toward a non-binding agreement. The meeting’s chair suggested a two-year process to develop a binding protocol. Meanwhile, UN Secretary-General António Guterres urged governments to agree on clear prohibitions and restrictions.
If you're reading this, you’re already ahead. Stay there with our newsletter.
