A reported AI error nearly triggered a U.S. military operation after officials discovered that intelligence supporting an armed mission against a Chinese vessel had been generated with inaccurate information from an AI chatbot.
According to CNN’s report, the operation was stopped before action was taken after officials identified that the intelligence report contained AI-generated errors.
AI-Generated Intelligence Raised Military Concerns
The false report circulated during the war with Iran and claimed the vessel was transporting components connected to a nuclear weapons program. The information originated from a Special Operations Command analyst who used an AI chatbot to combine open-source information with classified signals intelligence.
The chatbot incorrectly identified details from the ship’s cargo manifest. The analyst later used the AI tool again to format the incorrect findings into a formal-looking summary that was shared through command channels.
The incident highlights the risks of using AI systems in military decision-making, where inaccurate outputs can move through established processes before independent verification.
Military AI Adoption Requires Human Oversight
The U.S. military has been expanding its use of artificial intelligence to speed decision-making and improve operational capabilities. However, officials and researchers have warned that AI systems can produce false information, commonly known as hallucinations, without clear safeguards.
Jake Steckler, a research scholar at GovAI and a U.S. Army veteran, said service members need to understand the uncertainty of large language models, especially when decisions involve intelligence analysis, targeting or operational planning.
Steckler said the incident should encourage stronger AI safeguards rather than stopping adoption completely. He noted that AI tools can provide value when used in appropriate situations with proper controls and human review.
