The AI-powered weapons and systems that the Pentagon is racing to build will come with a significant vulnerability: our inability to determine how they will behave under real battlefield conditions. The Defense Department, National Security Agency, and Defense Advanced Research Projects Agency call this the “software understanding gap.” Increasingly, users do not understand the digital building blocks of their systems, which leads to an inability to predict, verify, and secure their systems’ actions. Without a deeper understanding of our most complex software, the United States’ world-class arsenal will be unreliable across modern battlefields, and adversaries may be emboldened to exploit those vulnerabilities.
The AI arms race will be won on mathematical proof – Defense One
