Senator Kelly's NDAA Amendment Requires 'Human Accountability' in AI Autonomous Weapons Kill Chain

WASHINGTON — The U.S. Senate committee that oversees the Pentagon has passed an amendment introduced by Arizona Democratic Senator Mark Kelly requiring that "human accountability" be maintained when force is used by autonomous weapons systems.

The United States and other nations are increasingly deploying autonomous weapons, and the use of artificial intelligence has accelerated sharply — both in U.S. operations against Iran and in the Russia-Ukraine war.

Zaza Tsotniashvili, a professor at Caucasus International University in Tbilisi, Georgia, said via email: "Congress is responding to public and allied concerns that the U.S. could face legal and moral exposure if autonomous systems cause civilian casualties with no identifiable human decision-maker in the chain of command."

Background to the Amendment

Kelly, a former U.S. Navy fighter pilot, inserted the provision into the Senate version of the National Defense Authorization Act (NDAA). The amendment codifies DoD Directive 3000.09 of 2023 — which requires human oversight over autonomous weapons, defined as military systems capable of operating without human intervention — directly into statute.

However, President Trump signed a presidential memorandum on June 5 directing the Pentagon to "eliminate unnecessary barriers to rapid AI deployment" and calling for an updated version of Directive 3000.09 within 90 days. Should Kelly's amendment successfully complete the legislative process, it would effectively override that executive order and prevent the removal of humans from the kill chain.

The Senate Armed Services Committee passed the amendment 18–9 on Thursday and sent the bill to the full Senate for consideration.

Notably, Kelly ultimately voted against the full NDAA despite the inclusion of the AI provision and other amendments he supports. He cited the bill's $1.15 trillion price tag and a lack of transparency over Trump's decision to strike Iran without notifying Congress.

"We can't keep giving them a blank check," Kelly said in a statement last week.

The Controversy Over AI on the Battlefield

U.S. military use of AI has attracted intense scrutiny in recent months.

In January, Anthropic's AI tool Claude was used to assist in efforts to arrest Venezuelan President Nicolás Maduro.

Claude was also deployed on the first day of Operation Epic Fury against Iran. A missile strike that hit a school near a military base — killing 175 people, many of them young female students — has been speculated to have a possible connection to Claude.

In the aftermath, technology scholar Kevin Baker argued that the AI targeting system — the Maven Smart System — and the officials behind it bore far greater responsibility than Claude itself. Baker said it was the failure to update an outdated database that caused Maven to flag the school as a target; the site had been an Islamic Revolutionary Guard Corps facility until at least as far back as 2016.

"Someone failed to update the database, and another group of people built a system fast enough to make that failure lethal," Baker wrote in The Guardian.

Maven was developed by Palantir Technologies in 2017 to analyze intelligence data and evaluate strike targets. According to The New Yorker, an operator can complete the process from target identification to destruction in just four clicks, making Maven a flagship example of AI compressing the kill chain over many years.

Expert Opinion Divided

Efforts to establish legal guardrails around the lethal use of AI have earned praise from some experts.

Paul Lushenko, a non-resident fellow at nonprofit RegulatingAI and a lecturer at George Washington University, called the move "a legal, moral, and ethical imperative."

"There is a shared understanding among the military and the public that these capabilities should be subject to human oversight," he said.

Lushenko noted that tools like Maven have played an important role in national security, but that clear standards are needed around how such tools are used and who is accountable for misuse — standards that must emerge from civilian oversight, military culture, and technical safeguards.

Tsotniashvili argued that Congress cannot afford to ignore the instability of DoD policy in this area. "The executive branch can reinterpret, waive, or quietly revise DoD directives without congressional approval," he said.

While Trump's memorandum does mention maintaining "rigorous oversight" of AI deployment, it does not specify who would carry out such oversight or what it would look like. "Ambiguity serves the interests of those who want to deploy these systems," Tsotniashvili said.

Senator Ruben Gallego wrote to Defense Secretary Pete Hegseth on Monday, asking him to clarify how the updated directive would protect the United States and its military personnel from autonomous weapons. The Arizona Democrat argued that Trump's revisions to Directive 3000.09 would increase the risk of civilian casualties.

Defining 'Human Oversight' Remains Unresolved

Lushenko noted that defining "human oversight" is itself a challenge. Both "meaningful human control" and "appropriate human judgment" leave room for interpretation, and arguments that oversight mechanisms can be built into the design of autonomous weapons complicate matters further.

However, Jovana Davidovic, a professor of military ethics at the University of Iowa, believes developers bear responsibility for the behavior of any AI system they create.

"The more autonomous the system, the more the developer's decisions shape the final outcome," she said via email.

Tsotniashvili agreed that codifying Directive 3000.09 is necessary, but argued it is insufficient without more specific standards to ensure that human oversight is "substantive rather than performative."

"The FY2027 NDAA provision is a step forward," he said, "but without accompanying accountability mechanisms, it risks preserving the appearance of human control while its substance continues to erode."

---

This article was originally published in the Small Wars Journal by Arizona State University.

---

原文來源： 查看原文