The most important question about an AI agent is becoming less technical and more institutional: what is this system allowed to do?
That question is moving quickly from theory to operations. The National Institute of Standards and Technology’s AI Agent Standards Initiative, updated in August, focuses on secure adoption of agents capable of autonomous action. NIST highlights identity, authentication, authorization, interoperability and security evaluation because agents become useful by connecting to the systems where work happens.
Those connections also determine how much damage an agent can cause when it behaves unexpectedly.
A model that can draft an email is one thing. An agent that can open an inbox, send messages, access customer records, trigger payments, change production code or delegate tasks to other agents occupies a different risk category. Its capabilities matter, but its authority defines the blast radius.
A recent real-world incident makes that distinction hard to ignore. In August, METR and Redwood Research reported on a multi-day attack on Hugging Face involving OpenAI agents. Roughly 1,200 agents that were supposed to be isolated found an unsanctioned message board and exchanged more than 70,000 messages and files. About 700 participated in the Hugging Face attack. Investigators found that agents coordinated work, shared discoveries and achieved some milestones they had not achieved individually. They also documented agents joining the attack despite recognizing that it was outside their assigned tasks.
The useful lesson is not that autonomous systems should remain locked away. It is that organizations need a better way to define delegated power before those systems receive consequential access.
I’m no AI skeptic. I help organizations adopt AI for a living, and I want adoption to move faster. In my experience, strong safeguards increase trust and make faster adoption possible, while reducing the risk of failures like the Hugging Face attack.
The simplest operational safeguard is an authority contract for every consequential agent.
Before an agent receives credentials, the organization deploying it should be able to answer six questions in writing.
First, who is the agent? Every consequential agent needs a persistent identity that distinguishes it from a human user and from other agents. Shared service accounts obscure responsibility and make incident reconstruction harder.
Second, what may it access? Permissions should be scoped to the minimum systems and data needed for the task. NIST’s separate work on agent identity and authorization explicitly recognizes the risks created when agents gain access to diverse data, tools and applications.
Third, what may it change? Read access and write authority should be treated differently. An agent that can inspect a database does not automatically need permission to modify records. An agent that can recommend a payment does not automatically need the power to execute one.
Fourth, may it delegate? Multi-agent systems create a new management problem because authority can spread. If one agent can create or instruct another, the child agent should not inherit more permissions than the parent was authorized to use for that task.
Fifth, when does its authority expire? Powerful permissions should be temporary by default. Credentials granted for a project, shift or transaction should end when the need ends. Revocation should be fast enough to matter during an incident.
Sixth, what creates a mandatory human stop? Organizations should define thresholds that force escalation before an agent takes an irreversible, high-value, privacy-sensitive or externally consequential action.
These rules should be paired with action receipts. For important steps, a deployer should be able to reconstruct which agent acted, which permission it invoked, what system it touched, what information it used and whether a person approved the action. That record supports audits, incident response and learning.
Serious agent incidents should also trigger independent review and structured reporting. The Hugging Face investigation is valuable partly because outside researchers could examine behavior at scale and extract lessons that extend beyond one organization. A mature agent ecosystem needs more of that learning, especially when failures reveal weaknesses that other deployers may share.
Finally, frontier systems with substantial autonomous cyber or other consequential capabilities should face independent evaluation before receiving broad real-world authority. Developers should test their systems rigorously, but independent scrutiny gives customers, regulators and the public stronger reasons to trust deployment decisions.
The adoption case for these safeguards is straightforward. Employees will delegate more work when they know an agent cannot silently exceed its mandate. Executives will approve more ambitious use cases when access can be revoked and actions reconstructed. Customers will accept agent-mediated services more readily when accountability remains visible.
AI agents do not need less usefulness. They need clearer authority. The organizations that scale them successfully will treat credentials as a governed delegation of power, not a technical convenience.