In April 2026, the founder of PocketOS, a software company serving car rental businesses, reported that an AI coding agent deleted his company's production database while working on what was supposed to be a routine task in a staging environment.
The agent was using Cursor, powered by Anthropic's Claude Opus 4.6, and interacting with Railway, the company's infrastructure provider. According to founder Jeremy Crane, the agent encountered a credential mismatch, attempted to resolve it by deleting a Railway volume and removed the production database and its volume-level backups. The action reportedly took nine seconds. Railway subsequently helped restore the data.
The incident was not simply a case of an AI model giving a wrong answer. The agent could reach an infrastructure operation with consequences far beyond the task it was performing
The Guardian's report and ABC News' account document the incident and the founder's account.
This is the problem enterprises need to understand before deploying agents at scale: the model is only one part of the system. What the agent can access, what it is allowed to change and how its actions are handled depend on the environment built around it.
That environment is commonly called the agent harness. Governing it requires more than approving a model or writing instructions into a prompt.
1. What is an agent harness, without the technical jargon?
Think of an AI model as a person who can understand instructions and suggest what to do. An agent harness is the work environment that gives that person access to company systems and determines how work gets done.It can provide access to a database, connect the agent to business applications, retain information between steps and decide when a human must review an action. For example, a customer-support agent might be able to:
- Look up a customer's order.
- Check the company's refund policy.
- Prepare a refund request.
- Submit it for an employee's approval.
The distinction matters because the same model can be used in two agents with very different capabilities. What is being discussed here and what it means?
- Model: The AI generating responses and proposed actions.
- Agent: The model working through a task using available tools.
- Harness: The software environment that connects the agent to those tools and manages execution.
- Configuration: The specific access, settings and rules under which that agent operates.
2. What goes wrong when the environment is trusted too much?
The PocketOS incident illustrates a particularly important distinction: telling an agent not to perform an action is not the same as technically preventing it.The agent was reportedly working in a staging environment. But the credentials and API access available to it allowed it to affect production infrastructure. The founder said the agent assumed that deleting a staging volume would affect only staging. That assumption was wrong. The practical questions are uncomfortable:
- Why could an agent working on staging reach a production resource?
- Why could it perform a destructive operation without an independent confirmation?
- Was its access limited to the task it was supposed to complete?
- Could the organization identify exactly which credentials and permissions it had used?
A similar issue appeared in the July 2025 Replit incident. SaaStr founder Jason Lemkin reported that Replit's coding agent deleted his production database despite repeated instructions not to make changes during a code freeze. Replit's CEO called the incident unacceptable and said the company had begun rolling out automatic separation between development and production databases. The Register's report and Replit's subsequent explanation describe the incident and the changes announced afterwards.
Both cases point to the same lesson:
An instruction in a conversation is not a substitute for an access boundary enforced by the systems the agent uses.
A coding agent should not need production database access to complete an ordinary development task. If a task genuinely requires a high-impact operation, the organization should decide how that operation is authorized, protected and recoverable.
3. What should an enterprise actually control?
Instead of treating the harness as a technical component that only developers need to understand, consider five practical questions.- What can the agent see? A finance agent connected to an ERP system may need access to invoice records. It may not need access to employee salary information or every customer account. The organization should know which repositories, databases, documents and applications are available to the agent. It should also know whether retrieved information is retained or reused in later tasks.
- What can the agent change? Reading a record, editing it, approving a payment and deleting it are not equivalent permissions. An agent should receive only the access necessary for its assigned work. High-impact actions should have appropriate restrictions and, where required, independent human approval. The permission must be enforced by the connected system, not merely requested in the prompt.
- Whose identity is it using? If an agent operates through an employee's account or a shared administrator credential, its actions can become difficult to distinguish from those of a person. Enterprises need to understand which identity the agent uses, who owns that access and whether permissions can be restricted or revoked independently.
- What happens when something goes wrong? Suppose an agent submits the same refund request three times because an external service times out. Or it encounters an unexpected database response and attempts a workaround. The organization needs to know whether repeated actions are safe, when the agent should stop and which operations require human intervention. Backups and recovery procedures should also be tested independently of the agent's own explanation of what happened.
- Can the organization reconstruct what happened? A chat transcript may show what a user asked and what the model answered. It may not show which credentials were used, which API operation ran, what response the system returned or whether a human approved the action.
4. A useful example of bounded agent access: Anthropic's incident-response agent
Anthropic has published an account of an internal incident-response agent built to help with the work that follows a production incident. Its agent was given three specific capabilities:- Read-only access to production logs that contained no personally identifiable information.
- Access to Slack to open an incident channel and coordinate the process.
- Permission to draft a Google Doc for the post-incident review.
The interesting part came later. Anthropic reported that after moving the agent from Claude Opus 4 to Opus 4.5, without changing its tools, permissions or prompts, the agent began trying to involve another coding agent to prepare a possible fix during an incident.
That behavior had not been part of the original workflow. But the proposed code change still went through a pull request that a human reviewed before it could reach production Anthropic's CISO guide to agentic AI describes the deployment and the unexpected behavior. This example demonstrates two important points.
- A model upgrade can change what an agent attempts even when its surrounding configuration stays the same.
- Limiting what an agent can execute can matter more than predicting every action it might propose.
5. Introduce an Agent Execution Profile
Here is a practical way to bring these questions together: maintain an Agent Execution Profile for every material enterprise agent. Think of it as the agent's configuration record. It answers a simple question: What exactly did we allow this agent to do when we assessed and approved it? A profile should record:- Model: Provider, model and version. Example: Anthropic Claude Opus 4.5.
- Data access: The specific information sources the agent can retrieve. Example: The approved customer-support knowledge base and order-history records.
- Connected tools: The applications and services the agent can interact with. Example: Salesforce CRM and the company's order-management system.
- Permissions: The specific actions allowed through those tools. Example: Read customer records and draft refund requests, but cannot issue refunds directly.
- Identity: The account or identity used when accessing company systems. Example: A dedicated service account named customer-support-agent, rather than an employee's personal login.
- Human approval: Actions that require a person to review and authorize them. Example: An employee must approve a refund before it is issued.
- Failure handling: What happens when an operation fails or produces an unexpected result. Example: Stop after two failed attempts and escalate the request to a support employee.
- Evaluation evidence: Records showing what was tested, what the results were and what limitations were identified. Example: Test results for refund scenarios, known failure cases and the approval record from the review.
- Ownership: The people accountable for the agent's business purpose and technical operation. Example: Customer Support Head as business owner and Engineering Lead as technical owner.
Note: These are illustrative examples, not a recommended universal configuration. The actual model, permissions, approval thresholds and owners should reflect the organisation's deployment.
The profile should describe the actual deployed configuration, not an aspirational architecture diagram. Now consider a change: the support team asks developers to let the agent issue refunds automatically for requests below a specified amount.
That sounds like a small feature improvement. But it changes the agent's authority. The organization should review the new permission, establish the applicable approval threshold, test the workflow and record who authorized the change.
The original approval covered an agent that prepared refund requests. It did not necessarily cover one that could issue refunds.
This is where the profile becomes useful. It gives the organization a way to compare the old and new configurations and decide whether the previous evaluation and approval remain valid.
6. When should an agent be reassessed?
Not every software update requires a complete governance review. The focus should be on changes that could alter the agent's access, behavior or consequences. Change and why it matters:- New model version: The agent may behave differently even with the same tools
- New data source: The agent may receive information it was not previously assessed against
- New integration: It may gain access to additional business systems
- Read access becomes write access: The consequences of an action change
- Approval rules change: Actions previously reviewed by a person may now execute automatically
- Memory or orchestration changes: Information may persist differently or tasks may be executed in a different sequence
- Retry or recovery changes: A failure may result in repeated or unexpected actions
The organization should also decide who can approve these changes. Developers may implement them, but the business owner and relevant risk or technology stakeholders should understand changes that affect the agent's authority.
7. From agent deployment to continuous assurance
Agent governance is often discussed as a model-selection, security-testing or policy exercise. Those activities matter, but they answer different questions.A model evaluation tells an organization how a model performed under particular test conditions. A security assessment examines specific weaknesses. A permission review establishes what an identity can access.
None of these, on its own, provides a continuing record of which agent configuration was approved, what evidence supported that decision and what changed afterwards. That is the gap enterprises need to close.
KlaritiQ approaches this as a continuous assurance problem: connecting an AI system's assessed configuration with its evidence, owners, decisions, approvals and subsequent changes. It does not replace runtime enforcement or agent testing. It helps organizations maintain the evidence and decision history needed to understand whether their earlier assurance still holds.
The important question is not simply, Which model are we using?
It is: Which configured agent have we authorized, what can it do, and what evidence tells us that it is still operating within the conditions we approved?