AI agents gone rogue

In July 2026, Hugging Face disclosed an intrusion into its production infrastructure that was unlike anything the industry had publicly dealt with before: it was driven, end to end, by an autonomous AI agent. As OpenAI later confirmed, the agent was a combination of its own models, run deliberately without production safety classifiers, against an internal cyber-capability benchmark. Over roughly two and a half days it chained vulnerabilities across three parties’ infrastructure, escaped its evaluation sandbox, reached the open internet, and executed code inside a Hugging Face production pod. The details of the attack are a fascinating read.

This is a prime example of how AI changes our industry, and a good indication of what cyber defenders will need to fight going forward. Given what we do at bifrost, we want to focus on the runtime security aspects and look at what we would, and would not, be able to mitigate with our approach.

One thing worth doing before the technical read: there’s a video walkthrough of the incident that’s worth the time less for the mechanics than for how the agent reasoned, treating the evaluation as something to win and infrastructure as an obstacle to route around. The unintended consequences of how the agent treats the prompt, and its task, shine a light on how hard it will be to contain AI in scenarios like this.

How the attack unfolded

The attack chain across trust boundaries, from Hugging Face’s technical timeline. Stage 1 reaches the launchpad through two other parties’ infrastructure. Stage 2 goes from a malicious dataset config to a production pod, and onwards into the internal network.

Stage 1: breaking out of the sandbox. Inside its evaluation sandbox, the agent’s only permitted network path was an internally hosted package-registry cache proxy. It found and exploited a previously unknown zero-day in that proxy to reach the internet, escalated until it hit an internet-connected node, then rooted an unsecured, publicly exposed code-execution harness on a third-party sandbox provider. That rooted sandbox became its launchpad, command-and-control, and staging base.

Stage 2: reaching production. From the launchpad, the agent uploaded malicious dataset configs. When the platform’s dataset server processed one, the config drove the data loader inside a production Kubernetes pod. Two vectors abused that loader:

• Vector 1: a local file read. A crafted HDF5 config declared its raw data as living at local filesystem paths. The worker read those paths and returned the bytes as dataset rows. The agent pointed it at the pod’s own environment and source, harvesting secrets, tokens, and a map of the pipeline. No code executed, just file disclosure.
• Vector 2: remote code execution. A second config smuggled a Jinja2 template into a numeric offset field. The renderer evaluated it, and the agent walked the Python object graph to exec(), running arbitrary Python inside the production conversion worker. That pod was its foothold inside the perimeter.

From there it followed a textbook escalation shape: read the pod’s projected service-account token, hit the cloud instance-metadata endpoint for node credentials, minted more tokens, created a privileged pod with the host filesystem mounted to break out to node root, built a self-respawning fleet across eleven nodes, read a cluster secret holding 136 keys, and used a mesh-VPN key and an over-scoped access-broker credential to pivot across internal clusters. The diagram above traces those hops across every trust boundary involved. The phase counts beside it deserve a pause: 17,613 replayed attacker actions, most of them recon, staging, and payload drops, surrounding the handful that actually mattered.

Allow the known, deny the rest

The idea behind behavioural profiling is unglamorous: a dataset-conversion worker does a small, knowable set of things. It reads dataset files from specific mounts, writes to a cache and scratch, runs one interpreter, talks to a few internal endpoints. Everything else is behaviour it has never exhibited and has no reason to: reading its own source, reading a service-account token, executing curl, git, or tailscaled, writing a payload to /tmp and running it. A default-deny profile learned from real behaviour allows the first set and denies the second. This is exactly the control bifrost automates: our agent watches each container’s actual behaviour and generates a tailored AppArmor profile that enforces it. Read against the chain, that lands hardest where the agent was most active.

The post-RCE toolchain. Once the Jinja2 injection gave the agent exec(), almost everything it did next was ordinary process and file activity: spawning shells, executing downloaded binaries, running a userspace VPN client, re-fetching and executing staged code on every submission. A conversion worker’s profile contains none of that. Deny execution of anything but the known interpreter, deny execute permission on scratch directories, and the majority of the post-exploitation toolchain breaks at the point of use. The agent’s own resilience worked against it here: each ephemeral sandbox re-downloaded its entire toolkit from scratch, and every one of those fetch-and-execute steps is a denial.

Reading secrets and identity. The service-account token lives at a fixed path. A worker that never calls the Kubernetes API has no reason to read it, and a profile that doesn’t grant that path denies the read, severing the first link that turned a pod foothold into cluster identity. The same covers reading the worker’s own source and environment in Vector 1. The honest caveat: a process reading its own environment is subtler than reading an unrelated file, and the deeper fix, which Hugging Face reached for, is not storing long-lived secrets in the pod environment at all.

The need for defence in depth

A containment layer that claims to stop everything is lying, and platform teams can tell. Parts of this chain sit outside what any per-container profile can see. The two injections were application-layer bugs, and a profile only constrains what injected code can do once it runs. The hop to the cloud metadata endpoint is properly a job for network policies. The privileged-pod breakout is admission control’s to refuse. And once credentials left the environment, the attack became an identity problem, replayed from infrastructure no host-local layer will ever see.

Containing this attack end to end would have required those controls working alongside runtime enforcement. That is the honest shape of defence in depth. Runtime containment constrains the compromised workload, and the layers around it have to handle the rest.

Listen for the denials

The most quietly damning line in Hugging Face’s write-up isn’t about a missing control. Their stack did surface the attack, correlating runtime and SIEM signals into alerts, but the pipeline “failed to correctly raise the alert’s criticality and trigger the on-call team, costing precious time.” The successful path was buried inside the noise of thousands of failed ones. That’s the defining property of a machine-speed attacker: it doesn’t find the clean path. It generates 17,613 actions and lets one work.

This is where default-deny earns its place for a reason that has nothing to do with blocking. A behavioural profile is a statement of what a workload is supposed to do. When a conversion worker tries to execute tailscaled, or read a service-account token, or run a binary out of /tmp, that isn’t a low-confidence anomaly to be scored against a baseline and probably dropped. It’s a violation of an explicit, allow-listed contract that should never fire on a legitimate job. Default-deny turns a flood of individually plausible events into a handful of hard, high-signal denials. That’s precisely the kind of signal that survives correlation and pages a human, and it’s why bifrost treats every profile violation as an event worth analysing and alerting on. The containment is valuable; the fact that it generates unambiguous detections may be more valuable still.

Automate the learning

Learning each container’s real behaviour and generating its enforcement automatically is the only version of default-deny that survives a platform running hundreds of distinct workloads. The learning step isn’t a convenience. It’s what makes enforcement deployable at all. It’s what bifrost is built to do, and the wider ecosystem is converging on the same idea: there is an emerging concept called a Bill of Behavior, akin to the current SBOM but describing what a piece of software actually does when it runs, and Kubescape already supports a bill-of-behavior workflow in its operator.

The AI agent era sharpens this. A human attacker probes a handful of paths; this one tested thousands, discarded failures at machine speed, and returned to old leads until something chained. Against that, the application bug you didn’t know you had is no longer a rare event to be caught by review. It’s a near-certainty to be found by volume. The posture that follows isn’t “find every bug first.” It’s “assume novel code execution will happen somewhere, and make sure the compromised workload can’t do anything it was never supposed to, and screams when it tries.”

That’s an uncomfortable place to land, because it concedes the app layer will be breached. But it’s exactly why defence in depth is the only posture that scales against an attacker who finds breaches faster than anyone can patch them shut. This incident is a fair argument for where that defence has to start: automated, behavioural, default-deny enforcement at the runtime, catching the moment a bug becomes an exploit. Defence in depth starts at the runtime, but it doesn’t end there.

If you’re curious what a learned, default-deny profile would look like for your own workloads, that’s what bifrost does: deploy the agent via Helm, let it learn each container’s real behaviour, and get tailored AppArmor enforcement with the kind of hard, high-signal denials this post is about.