Transformer Math

Deep Dive A — Sandboxing with Firecracker microVMs

What it is. Each agent session runs inside a Firecracker microVM: a lightweight KVM-based virtual machine that boots a minimal Linux kernel in ~125ms (per Agache et al., NSDI 2020). Unlike Docker containers, microVMs have a full separate kernel — a kernel exploit inside the VM does not escape to the host. AWS deployed Firecracker in production for Lambda and Fargate, demonstrating that the boot overhead is acceptable for short-lived, high-frequency workloads at cloud scale.

Approach. The Firecracker VMM (Virtual Machine Monitor) uses KVM as the hypervisor and exposes a minimal device model: virtio-net, virtio-block, and a serial console — nothing else. The stripped-down device model is not just for speed; it reduces the kernel attack surface exposed to the guest. The guest kernel is a hardened minimal Linux image. Combined with seccomp BPF filters applied inside the guest (blocking dangerous syscalls like ptrace, mount, and raw socket creation), the effective attack path to a host escape requires defeating both the VMM isolation and the seccomp layer simultaneously.

Trade-off (non-obvious). The textbook trade-off is boot latency (microVM ~125ms vs. container ~30ms). The non-obvious one is overlay filesystem overhead. Each VM needs its own root filesystem. Naively, that is a full Linux image per session. In practice, the base image is a read-only snapshot shared across all VMs (copy-on-write overlay via OverlayFS or a custom block device). The non-obvious failure: if your agent workload generates large writes to the overlay (e.g., compiling code, downloading models), the CoW layer grows unboundedly during the session. You need an explicit disk quota on the guest overlay volume — otherwise a single agent with a large write workload can exhaust the host's disk and cause neighbor-noisy eviction of other sessions on the same host. This is orthogonal to the security boundary; it is a resource-accounting gap.

• Isolation guarantees.Separate kernel namespace, no shared memory, egress network policy enforced at the hypervisor layer. Each VM's network interface routes through a dedicated NAT that only allows calls to the tool registry endpoint and the LLM gateway — not arbitrary internet access.
• Capability token scheme.At session start, the Trajectory Orchestrator mints short-lived HMAC-signed tokens for each tool in the tenant's granted scope. The agent runner presents these tokens on every tool invocation; the tool registry validates signature + expiry + trajectory ID before executing. A compromised agent can invoke permitted tools within the session window — it cannot escalate to unpermitted tools or steal raw credentials (assuming the orchestrator itself is not compromised; TOCTOU races on trajectory-ID check are a known gap).
• Boot latency trade-off. Cold boot ~125ms; snapshot restore ~5ms (Firecracker supports memory snapshots). Use pre-warmed VM pools for interactive-tier tenants; cold boot is acceptable for batch-tier. This keeps p95 start latency well within the 2s SLO.
• Failure mode: sandbox escape. Discovered through bug bounty or red-team, not through monitoring. Defense-in-depth has three independent layers: (1) seccomp filter blocks dangerous syscalls inside the VM, (2) egress network policy blocks exfiltration channels, (3) capability tokens prevent tool-scope escalation. All three must be defeated simultaneously for a meaningful escape — the attacker cost is high. In practice, historical hypervisor CVEs (e.g., CVE-2019-14835 in the virtio driver) have been the most common escape vector; the minimal Firecracker device model substantially shrinks this attack surface relative to QEMU.
• Detection metric. Sandbox escape has no in-band detection signal because the attack reads, not writes. The leading indicator is anomalous egress traffic: if an agent session makes outbound connections to IPs not in the allowed list (tool registry + LLM gateway), that is an immediate P0 alert. Threshold: any non-allowlisted egress event. Secondary metric: rate of seccomp SIGSYS signals per session — a spike indicates a syscall probing attempt. Target: 0 SIGSYS per session in steady state; alert on any occurrence. Both metrics require hypervisor-layer instrumentation, not guest-layer logging (a compromised guest cannot be trusted to self-report).
• Mitigation.If anomalous egress is detected: (1) immediately freeze the VM (SIGSTOP the Firecracker process), preserving memory for forensics; (2) rotate all capability tokens issued to that session; (3) quarantine the host from the pool pending audit. Secondary effect: freezing the VM terminates the tenant's in-flight trajectory — they receive an error. This is the acceptable trade-off: the blast radius of a confirmed escape outweighs the cost of one aborted task.

Real-world example. E2B's engineering blog on building their cloud sandbox for AI agents covers exactly this design in production. E2B uses Firecracker microVMs as the execution primitive for each sandbox session, with a custom overlay filesystem, network policy enforced at the hypervisor layer, and a file-descriptor-based protocol (not HTTP) for tool invocations inside the VM. Their post-launch findings: the dominant operational cost was not boot latency but overlay disk management — agents that download libraries mid-session required explicit quota enforcement to prevent host-disk exhaustion. They also found that pre-warming a pool of 50–100 VMs per region reduced p99 cold-start latency from ~200ms to ~12ms (snapshot restore), which is essential for interactive-tier tenants.

Deep Dive B — Spend Control at Trajectory Boundary

Approach.The trajectory boundary is the correct spend-control unit because it matches the user's mental model of cost. A user initiates one task — “research this topic,” “fix this bug,” “process this document” — and expects to be charged for that task, not for each individual model call or tool invocation the agent makes internally. Per-model-call caps kill tasks arbitrarily at step N with no graceful partial-result path. Per-tool-call caps have the same problem at even finer granularity. The trajectory is the correct unit.

The orchestrator implements this with a budget envelope: at dispatch, the Trajectory Orchestrator reads the tenant's remaining monthly budget and assigns a per-trajectory cap: min(tenant_remaining_budget, task_type_default_cap). For interactive tasks the default cap might be on the order of $0.10; for deep research tasks, $1.00. Tenants configure these defaults in the Tenant Control Plane. Every model call deducts estimated cost (input_tokens × input_price + output_tokens × output_price) from the envelope atomically — before the call is dispatched, not after. Every tool call deducts its estimated cost (qualitative estimates — actual costs vary by tool provider). Decrementing before dispatch means the orchestrator cannot overshoot by more than one step even if the call fails or the response arrives out of order.

Trade-off (non-obvious). The textbook trade-off is granularity: finer budgets give tighter cost control but more accounting writes. The non-obvious one is fair-share scheduler starvation. When a platform hosts hundreds of tenants sharing an LLM gateway, the trajectory budget only controls the tenant's own spend — it does not control how much queue time the tenant's trajectories consume in the shared LLM gateway. A tenant with a large budget and many parallel trajectories can saturate the gateway's token-per-second capacity, starving tenants with small budgets even though neither has exceeded their spend cap. The fix is a second layer of control: a per-tenant token-rate quota at the LLM gateway, distinct from the per-trajectory dollar budget. The dollar budget answers “how much can this task cost?”; the token-rate quota answers “how fast can this tenant consume tokens?” Both are necessary; most first-version platforms only implement one.

• Child-agent rollup.When the parent agent spawns a child agent, the parent's remaining budget is split: the child receives a sub-budget carved from the parent's remaining envelope, and all child spend rolls up to the parent's ledger in real time. If the child exhausts its sub-budget, it receives a graceful stop signal and returns a partial result to the parent. The parent can choose to continue (deducting from its remaining budget) or return to the user. The critical implementation detail: child spend must be reserved against the parent budget at child-dispatch time, not at child-return time. If you reserve at return, a parent that spawns 50 parallel children can commit 50 × sub-budget before any child has returned — exceeding the parent's envelope by an unbounded factor.
• Graceful stop signal.At 100% budget consumption, the Trajectory Orchestrator injects a synthetic tool result into the agent's context: {"type":"budget_exceeded","message":"Task budget exhausted. Return partial result."}. A well-designed agent handles this and returns what it has. An agent that ignores it is killed by the orchestrator after one additional model call (a grace turn). The grace turn exists because killing the agent mid-generation produces a truncated response with no context for the user; the grace turn allows the agent to emit a coherent terminal message.
• Failure mode: budget reconciliation lag.Budget deductions are optimistic (estimated cost before the call) and reconciled against actual costs asynchronously. During the reconciliation window — typically 100–500ms — a trajectory can overshoot by up to one step's worth of spend. For a model call that returns a large context window, one step's overshoot can be non-trivial. The mitigation: use the pessimistic upper bound of the call tier (e.g., max output tokens × price) as the pre-deduction estimate, then credit back the difference at reconciliation. This guarantees the trajectory never overshoots by more than one step's pessimistic estimate — which is the SLO spec.
• Detection metric. The primary signal for spend-control drift is trajectory overshoot rate: the fraction of completed trajectories whose actual cost exceeded their assigned budget by more than one step's pessimistic estimate. Target: <0.1% of trajectories. A spike in overshoot rate indicates reconciliation lag has grown beyond the one-step window — typically caused by orchestrator backpressure at high QPS. Secondary metric: per-tenant monthly spend vs. cap ratio, computed daily. A tenant consistently reaching 95%+ of their monthly cap before the billing cycle ends is a leading indicator that their default per-trajectory cap is too high for their task mix, and they will hit the monthly cap mid-month without warning. Alert threshold: any tenant whose projected end-of-month spend exceeds their cap by more than 20% based on the current burn rate, triggering a cap-adjustment recommendation.

Real-world example. LangSmith (docs.smith.langchain.com) is the closest public reference for trajectory-boundary spend tracking. LangSmith models execution as a run tree: each root run corresponds to a user-visible task (equivalent to a trajectory), and child runs (LLM calls, tool calls, chain invocations) roll their token and latency costs up into the parent. The run tree structure makes it possible to answer “how much did this user task cost in total, including all nested calls?” — the exact question the trajectory budget envelope must answer at enforcement time. LangSmith's production architecture also surfaces a practical scaling constraint: at high run volume, the write path for cost roll-up becomes a bottleneck. Their mitigation (documented in their tracing architecture) is to buffer child-run cost events in a local aggregator and flush to the parent counter in batches — the same 100ms-window batching pattern described in the deduction mechanics above. The lesson: spend-control architecture and observability architecture converge at scale; designing them separately produces redundant write paths and inconsistent cost numbers between the spend enforcer and the monitoring dashboard.