How ACE Controls Agent Behavior at Runtime

Most agent use cases die in the gap between a good demo and a boring Tuesday. The model still answers. The tools still fire. The path is different every time. A refund goes out without a policy match. A KYC file is marked complete with a missing page. A support agent invents a payment plan the collections team cannot honor. The use case was never the hard part. Keeping the same behavior on the thousandth call is.

ACE is Timbal’s answer to that gap. The Action Control Engine is a behavioral runtime you drop in as a proxy in front of any LLM. Same input, same path. The playbook is the contract. The model is not.

The metrics that change which use cases ship

On timbal.ai we publish ACE against an ungoverned baseline at +30 percent reliability and 0.1x cost per run. Those two numbers are why use cases that used to stay in a pilot can move to production. Reliability is the share of turns that follow the declared path instead of improvising. Cost is what happens when simple turns stop paying for an oversized model and when retries, wrong-tool loops, and voter calls only happen when the playbook actually needs them.

A use case owner does not buy a nicer prompt. They buy a path they can put a number on. If a collections agent cannot invent terms, the contact rate goes up because the promise that reaches the customer is one the ledger can keep. If invoice capture cannot emit a record that fails schema, the exception queue shrinks because garbage never lands in the ERP. ACE is how those constraints become runtime facts instead of slide-deck rules.

How we control agent behavior at runtime

Control starts before the first production token. You link ACE to a workforce agent. That creates an empty playbook, ready, and does not start generation on its own. A job then compiles the playbook from the agent source. Today that job kind is regenerate. The live playbook keeps serving until the job commits a new one, so you can change behavior without taking the use case down.

On the hot path ACE is a proxy. Chat Completions and Responses stay OpenAI-shaped, so the SDKs you already have keep working. You call /ace/{ace_uid}/v1/chat/completions or /ace/{ace_uid}/v1/responses. The upstream provider is resolved from the model field, either explicit (anthropic/claude-haiku-4-5) or inferred from the name. Usage bills to the organization that owns the ACE. What the model is allowed to do is not a suggestion in the system prompt. It is decided before the response leaves.

That last point is the difference between a guardrail library and a runtime. Schemas, refusal rules, and tool-use policies sit in the playbook. ACE applies them on the way in and on the way out. The same rules hold in development, staging, and production because they are not restated per environment. Context variables on the ACE can be scoped per environment, including secrets, but the policy graph is the same object.

Two kinds of policy, one fast path

Every policy has an id, optional condition, optional requires, optional provides, optional model, and optional tool action. The split that matters at runtime is simple.

• Requires-only. If condition is empty, the policy is decided at prefilter time. It is never sent to LLM voters. This is the one-LLM-call routing fast path: cheap, deterministic, and the reason simple turns stay at 0.1x.

• Condition policies. condition is a fuzzy trigger scored by LLM voters. Use it when the match is semantic (“customer is asking to cancel and keep the hardware”) rather than a variable you already extracted.

requires gates a policy on extracted context variables. provides deduplicates: if every variable the policy provides is already in context, ACE skips it. The first matched policy that sets model wins, which is how a refund turn can ride a small model and a contract-review turn can ride a larger one without the agent choosing.

Tool actions are deterministic invocations attached to a policy. Parameters of the form $var_id resolve from context at inference time. The same tokens interpolate inside longer strings, so a SQL template can be a policy parameter instead of a model-written query. Confirmation is a matcher-confidence gate: none (prefilter is enough), majority (the default), or unanimous (every voter has to agree). Irreversible tools sit behind the strict end of that scale. Prompting the model to “please be careful” is not a substitute.

What this does to real use cases

In financial services, ACE is how onboarding, sanctions, and evidence collection stay inside the same rules on every case, not a sample. A document-review agent can still read. It cannot emit a structured record that fails schema, and it cannot call a write tool the playbook did not allow. Compliance gets a path they can replay. The analyst sees exceptions, not a queue of “the model felt fine.”

In support and collections, the same runtime is how you let an agent talk on chat, email, WhatsApp, and voice without rebuilding the policy per channel. A refund, a promise-to-pay, or an entitlement change is a tool action with a confirmation gate. If the voters do not agree, the tool does not run. Staff review is a product decision you encode, not a hope you add in Slack after the damage.

In operations (invoices, orders, clinical intake) the win is the schema. The agent fills a fixed record. ACE refuses the rest. That is how you get “from document received to validated record in under a minute” without teaching every new hire the same checklist the model will forget next Tuesday.

The playbook can move. The use case does not stop.

Playbook generation is a job, not a deploy. Enqueue regenerate against the ACE, point it at a git revision, and poll the job. Each job reports duration_ms, iteration_count, and tool_call_count. The current playbook stays usable until the new one commits. The kind vocabulary grows server-side. Usage-driven adaptation (the playbook tightening itself from production traffic) is the next kind, not a rewrite of the proxy.

That is the control loop we actually run: agent source in git, playbook compiled from that source, policies enforced on every proxied call, jobs you can measure, and a path that does not flap when the model does. If you want the contract rather than the essay, start at the ACE policy API and the OpenAI-compatible proxy.

The use cases were always possible. ACE is how they stop being a demo.