⟶ Insights / AI Engineering
Custom AI agents that survive production.
How senior teams design custom LLM pipelines, tool-using agents and RAG systems that hold up under real load, real compliance and a real invoice — not just a demo reel.
11 min read · Updated Jul 2026
Design premise
The failure mode of custom agents is not intelligence — it is unbounded action. Every production agent we ship is scoped by tool allow-lists, step budgets and typed output contracts before a prompt is written.
The winning architecture is boring on purpose: retrieval that is measured, tools that are versioned, prompts that are diffed, evals that gate every deploy. Cleverness lives in the workflow, not in the model call.
Frequently asked.
What is a 'custom AI agent' in production terms?+
An LLM-driven workflow that can read context, call tools (databases, APIs, internal services), plan multi-step actions, and produce structured output that a downstream system can consume. In production it also ships with evaluation, guardrails, a reviewer UI and a rollback path. Anything without those four is a prototype.
RAG, fine-tuning, or agents — how do we choose?+
RAG when the answer lives in your corpus and freshness matters. Fine-tuning when tone, format or a narrow judgement task is repeatable and evaluable. Agents when the workflow spans multiple systems, needs planning, and a single prompt cannot do it. Most real systems blend all three: RAG for retrieval, fine-tuned smaller models for classification steps, and an agent loop for orchestration.
How do we stop agents from hallucinating or going off-task?+
Five layers. (1) Strict output schemas — validate every step against a typed contract. (2) Tool allow-lists — the agent can only call the tools it has been granted. (3) Step budgets and cost caps — hard limits per run. (4) Eval harness with regression tests on every prompt or model change. (5) A human-review UI on high-stakes actions until confidence is measured, not assumed.
What about prompt injection and data exfiltration?+
We treat any retrieved or user-supplied text as untrusted. Defences include content isolation (retrieved data is never mixed into the system prompt), egress allow-lists on tools, PII redaction on both inbound and outbound payloads, and dedicated red-team eval sets that live in CI. Prompt injection is a design-time problem, not a runtime patch.
How do we control the token bill?+
A model gateway sits between the app and the providers, so we can route by task class — frontier model for reasoning, smaller open-weight for extraction, cache for repeats. We instrument cost per request, per user and per workflow from day one, and set alerts on unit-economics regressions before they hit the invoice.
How long until a custom agent is in production?+
A scoped single-domain agent (one workflow, one team, real users) ships in 10-14 weeks with a supervised rollout. Multi-agent systems with reviewer UI, escalation flows and audit trails typically take 16-24 weeks. Discovery workshop first, always — we do not quote agent work without a two-week scoping engagement.
Scoping a custom agent?
30 minutes with a senior architect. We will map the workflow, the guardrails and the first eval set before you sign anything.