Practice · 4 min read ·

Designing a good agent flow

The difference between a flow that demos well and one that survives unattended runs: narrow LLM steps, deterministic control flow, honest testing, and knowing your worst-case cost.

Most agent flows are built forward: start with a trigger, add an LLM node, keep appending steps until the output looks right. Flows built that way tend to demo well and then degrade quietly in production, because nothing in the process forced the builder to decide what the flow is actually contractually responsible for producing. The flows that hold up are built backward.

Start from the output

Before placing a single node, write down the exact shape of what a successful run produces. Not "a summary" but a JSON object with named fields, or a drafted message with known length and tone constraints, or a score with a defined range and a threshold that means something. Two tests tell you whether the output definition is real: could a program consume it without a human interpreting it, and could you look at any given run's output and say unambiguously whether it succeeded?

This matters double if the flow will ever be published as a paid endpoint. A caller paying per run is paying for that output contract. "Interesting text" is not a contract.

Keep the LLM steps narrow

The model is the most expensive, slowest, and least predictable component in the graph, so give it the smallest job that still requires judgment. One decision per LLM node is a good default: classify this, score this, draft this. If you find yourself writing a prompt that says "first extract the fields, then decide the category, then compose a reply," that is three nodes wearing one node's trench coat. Split them, and the middle step becomes independently testable.

Everything that does not require judgment should be a deterministic node. Reshaping JSON, plucking fields, formatting lists: that is Transform work, done by a bounded expression language in microseconds for free, not by a model in seconds for tokens. Fetching data is HTTP-node work. Routing is Branch work: a branch that checks score >= 70 is auditable and free, while a prompt instruction hoping the model routes correctly is neither. A useful rule: the LLM decides values, the graph decides paths.

Design the failure paths on purpose

Decide per step what a failure should do to the run. Agent Studio's engine halts downstream execution when a node on the main path fails, which is the right default; you rarely want an output built on a failed fetch. Loops are the deliberate exception: when a flow fans out over a list, per-item failures land in a separate errors output while the other items complete. The design decision that remains yours is what to do with those errors: surface them in the output contract, or feed them to a branch that decides whether partial success is success. Silently discarding them is the one wrong answer.

Know your worst case, in dollars. A loop over N items runs the inner flow up to N times, so its worst-case cost is N times the inner cost, whether or not some items fail. The engine enforces a hard in-run spend ceiling ($5 per run by default, and never more than the agent's remaining daily budget), and iteration caps bound loops at 200 items with concurrency capped at 4, but hitting the platform's guardrail should be the backstop, not the plan. If your napkin math says a normal run costs more than a few percent of the ceiling, tighten the flow before shipping it.

Test with dry runs, and know what a dry run proves

Dry-run mode executes the real graph: real wiring, real Transform expressions, real Branch decisions, real Loop iteration. What it stubs is precisely the nodes that cost money or touch the outside world: the LLM node returns without calling the model, and the HTTP node returns a fixed placeholder without making a request. That makes a dry run a complete test of your flow's structure and logic, and no test at all of your prompt quality or the target API's actual behavior. Use dry runs to prove the plumbing, then a small number of live runs in the studio to prove the judgment. Both are cheap; confusing one for the other is not.

Feed the flow ugly inputs while you are at it: the empty list, the missing field, the 4,000-word ticket, the input in the wrong language. Every input you do not try during testing is an input production will try for you.

Ship the smallest flow that honors the contract

There is a strong temptation, once the canvas is open, to add one more branch, one more enrichment step, one more nice-to-have. Every node you add is another thing that can fail, another line in the cost ledger, and another thing to reason about at 2 a.m. when a run misbehaves. The flows that earn trust unattended are almost boring to look at: a trigger, two or three deterministic steps, one or two narrow LLM steps, an output that matches its contract every single run. Build that first. Let real runs (and, if you publish it, real callers) tell you what the second version needs.

Keep reading