Dev runs

Say “dev run”, not “test run”. The development version is a real environment — its own data store, but the same integrations, the same node code as live, and the same metering. There is no simulation mode. Running is still a required part of building (one successful run is how a dynamic node’s output type resolves), so the craft is running with mitigations, not avoiding runs — see Safety and blast radius for the mitigation playbook.

Prepare exact inputs

cai run inputs <nodeId> --json
cai run inputs <flowId> --flow --json

Inputs are keyed by opaque input ids, not display names. Build a values file keyed by the returned ids — a guessed display-name key silently falls back to saved test values, and your run proves nothing.

Smallest scope first

cai run node <nodeId> --inputs values.json --json
cai run flow <flowId> --inputs values.json --json

Start with the changed node, then run its containing flow. The distinction matters:

The isolated-test blind spot

run node is deliberately isolated — it does not execute wired upstream nodes. A node can go green while every upstream-derived value is empty. Read the result’s runScope: if it flags that end-to-end validation is required, the node run is a configuration check only, and only the flow run proves the expression. This is the single most common false pass in the product.

Read the result like an operator

For every run, inspect — in this order:

1. Status and error — but a green status is the beginning, not the end.
2. resolvedConfigs — every dynamic prop after expression evaluation. Null or empty on a green node = failure in disguise (0 and false are fine).
3. Output values — the actual data, not the checkmark. A completed run with empty output is a failure until the emptiness is explained (usually a mis-mapped input or silent null-propagation).
4. outputTypeAdoption — whether this run resolved the node’s output type (adopted / already_typed), so downstream expressions can be authored.

When you shouldn’t run it

For an action too consequential to fire even once (reject a candidate, charge a card), the honest alternative is declaring the output contract from the provider’s documented response:

cai node declare-output <nodeId> --sample-file shape.json

It is marked declared (unverified); the first real run verifies and upgrades it. Say so in your report — never present an unrun action as verified.

cai run discover (a scratch node that deletes itself after executing) exists for learning a component’s output shape — it calls the real third-party action, so the same safe-target rules apply.

The first live run is part of development

Publishing and running live once, supervised, is a legitimate step — not a process failure. For risky sends, build the safe-audience harness first: branch on isLiveVersion, false branch targeting you, true branch targeting the real audience. The supervised first live run is where the remaining problems surface.

Completion gate

Before calling a build verified: the changed path ran with credible values; the first unexpected node (if any) was identified rather than inferred from the final error; all dynamic configs resolved to intended values; branch skips are intentional; a full flow run passes after isolated fixes; and the report cites execution ids and separates development evidence from live verification.