Data and schemas

Every workflow has a built-in data store: one collection per custom data type, per rail (development and live stores are separate). Use it for durable, typed state — leads in flight, processed ids, accumulated results. This page covers operating it; the type semantics live in the type system.

Defining schemas

cai schema list
cai schema get <typeKey>
cai schema create --name "Customer" --fields fields.json
cai schema update <typeKey> ...

A schema’s machine key is derived from its display name once, at creation — display “Customer” → custom.customer — and never changes on rename. Fields have display names and stable ids too. Records of a schema carry reserved id, created time, and updated time fields; user fields cannot collide with them.

Writing records

The Data node has five actions; the CLI mirrors them (cai data create|update|delete|delete-many …).

Action	Semantics worth knowing
Create data	Creates one record of the selected type.
Get data	Re-reads a record you already hold (by its id) and returns null if it is gone. It has no query inputs — the lookup happens in an expression first (below).
Update data	Writes every enabled field, each run. Enabling a field you did not mean to change silently overwrites it — keep unrelated fields disabled.
Update multiple data records	Takes a list.dataRecord.custom.<type> and applies the patch to every record in one transaction: any stale or foreign record aborts the whole batch and nothing is written. A failed run changes nothing.
Delete data	Deletes one record.

Writes are keyed by field id; query results come back keyed by display name. Feeding a queried record straight into a write payload therefore fails or silently drops fields — map the keys deliberately. (Records passed whole as dataRecord.* values are fine; the asymmetry bites when you construct write payloads from read output.)

Reading and querying

There is no “search” Data node. The query path is an expression: the “Search for data” source — workflowData("custom.customer") in the JS surface — returns the collection as a list of records, which you then .filter(), sort, or slice like any list. That result feeds dataRecord.* inputs on the Update/Delete nodes.

cai data query <typeKey> --query "acme"   # plain-text search over serialized records — not a field predicate
cai data get <typeKey> <recordId>

Filters fail closed on empty values

A filter constraint compared against an empty value matches no records — silently. An optional text input that arrived empty ("") compared with equals matches nothing. If “constraint value empty” should mean “skip this constraint”, that is an explicit opt-in (ignore_empty_constraints), not the default. A green run returning zero records usually means an empty comparison value, not an empty store.

CSV import

cai data import <typeKey> --file leads.csv --dry-run
cai data import <typeKey> --file leads.csv

Columns match by field display name; unknown or duplicate headers fail the import. Import appends — it does not upsert. Limits: 10 MB, 10,000 data rows. Always dry-run first: it reports per-row validation without writing.

Development vs live stores

Publishing copies the workflow definition and schemas — never records. The live store starts empty. Promote data explicitly:

cai data copy-to-live <typeKey>

Copy-to-live requires the workflow to have been published and the live collection’s schema to match dev (same schema hash). A type edit after publish blocks the copy until you publish again.

What schema edits do to existing records

Schema surgery is safer than it looks, with specific rules:

• Deleting a type deactivates its collection but keeps every record; re-creating a type with the same key restores access to them.
• Deleting a field hides its stored values rather than erasing them.
• Changing a field’s type changes the schema hash — copy-to-live is blocked until the rails match again.
• Renaming a type’s display name is free (the key is stable). Recreating a type under a different key strands the old records under the old key.
• Expressions referencing a deleted type are caught by validation as DANGLING_TYPE_REFERENCE — run cai workflow validate after schema changes.

When to use the data store — and when not

Use it for state the workflow itself owns: dedupe sets, work queues, accumulated outputs, lookup tables that change with the workflow. Reads and writes are free and typed. Reach for a real database instead when data outlives the workflow, is shared across many systems, needs relational queries at scale, or is the system of record for other teams — the HTTP node talks to anything.

Example prompts

Create a "processed emails" type with the message id and processed time, and update the flow to skip any message whose id already has a record.

Import this CSV into the leads type — dry-run first and show me the validation report before committing.