Test suites
A test suite is a named, reusable collection of workflows you run together. Where a [test run](/reference/test-runs) is one execution, a test suite is the *template* that defines which workflows belong, what order they run in, and how failures are handled. Running a suite produces a test run.
01Layers
A suite organizes its workflows into layers:
- Workflows in the same layer run in parallel.
- Layers run in order — layer 0 finishes before layer 1 starts.
Use layers to express dependencies. Put setup-style flows (sign-up, seed an account) in an early layer, and the flows that depend on them in a later layer, so prerequisites are in place before the dependent workflows run.
02Staged members
A workflow can be a member of a suite but staged — included in the suite's definition without running. Staged members are excluded from execution until you assign them to a layer. Use staging to add a workflow to a suite while you're still stabilizing it, without affecting the suite's results.
03Failure handling
A suite runs with one of two failure modes:
| Mode | Behavior |
|---|---|
| Continue (default) | A failure in one layer doesn't stop later layers. You get full coverage every run. |
| Stop on layer failure | If any workflow in a layer fails, later layers don't run. Use this when a later layer is pointless once an earlier one breaks. |
04Running a suite
Run a suite from the app, or from the CLI:
canary test-suite run canary test-suite run --tag smoke
Each run creates a test run tied to the suite, so you get the same grouped results, pass rate, and triage as any other test run — plus a record of which suite produced it. Re-running the same suite produces a new test run each time, so you can compare results across runs.
05Related
- Test runs — what a suite produces when it runs
- Runs & results
- Flows — the workflows a suite is built from
- Triage & diagnostics