Using the CLI in CI

Everything the CLI does locally it does headlessly in CI: run a suite on every push, gate merges on the result, and leave triage breadcrumbs when something fails.

01Authenticate the runner

Create an API key in Settings → API Keys, store it as a CI secret, and expose it as COFACTOR_API_TOKEN — the CLI picks it up with no login step:

env:
  COFACTOR_API_TOKEN: ${{ secrets.COFACTOR_API_TOKEN }}

See CLI authentication for the details.

02Gate merges on a suite

cofactor test-suite run streams per-flow results and exits 1 if any flow fails — which is the entire CI contract:

name: e2e
on: [push]

jobs:
  suite:
    runs-on: ubuntu-latest
    env:
      COFACTOR_API_TOKEN: ${{ secrets.COFACTOR_API_TOKEN }}
    steps:
      - run: npm install -g @runcofactor/cli
      - run: cofactor test-suite run --suite "smoke" --application "Acme Store"

Target by --suite (a named, layered suite), or filter ad hoc with --tag and --name-pattern. CI runs the promoted flow set selected by those filters, so approve a flow to record review and promote it before expecting it to appear in a suite. The execution id prints at the end of the stream — capture it if a later step wants the detail (cofactor test-suite get <id> --failures-only).

03When the gate goes red

Don't make humans read raw CI logs — point them at triage, which clusters the failures by root cause:

cofactor triage wait <executionId> --timeout 600
cofactor triage failures <executionId> --new-only   # just the regressions

A useful pattern: a follow-up CI step that runs triage wait and posts the cluster summary to your PR or Slack channel, so the human conversation starts at "one auth regression" instead of "7 failures." See Triaging a failure.

04Tips

• Pin nothing — npm install -g @runcofactor/cli per job keeps runners current; the CLI is backwards-compatible with itself.
• Scope the suite to the change when you can (--tag checkout) and save the full sweep for merges to main.
• Schedules are just CI cron — a nightly full-suite run with triage posting to Slack catches what per-PR runs scope out.

05Next steps

• Running your tests — the same suites, on demand and scheduled
• Command reference — everything else the CLI can do in a pipeline