Adam - Building a CI/CD Pipeline I Actually Trust

TL;DR

CI = automatically test/build every push. CD = automatically deliver/deploy the result.
A good pipeline is a gate: nothing reaches production it didn't pass through.
Cache dependencies and Docker layers or your pipeline becomes the slow part.
Rolling and canary deploys let you ship without downtime - and roll back fast.

Why I finally automated

I was still deploying by SSH-ing in and pulling. It was error-prone, undocumented, and only I could do it - a recipe for the classic "works in CI, broken in prod because I forgot a step." The whole point of the Docker work earlier was reproducibility; deploying by hand threw that away. Time to make the machine do it.

CI: the safety gate

Continuous Integration means every push runs the same checks automatically. The principle that reframed it for me: if a check isn't in the pipeline, it isn't enforced - it's a suggestion people skip under deadline.

My pipeline runs, in order, failing fast:

name: CI
on: [push, pull_request]
 
jobs:
  verify:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 20
          cache: npm           # cache deps between runs
      - run: npm ci
      - run: npm run lint
      - run: npm run typecheck
      - run: npm test
      - run: npm run build

Lint → typecheck → test → build. Cheap checks first so feedback is fast. A red pipeline blocks the merge - the gate does its job.

Speed matters (or people route around it)

My first pipeline took 9 minutes and people started merging without waiting. Two fixes brought it under 2:

Cache dependencies (cache: npm above) so npm ci isn't re-downloading the world.
Cache Docker layers with buildx so image builds reuse the npm ci layer from the Docker post - same principle, different context.

A slow pipeline that gets bypassed protects nothing. Speed is a correctness feature.

CD: from green build to running code

Once CI is green on main, the delivery job builds a versioned image, pushes it to a registry, and triggers the deploy:

  deploy:
    needs: verify
    if: github.ref == 'refs/heads/main'
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Build & push image
        run: |
          docker build -t $REGISTRY/taskflow:${{ github.sha }} .
          docker push $REGISTRY/taskflow:${{ github.sha }}
      - name: Deploy
        run: ./scripts/deploy.sh ${{ github.sha }}

Tagging images by git SHA instead of latest is something I now consider essential - every deploy is traceable to an exact commit, and rollback is "deploy the previous SHA."

Deploy strategies (shipping without downtime)

Rolling: replace instances a few at a time. The load balancer drains old ones and health-checks new ones in. Zero downtime, gradual rollout.
Blue-green: stand up a full parallel environment (green), switch traffic over, keep blue as instant rollback.
Canary: route a small slice (say 5%) of traffic to the new version, watch the metrics/traces from my observability post, then ramp to 100% only if error rate and latency stay healthy.

I default to rolling for routine changes and canary for anything risky. The observability work pays off directly here: the canary decision is "do the dashboards look healthy?", made from real signal rather than a gut feeling.

Secrets and config

Secrets never live in the repo. They come from the CI provider's encrypted store (GitHub Actions secrets) injected as env vars at deploy time, and config follows the twelve-factor rule: config in the environment, not the code. Same image, different config per environment - which loops right back to the reproducibility goal I started with.

Takeaways

The pipeline is the only path to prod. No side doors.
Fast pipelines get used; slow ones get bypassed. Cache aggressively.
Tag by commit SHA for traceability and trivial rollback.
Match deploy strategy to risk - rolling for routine, canary for scary.

My delivery is automated, but it's still pushing to a single hand-managed server. It's time to go to the cloud properly. I'm starting my AWS journey - and the first building block is storage: S3.