Automation & CI

Everything here works with an API key. No browser, no session.

A working GitHub Actions job

name: Seed test data
on:
  workflow_dispatch:
  schedule:
    - cron: '0 2 * * *'

jobs:
  seed:
    runs-on: ubuntu-latest
    steps:
      - name: Push test data
        env:
          SEEDQL_KEY: ${{ secrets.SEEDQL_KEY }}
          TARGET_DSN: ${{ secrets.STAGING_DSN }}
        run: |
          set -euo pipefail
          API=https://seedql-production.up.railway.app

          RESPONSE=$(curl -sS -X POST "$API/api/generate/push" \
            -H "Authorization: Bearer $SEEDQL_KEY" \
            -H "Content-Type: application/json" \
            -d "{
              \"connection\": { \"db_type\": \"postgresql\", \"connection_string\": \"$TARGET_DSN\" },
              \"tables\": [
                { \"table_name\": \"users\",  \"row_count\": 1000, \"enabled\": true },
                { \"table_name\": \"orders\", \"row_count\": 8000, \"enabled\": true }
              ],
              \"truncate_first\": true
            }")

          echo "$RESPONSE"

          # Fail the job on a partial or empty write, not just on HTTP status.
          echo "$RESPONSE" | jq -e '.pushed_to_db == true' > /dev/null
          echo "$RESPONSE" | jq -e '.total_rows == .requested_rows' > /dev/null

          # Record the seed so this exact dataset can be reproduced.
          echo "seed=$(echo "$RESPONSE" | jq -r '.seed // "n/a"')" >> "$GITHUB_STEP_SUMMARY"

The two jq -e assertions matter. A push that writes nothing still returns 200; pushed_to_db is what tells you whether rows landed.

Long-running pushes

Above roughly 100,000 rows, use the async path so the job is not holding an HTTP connection:

JOB=$(curl -sS -X POST "$API/api/generate/push-async" \
  -H "Authorization: Bearer $SEEDQL_KEY" -H "Content-Type: application/json" \
  -d "$PAYLOAD" | jq -r .job_id)

while :; do
  STATUS=$(curl -sS "$API/api/jobs/$JOB" -H "Authorization: Bearer $SEEDQL_KEY" | jq -r .status)
  case "$STATUS" in
    completed) echo "done"; break ;;
    failed|cancelled) echo "job $STATUS"; exit 1 ;;
    *) sleep 5 ;;
  esac
done

Reading what happened

Four org-scoped list endpoints exist so a job can answer questions in one request instead of walking every project.

GET /api/generations?org_id=…      # run history
GET /api/snapshots?org_id=…        # captures, including scheduled ones
GET /api/schedule-runs?org_id=…    # every scheduled fire
GET /api/activity?org_id=…         # the audit stream

All accept limit (max 500) and offset; generations and snapshots also accept project_id, and generations and schedule-runs accept status.

Did last night's scheduled backups fail?

curl -s "$API/api/schedule-runs?org_id=$ORG&status=error" \
  -H "Authorization: Bearer $SEEDQL_KEY" | jq length

Did anything fail to generate today?

curl -s "$API/api/generations?org_id=$ORG&status=failed&limit=20" \
  -H "Authorization: Bearer $SEEDQL_KEY"

These lists are restricted to projects you can actually open. If a project has an explicit member roster and you are not on it, its rows never appear, even though you are in the org, and even if you name the project explicitly.

Reproducing a run

Every generation returns the seed it used. Given the same seed and the same table config, you get the same rows.

curl -s "$API/api/generations?org_id=$ORG&limit=1" \
  -H "Authorization: Bearer $SEEDQL_KEY" | jq '.[0].seed'

Feed that back as "seed": <value> to recreate the dataset a failing test ran against. This is the difference between a bug report and a rumour.

Practical notes

Store the key as a secret, never in the repo. It carries your permissions.

Use one key per pipeline, named for its consumer. Revoking one then does not break the others, and last_used_at tells you which are actually live.

A 503 with Retry-After means the backend is briefly saturated. Honour the header and retry; it is not a failure.

Rate limits apply per account. Sustained parallel pushes from many jobs will hit them; stagger schedules rather than firing every pipeline at midnight.