Visual Regression Testing with GitHub Actions and PixelEagle

Adding visual regression testing to a GitHub Actions workflow takes about 10 minutes. By the end of this guide, you'll have automatic visual comparisons on every pull request, with a link to review the diffs.

Prerequisites

• A GitHub repository with a web application
• A PixelEagle account and a project token (create a project at pixel-eagle.com, then generate a token in the project settings)
• A way to capture screenshots (we'll use Playwright)

Step 1: Capture Screenshots

PixelEagle works with any screenshot source. The most common approach for web apps is Playwright: write the screenshots to a directory, and the upload step picks them up from there:

// tests/visual.spec.js
const { test } = require('@playwright/test')

test('homepage', async ({ page }) => {
  await page.goto('http://localhost:3000')
  await page.waitForLoadState('networkidle')
  await page.screenshot({ path: 'screenshots/homepage.png', fullPage: true })
})

You can also use Puppeteer, Cypress, Selenium, or any tool that produces PNG screenshots. PixelEagle doesn't care how you capture; it handles comparison and review.

Step 2: The Simple Path, One Action

The PixelEagle composite action installs the CLI and runs the full create-upload-compare flow in one step:

name: Visual Regression Tests

on:
  pull_request:
    branches: [main]

jobs:
  visual-tests:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-node@v4
        with:
          node-version: '20'

      - name: Install dependencies
        run: npm ci

      - name: Install Playwright browsers
        run: npx playwright install --with-deps chromium

      - name: Start application
        run: npm run dev &

      - name: Wait for app to be ready
        run: npx wait-on http://localhost:3000

      - name: Capture screenshots
        run: npx playwright test tests/visual/

      - name: Upload to PixelEagle and compare
        uses: vleue/PixelEagle-cli/run@v0.3.0
        with:
          token: ${{ secrets.PIXEL_EAGLE_TOKEN }}
          screenshots-path: screenshots
          metadata: '{"commit": "${{ github.sha }}", "branch": "${{ github.head_ref }}"}'
          compare-filter: branch:main

That one step creates a run tagged with your branch and commit, uploads everything in screenshots/, and compares it against the most recent run whose metadata matches branch:main.

Step 3: Configure Your Secret

In your GitHub repository settings, add PIXEL_EAGLE_TOKEN as a repository secret (Settings → Secrets and variables → Actions). The token comes from your PixelEagle project settings and is scoped to that project: no account-wide credentials in CI.

Step 4: Keep the Baseline Fresh

The compare-filter: branch:main line means PRs are compared against the latest run from your main branch. For that to work, main needs runs too. Add the same workflow on push to main, with the branch metadata set accordingly:

on:
  push:
    branches: [main]

The metadata expression ${{ github.head_ref }} is only set on pull requests; on push events use ${{ github.ref_name }} instead. A common setup is a single workflow triggered on both, with:

metadata: '{"commit": "${{ github.sha }}", "branch": "${{ github.head_ref || github.ref_name }}"}'

Step 5: The Manual Path, Full Control

If you want to drive the CLI yourself (custom metadata, gating the job on results, multiple screenshot batches), install it and script the three steps. The install script drops a pixeleagle binary in the current directory:

- name: Install PixelEagle CLI
  run: curl -fsSL https://pixel-eagle.com/install.sh | sh

- name: Upload & compare screenshots
  env:
    PIXEL_EAGLE_TOKEN: ${{ secrets.PIXEL_EAGLE_TOKEN }}
  run: |
    RUN_ID=$(./pixeleagle new-run --metadata '{"commit": "${{ github.sha }}", "branch": "${{ github.head_ref }}"}')
    ./pixeleagle upload-screenshots --clean-name $RUN_ID screenshots/*.png
    ./pixeleagle compare-run $RUN_ID --filter branch:main --wait --print-details

new-run prints the run ID to stdout, so capturing it in RUN_ID is all the wiring you need. --clean-name uses each file name (without extension) as the screenshot name. compare-run --wait --print-details blocks until the comparison finishes, then prints a link to the comparison and a summary of unchanged, new, missing, and changed screenshots, right in your workflow log.

Step 6: Fail the Job on Visual Changes (Optional)

By default the comparison is informational: the workflow log links to the diff review. If you want the job to fail when screenshots changed, use the JSON output and check the result:

- name: Compare and gate
  env:
    PIXEL_EAGLE_TOKEN: ${{ secrets.PIXEL_EAGLE_TOKEN }}
  run: |
    RESULT=$(./pixeleagle --format json compare-run $RUN_ID --filter branch:main --wait)
    echo "$RESULT" | jq '{changed: (.diff | length), new: (.new | length), missing: (.missing | length)}'
    test "$(echo "$RESULT" | jq '.diff | length')" -eq 0

Whether to gate is a team choice: some teams prefer a red check on any visual change so it must be reviewed; others keep it green and review the link. Intended changes are part of normal development, so if you gate, make sure reviewing and approving is part of your PR routine.

Step 7: Review Visual Diffs

When a pull request triggers the workflow:

1. Screenshots are captured from the PR branch
2. PixelEagle compares them against the baseline run from main
3. The comparison link appears in the workflow output
4. Open the link to review diffs side-by-side, with heatmaps highlighting what changed
5. Intentional changes get merged and become the new main baseline on the next push

Tips for Reliable Visual Tests

Freeze Animations

Add this to your test setup to prevent animation-related flakiness:

await page.addStyleTag({
  content: `
    *, *::before, *::after {
      animation-duration: 0s !important;
      animation-delay: 0s !important;
      transition-duration: 0s !important;
      transition-delay: 0s !important;
    }
  `,
})

Handle Dynamic Content

Mock timestamps, user avatars, and other dynamic content:

await page.evaluate(() => {
  // Freeze the clock
  Date.now = () => 1700000000000
})

Test Multiple Viewports

Capture screenshots at different breakpoints to catch responsive layout issues:

const viewports = [
  { width: 375, height: 812, name: 'mobile' },
  { width: 768, height: 1024, name: 'tablet' },
  { width: 1440, height: 900, name: 'desktop' },
]

for (const vp of viewports) {
  await page.setViewportSize({ width: vp.width, height: vp.height })
  await page.screenshot({ path: `screenshots/homepage-${vp.name}.png`, fullPage: true })
}

Use ꟻLIP Comparison

Enable NVIDIA ꟻLIP comparison in your project settings to filter out sub-pixel rendering differences between CI environments. This dramatically reduces false positives compared to exact pixel comparison; see our pixel diff vs perceptual diff deep dive.

Scaling Up

As your test suite grows, a few things help:

• Use Playwright's built-in parallelism to capture screenshots faster
• Only capture screenshots for pages affected by the PR's changes
• Add Storybook visual tests for individual components alongside page-level tests
• If you test in multiple environments, tag runs with metadata like os or browser and use compare-same: os so each environment is compared against its own baseline

Key Takeaways

• The composite action (vleue/PixelEagle-cli/run) gets you from zero to visual comparisons in one workflow step
• Metadata filters (branch:main) pick the comparison baseline automatically, no manual run selection
• The CLI's JSON output lets you gate the job on visual changes when you want a hard check
• Freeze animations and mock dynamic content for reliable results
• Start with critical pages, expand coverage as you gain confidence