Why Golden Tests Pass on macOS but Keep Failing in Linux CI

Almost every team that adopts golden testing goes through the same sequence of frustration, without exception. Running flutter test locally gives you a green light, but push that exact same commit to CI and the golden tests turn red. Open the diff image and it’s just a handful of pixels off, so people start shrugging it off with “it’ll probably pass if we just retry” — and eventually the entire golden test job loses the team’s trust.

This isn’t an introductory “what is a golden test” post. It assumes you’ve already adopted golden testing, and it digs into why matchesGoldenFile produces different results between a local macOS machine and a CI Linux container — and how to fix that structurally.

Key Takeaways

• The root cause of golden test flakiness is almost always one of three things: fonts, DPR (device pixel ratio), or platform-specific rasterizers. It’s not a code logic problem — it’s a rendering environment problem.
• Your local macOS machine draws text with CoreText, while a CI Linux container uses FreeType/fontconfig. Even with the identical font file, different antialiasing and hinting algorithms produce pixel-level differences in the output image.
• There are broadly three fixes: (1) bundle fonts into the test environment to eliminate renderer dependence, (2) pin DPR and text scale explicitly in the test code, and (3) use tolerance-based diffing instead of exact byte comparison.
• From a CI design standpoint, you need a pipeline that runs golden tests as a separate job, pins the rendering environment (Docker image) to a specific version, and uploads diff images as artifacts on failure.

Why Golden Tests Are Especially Prone to “It Works on My Machine”

A typical unit test compares logical input and output values, so the result is the same regardless of platform. A golden test, though, compares the actual rasterized pixel image of a widget. Since what’s under test isn’t “is the logic correct” but “is the picture identical,” even a small difference in the rendering engine, font, or screen density is enough to fail it.

Flutter’s matchesGoldenFile performs an exact byte-for-byte comparison by default. The official Flutter documentation itself states that custom fonts can render differently depending on the platform or Flutter version, and it warns that a golden file generated on Windows will almost certainly fail on a different OS. In other words, this isn’t a bug — it’s a documented limitation.

Root Cause 1: Fonts Differ

By default, the Flutter test binding only loads the Ahem font. Ahem is a test-only font that draws a solid black box (more precisely, a glyph box) for every character, so if you don’t load a real font, the golden image of any widget containing text ends up filled with nothing but boxes.

Teams typically make one of two mistakes here.

• A system font (e.g., San Francisco on macOS) happens to be installed locally only, so a plausible-looking golden image gets produced, and it gets committed as-is.
• The app’s font is loaded, but the CI container doesn’t have that font file, or it can’t be bundled due to licensing, so it renders with a fallback font instead.

Both cases produce exactly the symptom of “works locally, fails on CI.”

Root Cause 2: DPR (Device Pixel Ratio) and Text Scale Differ

The actual pixel dimensions of the golden image WidgetTester draws are determined by logical pixels × DPR. If your local macOS dev machine’s Retina display settings or your IDE’s default test runner settings differ from CI’s defaults, the image resolution itself ends up different even for the exact same widget tree, producing a diff. The text scale factor (textScaleFactor) works the same way — even a tiny difference in system accessibility settings or the test environment’s initial value can shift where line breaks occur and make the image come out completely different.

Root Cause 3: Platform-Specific Rasterizer Differences

There are cases where tests still fail even after perfectly pinning font and DPR. That’s because text antialiasing, subpixel rendering, and hinting algorithms differ across rendering backends. The Flutter GitHub issue tracker even has reports that “golden test pixel differences range from 1 to 90 pixels depending on the host platform (Windows Docker vs. macOS Docker), even with the identical Docker image” — this problem originates deep inside Flutter’s text shaping and rasterization stack. In other words, you have to go in accepting that pinning the container with Docker alone may not solve the problem 100%.

Why Local macOS and a CI Linux Container Structurally Draw Different Pictures

To summarize, the following structural differences stack up.

• Text rendering stack: macOS uses CoreText, while Linux mostly uses a FreeType + fontconfig combination. Even passing in the same TTF file, differing hinting and antialiasing algorithms produce different pixel values.
• Font availability: A local macOS machine has a rich set of system fonts installed, but a minimal CI Linux container (e.g., the ubuntu-latest GitHub Actions runner or a Flutter CI Docker image) has no fonts at all beyond what you explicitly specify, so the fallback chain ends up different.
• GPU vs. software renderer: CI is mostly a headless environment and thus uses a software rasterizer, while local macOS can take a hardware-accelerated path.
• Flutter/Skia engine version mismatch: If the local SDK version differs from the one cached on CI, Skia’s own rendering output changes.

When these four factors overlap, you get the classic flakiness pattern of “the code didn’t change, but the golden test broke anyway.”

Fix 1: Bundle and Pin Fonts in the Test Environment

The first thing to do is force the test to always render with the same font, regardless of which environment it runs in.

Loading App Fonts via flutter_test_config.dart

The Flutter test framework walks up from the directory containing the test file looking for flutter_test_config.dart and applies it before running. If you load fonts there, it applies to every test at once.

// test/flutter_test_config.dart
import 'dart:async';
import 'package:golden_toolkit/golden_toolkit.dart';

Future<void> testExecutable(FutureOr<void> Function() testMain) async {
  await loadAppFonts(); // Loads Roboto + any custom fonts registered in pubspec
  return testMain();
}

loadAppFonts() automatically reads the fonts: section registered in pubspec.yaml along with fonts from dependent packages and injects them into the test binding. There are a few gotchas, though.

• The Flutter test environment only supports a single weight (.ttf) per font family, so widgets that mix Bold and Regular can look subtly different from the real app.
• To use Material icons, pubspec must have uses-material-design: true, or the icon font won’t load.
• Framework-internal text, like the debug banner, can still be drawn with Ahem, leaving a corner you can’t fully control.

Leaning Into Ahem Instead

Paradoxically, there’s also a strategy of giving up on “rendering beautifully with a real font” and instead using Ahem as a CI-only verification tool. If what you actually want to verify is layout — alignment, size, whether text wraps — rather than text content, Ahem, which always draws the same box, is actually more stable than a real font whose glyph shapes risk varying by platform. alchemist, which we’ll cover shortly, officially supports this strategy under the concept of a “CI golden.”

golden_bricks: A Middle Ground Between Ahem and Real Fonts

The problem with Ahem is that every character is an identical box, so it can’t test cases that only get validated if character widths actually differ — caret position, text selection, line wrapping. golden_bricks is a package that fills this gap: it draws a differently sized rectangle per character, rendering “boxes, but with widths that vary like real text.”

# pubspec.yaml (dev_dependencies)
golden_bricks: ^1.0.0

MaterialApp(
  theme: ThemeData(fontFamily: goldenBricks),
  home: const MyWidget(),
)

Adopting a deterministic font like this as your standard instead of a platform-dependent real font can eliminate, from the outset, any room for CoreText/FreeType differences to affect the result at all.

Fix 2: Explicitly Pin DPR and Text Scale in the Test Code

You can explicitly pin screen density and size through the TestFlutterView (tester.view) that WidgetTester provides. If you don’t specify these values, you end up quietly depending on the defaults of whatever machine runs the test.

testWidgets('Product card golden test', (tester) async {
  tester.view.physicalSize = const Size(1080, 2400);
  tester.view.devicePixelRatio = 3.0;

  // Always reset after the test so it doesn't affect the next one
  addTearDown(tester.view.reset);

  await tester.pumpWidget(const MyApp(home: ProductCard()));
  await tester.pumpAndSettle();

  await expectLater(
    find.byType(ProductCard),
    matchesGoldenFile('goldens/product_card.png'),
  );
});

The key points are as follows.

• Explicitly specifying physicalSize and devicePixelRatio means the local machine’s display settings or the CI runner’s default resolution can’t influence the result.
• Be sure to call addTearDown(tester.view.reset). Otherwise, a later test in the same test file inherits the DPR value left behind by the previous test, creating yet another form of flakiness that depends on execution order.
• By the same logic, it’s safer to pin text scale via tester.platformDispatcher.textScaleFactorTestValue, or to wrap with MediaQuery and explicitly override it, e.g. with textScaler: TextScaler.noScaling.

Rather than repeating this pattern in every golden test, it’s recommended to wrap it in a shared helper function (something like pumpGolden) so the whole team is forced to use the same baseline values. If baseline values differ from file to file, you get yet another inconsistency — “this file uses DPR 2.0 but that one uses 3.0.”

Fix 3: Use a Tolerance-Based Diff Tool

Even with fonts and DPR fully pinned, there realistically remain cases where you don’t get byte-for-byte identical output, because of the rasterizer differences mentioned earlier. At that point, switching from exact byte comparison to tolerance-based comparison is the practical choice.

golden_toolkit vs. alchemist

For a new project, it’s reasonable to consider alchemist ahead of golden_toolkit. golden_toolkit is marked discontinued on pub.dev, whereas alchemist was designed to directly target the exact problem this post covers — cross-platform rendering differences.

Using alchemist’s diffThreshold

void main() {
  setUpAll(() {
    AlchemistConfig.current = AlchemistConfig(
      platformGoldensConfig: const PlatformGoldensConfig(
        enabled: true,
      ),
      ciGoldensConfig: const CiGoldensConfig(
        enabled: true,
      ),
    );
  });

  goldenTest(
    'Product card',
    fileName: 'product_card',
    pixelDiffConfig: const GoldenTestPixelDiffConfig(threshold: 0.01),
    widget: const GoldenTestGroup(
      children: [ProductCard()],
    ),
  );
}

threshold: 0.01 means differences affecting less than 1% of total pixels won’t be treated as a failure. Setting this value too high has the pitfall of letting real UI regressions pass through, so it’s recommended to start with a small value between 0.005 and 0.01 and tune it based on your team’s actual flakiness frequency.

The Custom Comparator Option

If you’d rather not add a package, another option is to subclass LocalFileComparator, compute the pixel difference ratio yourself in compare(), and register the custom comparator in flutter_test_config.dart. That route requires you to implement image decoding, resizing, and antialiasing edge handling yourself, though, so if your team is small, adopting alchemist is far more cost-effective.

Fix 4: Split Golden Tests Into Their Own CI Job

Even after nailing down fonts, DPR, and tolerance, flakiness will resurface if the CI pipeline design itself is sloppy. The following principles are recommended.

• Split golden tests into a job separate from regular unit tests. Golden tests are sensitive to the rendering environment, so their retry policy and artifact upload settings on failure need to be different from unit tests’.
• Pin the Flutter SDK version and Docker image precisely (specify flutter --version explicitly, or use a SHA-pinned Docker image). You can’t avoid the situation where an SDK minor version bump changes Skia’s rendering output and forces you to regenerate all goldens, but you can at least eliminate the “my local SDK differs from CI’s SDK” cause.
• Upload the diff image as an artifact on failure, so reviewers can visually compare the actual images instead of guessing “how many pixels off” from a log.
• Always regenerate goldens in the same environment as CI (Docker). Committing a file updated locally on macOS via flutter test --update-goldens as-is is the same as reproducing, on your own, every problem this post describes.

GitHub Actions Example

name: golden-tests

on:
  pull_request:
    paths:
      - 'lib/**'
      - 'test/**'

jobs:
  golden:
    runs-on: ubuntu-latest
    container:
      image: ghcr.io/your-org/flutter-ci:3.35.0  # Custom image with the SDK version pinned
    steps:
      - uses: actions/checkout@v4

      - name: Cache pub dependencies
        uses: actions/cache@v4
        with:
          path: |
            ~/.pub-cache
            .dart_tool
          key: pub-${{ hashFiles('pubspec.lock') }}

      - run: flutter pub get

      - name: Run golden tests
        run: flutter test --tags golden

      - name: Upload golden diff on failure
        if: failure()
        uses: actions/upload-artifact@v4
        with:
          name: golden-failures
          path: |
            test/**/failures/*.png

• Tag golden tests separately with --tags golden to cut down run time and narrow down failure causes (add the @Tags(['golden']) annotation at the top of the test file).
• Key the pub cache to cut dependency install time, but don’t cache the SDK image itself — since it affects rendering output — and instead pin its version explicitly. If a cached image gets silently updated, you get a fresh new source of flakiness: “the golden that passed yesterday suddenly breaks today.”
• To reproduce CI conditions locally, the most reliable approach is running the exact same Docker image CI uses, locally, to regenerate goldens. That said, as with the GitHub issue mentioned earlier, keep in mind that a different host OS may not guarantee identical pixels even from the same image. In that case, the last resort is nudging diffThreshold up a bit.

Checklist for When It Still Fails

• Whether flutter_test_config.dart is actually within the directory scope being run (commonly missing per-package in a monorepo)
• Whether the Flutter SDK version that generated the golden file exactly matches CI’s SDK version (compare flutter --version)
• Whether you’ve omitted tester.view.reset() or addTearDown, letting a previous test’s DPR/text scale leak in
• Whether the font license file in pubspec is actually committed to the repository and accessible from CI too (some commercial fonts are only installed on local developer machines and aren’t in the repo)
• If you’re using alchemist, whether the one you actually want CI to verify — ciGoldensConfig or platformGoldensConfig — is enabled correctly

Wrapping Up

Golden test flakiness mostly arises not because “the test code is wrong” but because the rendering environment the test runs in is uncontrolled. Bundle fonts to eliminate renderer differences, explicitly pin DPR and text scale in code, absorb whatever fine-grained differences remain with tolerance-based diffing, and finally design the CI pipeline itself to be reproducible. Walk through these four steps in order, and the “works on my machine, breaks only on CI” golden test problem mostly disappears.