Freshly generated snapshot is corrupt

[angeloashmore]

Hi there!

AVA reports the the following error when using snapshots. The snapshot has been freshly generated locally and pushed to GitHub to run in an GitHub Actions workflow.

  value-group.ts › creates a mock Group field value

  Could not compare snapshot

  The snapshot file is corrupted.

  File path: /home/runner/work/prismic-mock/prismic-mock/test/snapshots/value-group.test.ts.snap

  Please run AVA again with the --update-snapshots flag to recreate it.

  › snapshotTwiceMacro (/home/runner/work/prismic-mocktest/__testutils__/snapshotTwiceMacro.ts:7:4)

This error only appears when running within GitHub Actions. The test passes locally.

Any idea what could cause a corrupted snapshot to be generated?

Relevant files and tests:

• Test: https://github.com/prismicio/prismic-mock/blob/97861bf3cf46cb9a42c6e2b5d6b4c5c60aab6cd2/test/value-group.test.ts#L8

• Macro used by the test: https://github.com/prismicio/prismic-mock/blob/97861bf3cf46cb9a42c6e2b5d6b4c5c60aab6cd2/test/__testutils__/snapshotTwiceMacro.ts#L3-L8

• Snapshot (Markdown): https://github.com/prismicio/prismic-mock/blob/97861bf3cf46cb9a42c6e2b5d6b4c5c60aab6cd2/test/snapshots/value-group.test.ts.md

• Snapshot (binary): https://github.com/prismicio/prismic-mock/blob/97861bf3cf46cb9a42c6e2b5d6b4c5c60aab6cd2/test/snapshots/value-group.test.ts.snap

• Failing test: https://github.com/prismicio/prismic-mock/runs/3305415519?check_suite_focus=true#step:7:199

AVA config

// ava.config.js

export default {
    extensions: ["ts"],
    files: ["./test/**/*.test.ts"],
    require: ["ts-eager/register"],
    verbose: true,
};

AVA version

$ npx ava --version
3.15.0

Thank you!

[novemberborn]

That's strange!

The snapshot format has changed in AVA 4 (in pre-release). Do you think you could give that a go? Perhaps the problem goes away… see https://github.com/avajs/ava/releases/tag/v4.0.0-alpha.2 and https://github.com/avajs/ava/releases/tag/v4.0.0-alpha.1.

[angeloashmore]

@novemberborn Thanks for the suggestion. I wasn't aware there was a AVA 4 pre-release.

Unfortunately, the same error occurs. This PR updates the previously linked test to AVA 4 with new snapshots: https://github.com/prismicio/prismic-mock/pull/1

I can get around this by not using snapshots for those tests so it's not blocking me. I'm happy to help diagnose the issue, however. Thank you!

[angeloashmore]

In case it is relevant, here is my local system info where tests pass:

  System:
    OS: macOS 10.15.6
    CPU: (12) x64 Intel(R) Core(TM) i5-10400 CPU @ 2.90GHz
    Shell: 4.4.23 - /nix/store/n6kzbax7sjy3kha78hpfh024ghbykgxa-bash-interactive-4.4-p23/bin/bash
  Binaries:
    Node: 14.17.0 - /nix/store/jvkhrd4b513zrsbbmssg07hpkjpc9bmh-nodejs-14.17.0/bin/node
    npm: 7.15.1 - /nix/store/v8wdcakdc4k9pbykikl7vsijq4dp6z1c-node_npm-7.15.1/bin/npm

[novemberborn]

Hey @angeloashmore I haven't had a chance yet to look into this further.

[angeloashmore]

@novemberborn Not a problem! I really appreciate you following up.

[novemberborn]

@angeloashmore sorry for the wait.

I couldn't actually run the tests from https://github.com/prismicio/prismic-mock/pull/1 but I copied a snapshot file into a separate project and set things up so AVA would load it.

AVA snapshots start with some readable text, followed by a version number, followed by a SHA256 checksum of the rest of the file. This checksum is corrupted.

This is how the checksum is computed and the data is prepared before file is written:

https://github.com/avajs/ava/blob/725c5d6241b3df63c030a826ab86acba6b9f1d67/lib/snapshot-manager.js#L190-L196

And how the checksum is compared:

https://github.com/avajs/ava/blob/725c5d6241b3df63c030a826ab86acba6b9f1d67/lib/snapshot-manager.js#L221-L223

I don't see room for false positives there, so I'm pretty confident the snapshot is corrupted. Indeed zlib.gunzipSync() throws its own data check error.

Why that is happening I do not know. My best guess is that there are concurrent writes, except that we're (attempting) to write the files atomically. And regardless that assumes you're running the same test file twice, concurrently.

Perhaps you can try and isolate the error?