percy / percy-puppeteer

Visual testing with Puppeteer and Percy
https://percy.io
MIT License
49 stars 16 forks source link

Support for Jest / use of ES modules dynamic import in SDK #426

Closed thibaudcolas closed 2 years ago

thibaudcolas commented 2 years ago

Apologies if this isn’t the best place to report this! I couldn’t find information about Jest support anywhere in official resources so I’m not sure whether this would be considered a bug or not.

The problem

In #411, the import of SDK utils has been switched to use an ES modules dynamic import. This is problematic for users of Jest, because Jest uses Node’s VM API to execute code. The VM modules API for ES modules support is experimental (only available behind the --experimental-vm-modules feature flag).

For the Puppeteer SDK – this means v2.0.0 works fine in Jest, and v2.0.1 doesn’t work, with the following error when running percySnapshot:

TypeError: _vm(...).SyntheticModule is not a constructor

Environment

Details

Jest has experimental support for ES modules, so this can be resolved by adding the correct feature flag for Node to expose its vm.Module API. I believe there are a few things worth addressing in the SDK nonetheless (assuming the SDK attempts to support Jest):

Debug logs

The only relevant bit is the stack trace.

[…]
[percy:cli:exec] Running "jest" (1ms)
  ● Groups › loads

    TypeError: _vm(...).SyntheticModule is not a constructor

      39 |   beforeAll(async () => {
      40 |     await page.goto(`${TEST_ORIGIN}/admin/groups/2/`);
    > 41 |     await percySnapshot(page, 'test name')
         |           ^
      42 |   }, 10000);

      at Runtime.loadCjsAsEsm (../../../node_modules/jest-runtime/build/index.js:657:20)
      at percySnapshot (node_modules/@percy/puppeteer/index.js:9:15)
      at Object.<anonymous> (groups.test.js:41:11)

A worker process has failed to exit gracefully and has been force exited. This is likely caused by tests leaking due to improper teardown. Try running with --detectOpenHandles to find leaks.

[…]
Ran all test suites.
[percy:core:browser] Closing browser (25951ms)
[percy:core:browser] Browser closed (29ms)
[percy:core] Build not created (0ms)

Code to reproduce issue

I can provide a small reproduction case once confirmed Jest is indeed supported.


Workarounds

For people running into this, there are three possible workarounds:

Revert to v2.0.0 of the Puppeteer SDK

It works.

Run the v2.0.1 SDK with Node flag

Following Jest’s documentation. Here is what it looks like:

NODE_OPTIONS=--experimental-vm-modules percy exec -- jest

This will output a lot of warning messages.

Use @percy/sdk-utils directly

The code of v2.0.0 of the SDK can be copy-pasted and tweaked as needed to improve Jest support.

Robdel12 commented 2 years ago

Thanks for the issue! Hopefully one day Jest will catch up to ESM. Until then, we've removed the dynamic import (fixed in #430 & released in v2.0.2)