Jest incompatibility: Dynamic Import Callback Error

blakewest commented 3 months ago

Hi, I was trying to introduce PG into my codebase, and needed a way to run my tests against it. I'm using Prisma, and saw this and thought the adapter looked great. However, it seems there's some compatibility issue with Jest. If I try to import it, I get an error that breaks everything. See below for an example stack trace.

rebalance-batches-store › completeSimulations › completes

    PrismaClientKnownRequestError: 
    Invalid `prisma.rebalanceBatch.create()` invocation in
    /Users/blakewest/code/warbler-labs/mono/packages/mesh/tests/models/rebalance-batches-model-pg.test.ts:24:35

      21 const now = new Date();
      22 // Create 20 batches, with batch 0 being the most recent
      23 for (let i = 0; i < 20; i++) {
    → 24   await prisma.rebalanceBatch.create(
    A dynamic import callback was invoked without --experimental-vm-modules

      22 |     // Create 20 batches, with batch 0 being the most recent
      23 |     for (let i = 0; i < 20; i++) {
    > 24 |       await prisma.rebalanceBatch.create({
         |       ^
      25 |         data: {
      26 |           id: i.toString(),
      27 |           createdAt: new Date(now.getTime() - i * Duration.ofSeconds(1).toMillis()),

      at _n.handleRequestError (../../node_modules/@prisma/client/runtime/library.js:121:7749)
      at _n.handleAndLogRequestError (../../node_modules/@prisma/client/runtime/library.js:121:7057)
      at _n.request (../../node_modules/@prisma/client/runtime/library.js:121:6741)
      at async l (../../node_modules/@prisma/client/runtime/library.js:130:9355)
      at async Object.<anonymous> (tests/models/rebalance-batches-model-pg.test.ts:24:7)

In my investigation, I saw that 1.) Jest seems to have had a similar issue last year with Prettier V3. They said it would be fixed in Jest 30, but I installed the latest (alpha) release of Jest 30, and the same error occurs. , and 2.) If I removed the PG lite adapter, the tests work fine.

I'm not actually 100% if this is a Jest issue or a ElectricSQL issue, but I thought I'd raise it.

Has anyone else been able to use this PG-lite package with Jest 29?

samwillis commented 3 months ago

This will need some investigation to solve, a few initial thoughts / pointers:

I believe Jest was CJS only until recently, we do have both ESM and CJS builds in the in the package, but the CJS build is newer and less exercised.
There are a few places in the codebase where we branch depending on environment being browser/node, on the node branches a dynamic import is used to pull in additional dependencies. We have had issues with bundlers with this and it could be similar.
We use import.meta and __dirname to locate the WASM binaries, Jest may not be happy with this.

Writing a library that supports both node and the browser is complex and prone to these issues unfortunately, add in WASM and it gets worse. One day we will all have moved to ESM and the tooling will be perfect.

I'm very keen to find the fix for this as Jest is so widely used.

firelizzard18 commented 1 week ago

In vscode I can add the following to settings:

"jest.nodeEnv": {
        "NODE_OPTIONS": "--experimental-vm-modules"
}

However now I get:

TypeError: The URL must be of scheme file
    at open (node:internal/fs/promises:638:10)
    at Module.readFile (node:internal/fs/promises:1254:20)
    at ri (/home/firelizzard/src/athletifi/website/node_modules/@electric-sql/pglite/src/utils.ts:73:16)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)

presumably in the code that's using import.meta to locate the WASM binaries.

saenzramiro commented 1 week ago

Having PGLite running in Jest test suites will be wonderful.

firelizzard18 commented 1 week ago

The import.meta.url error was caused in part by @jest-environment: jsdom. The bundled version of pglite attempts to guess at what import.meta.url should return, and if document exists it assumes it's being run in a browser and returns http://localhost/main.js. By setting the Jest environment to node I was able to get past this. However, I ran into another issue:

https://github.com/electric-sql/pglite/blob/75f9f6d17db94992d50dcd6cab0f8c6bd1cee2e2/packages/pglite/src/postgresMod.ts#L1

Because the explicit .js, Jest throws an error when release/postgres is not an ESM module. I fixed this simply by removing .js. I opened #423 to make this change.

electric-sql / pglite

Jest incompatibility: Dynamic Import Callback Error #224