Memory allocation error

kierangraham commented 4 years ago

When running Camaro using jest --watch, eventually a memory allocation error occurs after it runs for a few minutes.

Here's the example output:

 FAIL src/test.spec.ts
  ● Failing test

    RangeError: WebAssembly.Memory(): could not allocate memory

      at ../node_modules/camaro/dist/camaro.js:16:416
      at ../node_modules/camaro/worker.js:12:26
      at Object.<anonymous> (../node_modules/camaro/worker.js:10:15)

Camaro version: "6.0.2"

tuananh commented 4 years ago

@kierangraham do you have a minimal reproducible code that I can try?

kierangraham commented 4 years ago

I'll try to find a minimum sample that triggers this and post it; until then here is some further info on the error:

Jest has detected the following 3 open handles potentially keeping Jest from exiting:

  ●  Piscina

      at new EventEmitterReferencingAsyncResource (../node_modules/eventemitter-asyncresource/src/index.ts:23:5)
      at new EventEmitterAsyncResource (../node_modules/eventemitter-asyncresource/src/index.ts:46:7)
      at new Piscina (../node_modules/piscina/src/index.ts:827:5)
      at Object.<anonymous> (../node_modules/camaro/index.js:4:14)

  ●  WORKER

      at ThreadPool._addNewWorker (../node_modules/piscina/src/index.ts:520:20)
      at ThreadPool._ensureMinimumWorkers (../node_modules/piscina/src/index.ts:514:12)
      at new ThreadPool (../node_modules/piscina/src/index.ts:508:10)
      at new Piscina (../node_modules/piscina/src/index.ts:876:18)
      at Object.<anonymous> (../node_modules/camaro/index.js:4:14)

  ●  MESSAGEPORT

      at ThreadPool._addNewWorker (../node_modules/piscina/src/index.ts:528:30)
      at ThreadPool._ensureMinimumWorkers (../node_modules/piscina/src/index.ts:514:12)
      at new ThreadPool (../node_modules/piscina/src/index.ts:508:10)
      at new Piscina (../node_modules/piscina/src/index.ts:876:18)
      at Object.<anonymous> (../node_modules/camaro/index.js:4:14)

addaleax commented 4 years ago

@tuananh I’ll try to take a look. The “3 open handles” warning doesn’t seem to be helpful here, though – none of these handles should be refed, i.e. none of them should keep the process running by itself.

addaleax commented 4 years ago

Hm yeah, there’s nothing here that sticks out to me. A reproduction would definitely be helpful. Knowing the Node.js version may or may not also be relevant?

kierangraham commented 4 years ago

@tuananh I've managed to put together a sample project where you can reproduce the error. You can find the repo here: https://github.com/kierangraham/camaro-memory-error

Let me know if there's anything I can try to help out with!

addaleax commented 4 years ago

@kierangraham Thanks for the reproduction!

I think the problem here is that the camaro module is re-loaded multiple times for a single process, which each create a new worker pool. This is not really a problem outside of testing, because in “the real world”, processes don’t generally re-load modules.

I’m not sure what the best way forward here is, tbh. /cc @jasnell

tuananh commented 4 years ago

@kierangraham i'm closing this as I honestly don't know what to do here either.

jasnell commented 4 years ago

Hey @tuananh ... My schedule is pretty crazy this week but I've got this issue open and a todo to take a look at it next week to see if something can be figured out. I'll need to take some time to go through your specific set up in detail.

kierangraham commented 4 years ago

Does anyone know of any pointers to shut down the worker pool correctly? Even when not in watch mode I see errors that something has prevented Jest from shutting down correctly when using detectOpenHandles. Any insights appreciated on how the worker pool could be released!

liltoto commented 3 years ago

WebAssembly.Memory(): could not allocate memory Node version 12.x.x I had the same issue and found out it was a bug in V8. Upgrading to version 14.15.0 solved this problem.

eduardomourar commented 3 years ago

I can confirm that jest worked best with Piscina (and worker threads in general) by using nodejs v14+ and jest circus runner.

tuananh commented 3 years ago

Glad to hear it’s working for you

On Wed, 25 Nov 2020 at 08:28 Eduardo de Moura Rodrigues < notifications@github.com> wrote:

I can confirm that jest worked best with Piscina (and worker threads in general) by using nodejs v14+.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/tuananh/camaro/issues/113#issuecomment-733404436, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAEZETXVMFMMLATGINQ64MDSRRMUZANCNFSM4OBTFPQQ .

jermainemercado commented 1 year ago

Hi @kierangraham, did you ever figure this out? Running into the same issue on Node 16x LTS.

kierangraham commented 1 year ago

Hey, I did not. I haven’t worked on a project where Camaro was used recently so I’m not sure about the current state at all unfortunately.

jermainemercado @.***> schrieb am Do. 22. Dez. 2022 um 19:23:

Hi @kierangraham https://github.com/kierangraham, did you ever figure this out? Running into the same issue on Node 16x LTS.

— Reply to this email directly, view it on GitHub https://github.com/tuananh/camaro/issues/113#issuecomment-1363209276, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAH4STOAS6HAV4W7RNMQNDWOSMDPANCNFSM4OBTFPQQ . You are receiving this because you were mentioned.Message ID: @.***>

tuananh / camaro

Memory allocation error #113