search

timgit / pg-boss

Queueing jobs in Postgres from Node.js like a boss
MIT License
2.15k stars 160 forks source link

Segfault in Nest JS tests #352

Closed kristijanhusak closed 2 years ago

kristijanhusak commented 2 years ago

Hi,

Yesterday I installed this plugin on my project from work that uses Nest JS. Everything was working fine while manually testing. Once I tried to run my automatic tests with Jest, I started getting segfaults (exit code 139). I tried figuring out what is segfaulting, by using something like https://github.com/Shiranuit/node-segfault-handler, but it didn't print anything.

Since it's tricky to reproduce, I set up a sample Nest JS project, with docker + docker-compose to spin up everything and reproduce. Here are the steps:

Setup:

  1. Make sure you have docker and docker-compose on your machine
  2. git clone https://github.com/kristijanhusak/pg-boss-queue-test && cd pg-boss-queue-test
  3. docker-compose up

Manual test to ensure it works: In your browser, open http://localhost:3000/queue/some-message. The response should be uuid and the server console where you ran docker-compose up should print { message: 'some-message' }

Run tests to cause segfault:

  1. docker-compose app npm run test:e2e
  2. Tests stops in the middle
  3. echo $? returns 139

To make tests pass, go to src/queues/queues.module.ts and comment out everything from onModuleInit method where the queue is being started.

Few more things that I tried but it didn't work:

Note that only start() is sufficient to cause a segfault, commenting out work() doesn't change anything.

Edit: Also tried setting noScheduling and noSupervisor to true, but that also didn't help.

kristijanhusak commented 2 years ago

It might be caused by this https://github.com/nodejs/node/issues/35889

kristijanhusak commented 2 years ago

After some debugging I can confirm it's the problem with https://github.com/nodejs/node/issues/35889.

I trier replacing serializeError with JSON.stringify, and replacing pMap with Promise.all, just to remove all dynamic imports. Tests passed without segfaulting. Replacement for pMap could be Bluebird, since I think it supports common js (it's a bit bigger dependency though), but I'm no sure about serialize-error.

timgit commented 2 years ago

I think it's easier to just downgrade to common js versions of the same package

timgit commented 2 years ago

Just released 8.2.0 with common js downgrades. Give that a try

kristijanhusak commented 2 years ago

That fixes the issue, thanks!