search

dherault / serverless-offline

Emulate AWS λ and API Gateway locally when developing your Serverless project
MIT License
5.16k stars 794 forks source link

free(): invalid next size (fast), Segmentation Faults (SIGSEGV) and other native Node.js crashes #1762

Open aardvarkk opened 4 months ago

aardvarkk commented 4 months ago

Bug Report

Current Behavior

Serverless offline sporadically crashes with various memory-related errors. The crash is within Node's native code.

Expected behavior/code

Serverless offline should not crash.

Environment

Both myself (Linux) and my colleague on (Mac M1) have experienced several native crashes in the Node process. We're using the latest version of serverless-offline (13.3.3).

Possible Solution

Neither of us have experienced a crash yet running serverless-offline with the --useInProcess flag.

Additional context/Screenshots

I spent a couple of days trying to figure out if this was some bug in Node.js memory handling or something, because the backtraces from the core dumps just pointed to code within Node. Eventually I tried out this flag on serverless-offline and stability seems to have greatly improved.

I'm creating this bug report for two reasons:

  1. I wanted to make this issue so searching for these errors may bring somebody here and let them try to run with --useInProcess. It seems to have greatly helped the stability of the project. At first I thought maybe there was something wrong with our Node.js version. We tried upgrading to v20 from v18 (which didn't fix anything), and spent a lot of time searching around for other people having segfaults or memory issues with Node, which was not helpful.
  2. To recommend that perhaps a large warning or notice be placed in the docs to indicate that running in the default mode with worker threads may cause instability issues. I'm not sure which native extensions in our project are to blame, but it was remarkably hard to track down exactly what was happening when every now and again our servers would just crash seemingly at random.

I'm not sure if I expect anything to change within serverless-offline, but it would have been really helpful to have an issue like this to look into to help figure out why our servers were crashing.

igo-alt commented 1 month ago

@aardvarkk My team is having this problem too, the flag --useInProcess solved the problem, but I don't know what side effects we may have using this flag.

DorianMazur commented 1 month ago

What node version are you using? @igo-alt @aardvarkk

aardvarkk commented 1 month ago

@DorianMazur Both v18 and v20

igo-alt commented 1 month ago

@DorianMazur 20, LTS

codyaweber commented 4 weeks ago

Likewise; node 20.11.1, serverless 3.38.0, M2 Macbook Pro. Adding --useInProcess flag on serverless offline also seems to prevent the issue in my case.