PM2 is not restarting the simplest crashing app possible

dzek69 commented 3 years ago

What's going wrong?

PM2 by default should restart my app from what I know. However it doesn't restart the app if error is thrown in top level code, like:

file.js

throw new Error("boom");

How could we reproduce this issue?

Create file as above. Run pm2 start file.js Use pm2 list to see that app is not restarting and it's "online"

Supporting information

I first discovered this on my Raspberry, i thought it may be some issue with the OS, Node version or something, but I tested it on my local PC and it's the same. Even old version (4.4.1) is behaving the same, so this must be a long living bug I guess.

I tried changing the code to:

setTimeout(() => {
    throw new Error("g");
}, 100);

and then it works as expected

$ pm2 report

// ON RASPBERRY:
--- PM2 report ----------------------------------------------------------------
Date                 : Mon Aug 30 2021 19:44:23 GMT+0200 (czas środkowoeuropejski letni)
===============================================================================
--- Daemon -------------------------------------------------
pm2d version         : 5.1.1
node version         : 15.9.0
node path            : /home/dzek/.yarn/bin/pm2
argv                 : /home/dzek/.local/node-v15.9.0-linux-armv6l/bin/node,/home/dzek/.config/yarn/global/node_modules/pm2/lib/Daemon.js
argv0                : node
user                 : dzek
uid                  : 1001
gid                  : 1001
uptime               : 27min
===============================================================================
--- CLI ----------------------------------------------------
local pm2            : 5.1.1
node version         : 15.9.0
node path            : /home/dzek/.yarn/bin/pm2
argv                 : /home/dzek/.local/node-v15.9.0-linux-armv6l/bin/node,/home/dzek/.yarn/bin/pm2,report
argv0                : node
user                 : dzek
uid                  : 1001
gid                  : 1001
===============================================================================
--- System info --------------------------------------------
arch                 : arm
platform             : linux
type                 : Linux
cpus                 : ARMv6-compatible processor rev 7 (v6l)
cpus nb              : 1
freemem              : 120193024
totalmem             : 450981888
home                 : /home/dzek
===============================================================================
--- PM2 list -----------------------------------------------
┌────┬────────────────────┬──────────┬──────┬───────────┬──────────┬──────────┐
│ id │ name               │ mode     │ ↺    │ status    │ cpu      │ memory   │
├────┼────────────────────┼──────────┼──────┼───────────┼──────────┼──────────┤
│ 0  │ index              │ fork     │ 0    │ online    │ 2.9%     │ 40.5mb   │
└────┴────────────────────┴──────────┴──────┴───────────┴──────────┴──────────┘
===============================================================================
--- Daemon logs --------------------------------------------
/home/dzek/.pm2/pm2.log last 20 lines:
PM2        | 2021-08-30T19:16:33: PM2 log: ===============================================================================
PM2        | 2021-08-30T19:16:33: PM2 log: --- New PM2 Daemon started ----------------------------------------------------
PM2        | 2021-08-30T19:16:33: PM2 log: Time                 : Mon Aug 30 2021 19:16:33 GMT+0200 (czas środkowoeuropejski letni)
PM2        | 2021-08-30T19:16:33: PM2 log: PM2 version          : 5.1.1
PM2        | 2021-08-30T19:16:33: PM2 log: Node.js version      : 15.9.0
PM2        | 2021-08-30T19:16:33: PM2 log: Current arch         : arm
PM2        | 2021-08-30T19:16:33: PM2 log: PM2 home             : /home/dzek/.pm2
PM2        | 2021-08-30T19:16:33: PM2 log: PM2 PID file         : /home/dzek/.pm2/pm2.pid
PM2        | 2021-08-30T19:16:33: PM2 log: RPC socket file      : /home/dzek/.pm2/rpc.sock
PM2        | 2021-08-30T19:16:33: PM2 log: BUS socket file      : /home/dzek/.pm2/pub.sock
PM2        | 2021-08-30T19:16:33: PM2 log: Application log path : /home/dzek/.pm2/logs
PM2        | 2021-08-30T19:16:33: PM2 log: Worker Interval      : 30000
PM2        | 2021-08-30T19:16:33: PM2 log: Process dump file    : /home/dzek/.pm2/dump.pm2
PM2        | 2021-08-30T19:16:33: PM2 log: Concurrent actions   : 2
PM2        | 2021-08-30T19:16:33: PM2 log: SIGTERM timeout      : 1600
PM2        | 2021-08-30T19:16:33: PM2 log: ===============================================================================
PM2        | 2021-08-30T19:16:34: PM2 log: App [index:0] starting in -fork mode-
PM2        | 2021-08-30T19:16:34: PM2 log: App [index:0] online

$ pm2 report
// ON PC, with the "test fix" applied before generating report

--- PM2 report ----------------------------------------------------------------
Date                 : Mon Aug 30 2021 19:43:31 GMT+0200 (Central European Summer Time)
===============================================================================
--- Daemon -------------------------------------------------
pm2d version         : 4.4.1
node version         : 15.2.0
node path            : /home/dzek/.yarn/bin/pm2
argv                 : /usr/bin/node,/home/dzek/.config/yarn/global/node_modules/pm2/lib/Daemon.js
argv0                : node
user                 : dzek
uid                  : 1000
gid                  : 1000
uptime               : 4min
===============================================================================
--- CLI ----------------------------------------------------
local pm2            : 4.4.1
node version         : 15.2.0
node path            : /home/dzek/.yarn/bin/pm2
argv                 : /usr/bin/node,/home/dzek/.yarn/bin/pm2,report
argv0                : node
user                 : dzek
uid                  : 1000
gid                  : 1000
===============================================================================
--- System info --------------------------------------------
arch                 : x64
platform             : linux
type                 : Linux
cpus                 : Intel(R) Core(TM) i7-6700HQ CPU @ 2.60GHz
cpus nb              : 8
freemem              : 1936064512
totalmem             : 12561264640
home                 : /home/dzek
===============================================================================
--- PM2 list -----------------------------------------------
┌─────┬──────────┬─────────────┬─────────┬─────────┬──────────┬────────┬──────┬───────────┬──────────┬──────────┬──────────┬──────────┐
│ id  │ name     │ namespace   │ version │ mode    │ pid      │ uptime │ ↺    │ status    │ cpu      │ mem      │ user     │ watching │
├─────┼──────────┼─────────────┼─────────┼─────────┼──────────┼────────┼──────┼───────────┼──────────┼──────────┼──────────┼──────────┤
│ 0   │ index    │ default     │ 0.0.2   │ fork    │ 14599    │ 4m     │ 0    │ online    │ 0.2%     │ 50.7mb   │ dzek     │ disabled │
│ 1   │ test     │ default     │ 0.0.2   │ fork    │ 0        │ 0      │ 16   │ errored   │ 0%       │ 0b       │ dzek     │ disabled │
└─────┴──────────┴─────────────┴─────────┴─────────┴──────────┴────────┴──────┴───────────┴──────────┴──────────┴──────────┴──────────┘
===============================================================================
--- Daemon logs --------------------------------------------
/home/dzek/.pm2/pm2.log last 20 lines:
PM2        | 2021-08-30T19:42:37: PM2 log: App [test:1] exited with code [1] via signal [SIGINT]
PM2        | 2021-08-30T19:42:37: PM2 log: App [test:1] starting in -fork mode-
PM2        | 2021-08-30T19:42:37: PM2 log: App [test:1] online
PM2        | 2021-08-30T19:42:37: PM2 log: App [test:1] exited with code [1] via signal [SIGINT]
PM2        | 2021-08-30T19:42:37: PM2 log: App [test:1] starting in -fork mode-
PM2        | 2021-08-30T19:42:37: PM2 log: App [test:1] online
PM2        | 2021-08-30T19:42:37: PM2 log: App [test:1] exited with code [1] via signal [SIGINT]
PM2        | 2021-08-30T19:42:37: PM2 log: App [test:1] starting in -fork mode-
PM2        | 2021-08-30T19:42:37: PM2 log: App [test:1] online
PM2        | 2021-08-30T19:42:38: PM2 log: App [test:1] exited with code [1] via signal [SIGINT]
PM2        | 2021-08-30T19:42:38: PM2 log: App [test:1] starting in -fork mode-
PM2        | 2021-08-30T19:42:38: PM2 log: App [test:1] online
PM2        | 2021-08-30T19:42:38: PM2 log: App [test:1] exited with code [1] via signal [SIGINT]
PM2        | 2021-08-30T19:42:38: PM2 log: App [test:1] starting in -fork mode-
PM2        | 2021-08-30T19:42:38: PM2 log: App [test:1] online
PM2        | 2021-08-30T19:42:38: PM2 log: App [test:1] exited with code [1] via signal [SIGINT]
PM2        | 2021-08-30T19:42:38: PM2 log: App [test:1] starting in -fork mode-
PM2        | 2021-08-30T19:42:38: PM2 log: App [test:1] online
PM2        | 2021-08-30T19:42:39: PM2 log: App [test:1] exited with code [1] via signal [SIGINT]
PM2        | 2021-08-30T19:42:39: PM2 log: Script /Projects/o-0/services/doorbell/app/dist/test.js had too many unstable restarts (16). Stopped. "errored"

cwagner22 commented 2 years ago

In my case, switching from running the script with yarn to npm worked.

dzek69 commented 2 years ago

@cwagner22 you mean installing pm2 globally with npm instead of yarn?

cwagner22 commented 2 years ago

I use pm2 to run a command inside my package.json. pm2 start yarn --name test -- start

I changed to: pm2 start npm --name test -- start

Maybe it doesn't apply to you.

dzek69 commented 2 years ago

i don't do that, as I said i run the app with pm2 start file.js

polaroidkidd commented 2 years ago

In my case, switching from running the script with yarn to npm worked.

It appears that this only works if you're not using pm2 in yarn workspaces

VichetN commented 7 months ago

Does anybody solve the problem? i am having the same issue.

MuslemRahimi commented 7 months ago

same problem here. My fastify backend crashes and pm2 decides not to do anything

Unitech / pm2