Remix serve crashes with pm2 in cluster mode

[prudhvi85]

Reproduction

start pm2 with the following config.

module.exports = { apps: [ { name: "remix-app", script: "remix-serve", args: "build/index.js", instances: 'max', exec_mode: 'cluster', autorestart: true, restart: 'on-failure', env: { NODE_ENV: "production" } } ] };

System Info

System:
    OS: Linux 6.5 Alpine Linux
    CPU: (4) x64 AMD EPYC 7J13 64-Core Processor
    Memory: 2.49 GB / 3.81 GB
    Container: Yes
    Shell: 1.36.0 - /bin/ash
  Binaries:
    Node: 19.9.0 - /usr/local/bin/node
    Yarn: 1.22.19 - /usr/local/bin/yarn
    npm: 9.6.3 - /usr/local/bin/npm
  npmPackages:
    @remix-run/css-bundle: ^2.8.1 => 2.9.2
    @remix-run/dev: ^2.8.1 => 2.9.2
    @remix-run/eslint-config: ^2.8.1 => 2.9.2
    @remix-run/node: ^2.8.1 => 2.9.2
    @remix-run/react: ^2.8.1 => 2.9.2
    @remix-run/serve: ^2.8.1 => 2.9.2

Used Package Manager

yarn

Expected Behavior

Start serving remix application with one process per CPU.

Actual Behavior

Application doesnot with with pm2 in cluster mode. it works fine in fork mode.

It crashed with following error:

2|remix-ap | You have triggered an unhandledRejection, you may have forgotten to catch a Promise rejection: 2|remix-ap | TypeError: Cannot read properties of undefined (reading 'unstable_singleFetch') 2|remix-ap | at run (/usr/local/share/.config/yarn/global/node_modules/@remix-run/serve/dist/cli.js:114:31)

[cayter]

Refer to this. The snippets should be added to the very start of the server entry file.

[ngbrown]

@cayter, while I agree that those snippets are useful, I wonder about why they would be needed to swallow an error right at server start up?

I have them on my server.js file after server.listen because I want the process to crash out if there is something obviously wrong. It seems like if the service is failing even before the first request, there is something wrong that should be fixed.

The error shows as having a problem on this line:

https://github.com/remix-run/remix/blob/ff06e1656108bc21244e1fd4b33ed53e22b85158/packages/remix-serve/cli.ts#L103

So why would build.future be undefined?

reimportServer() is clearing require.cache even in non-dev mode, so maybe that is not compatible with how pm2 interacts with node.js's Cluster module?

Try modifying cli.js to skip the cache clearing in production mode and see if that helps; record the change with using patch-package. You could also try using the express template as its server.js file doesn't clear the cache when not in development mode.

[cayter]

The snippet makes sure that your nodejs process doesn't crash in case you have code that doesn't handle error properly.

As for fixing the error you are facing, it's hard to tell what's wrong without seeing a minimal reproducible repo.

[prudhvi85]

@cayter this snippets did not solve the problem.

I added code in the shared snippet in entry.server.tsx and the issue still persists

[cayter]

@cayter this snippets did not solve the problem.

I added code in the shared snippet in entry.server.tsx and the issue still persists

Any chance you can show it in a minimal reproducible repo?

[prudhvi85]

@cayter Here it the repo - https://github.com/prudhvi85/debug-pm2-remix

Instruction for running: docker build -t test_image/test-image -f Dockerfile .

docker run -p 3120:3120 test_image/test-image

[prudhvi85]

@cayter apologies for bothering you. We are sort of stuck on this. Any help is really appreciated. Thank you

[cayter]

Root Cause

remix-serve is expecting 3rd argument as the build path from the terminal but PM2 executes in the terminal with the build path being 4th argument which leads to the imported build doesn't have any exported build.future.unstable_singleFetch defined because PM2 is misleading remix-serve to load the wrong server build.

[prudhvi-tg]

@cayter Thank you!! any solution to this issue?

Also, I am wondering how it works in fork but not in cluster mode.

[cayter]

Also, I am wondering how it works in fork but not in cluster mode.

Fork mode doesn't hijack the build path in 3rd argument that remix-serve is expecting.

any solution to this issue?

The cleanest approach is to avoid using pm2 config and pass the arguments instead. In addition, I would like to point out that if you're using Docker, there's no much points in using PM2. Can refer to this for the reasoning.

$ pm2-runtime start remix-serve -- build/server/index.js --name remix-app --instances 1 --exec_mode cluster --env NODE_ENV=production --restart_on_failure true

[cayter]

@prudhvi-tg I assume the issue is resolved? If yes, please close it. Thanks.

[prudhvi85]

@cayter Tried passing the arguments instead of config. Now I am getting another issue

You have triggered an unhandledRejection, you may have forgotten to catch a Promise rejection:
Error: ENOENT: no such file or directory, stat '/app/start'
 at Object.statSync (node:fs:1658:25)
at reimportServer (/usr/local/share/.config/yarn/global/node_modules/@remix-run/serve/dist/cli.js:83:39)
at run (/usr/local/share/.config/yarn/global/node_modules/@remix-run/serve/dist/cli.js:112:21)
at Object.<anonymous> (/usr/local/share/.config/yarn/global/node_modules/@remix-run/serve/dist/cli.js:58:1)
at Module._compile (node:internal/modules/cjs/loader:1358:14)
 at Object.Module._extensions..js (node:internal/modules/cjs/loader:1416:10)
  at Module.load (node:internal/modules/cjs/loader:1208:32)
  at Function.Module._load (node:internal/modules/cjs/loader:1024:12)
 at /usr/local/share/.config/yarn/global/node_modules/pm2/lib/ProcessContainer.js:304:25
  at wrapper (/usr/local/share/.config/yarn/global/node_modules/pm2/node_modules/async/internal/once.js:12:16

[prudhvi85]

@cayter I am trying to use pm2 for load balancing features