Webpack fails with "Cannot find module /home/USER/PROJECT/--max-old-space-size=2048"

[kpeters-cbsi]

Current behavior

Using the Serverless framework with Webpack. I type in sls deploy, which bundles my application with webpack, and I get:

Serverless: Bundling with Webpack...
Starting type checking service...
internal/modules/cjs/loader.js:983
  throw err;
  ^

Error: Cannot find module '/home/USER/PROJECT/--max-old-space-size=2048'
    at Function.Module._resolveFilename (internal/modules/cjs/loader.js:980:15)
    at Function.Module._resolveFilename (pkg/prelude/bootstrap.js:1346:46)
    at Function.Module._load (internal/modules/cjs/loader.js:862:27)
    at Function.Module.runMain (pkg/prelude/bootstrap.js:1375:12)
    at internal/main/run_main_module.js:18:47 {
  code: 'MODULE_NOT_FOUND',
  requireStack: []
}

Disabling the plugin in my webpack.config.js removes the problem

Expected behavior

Application would build successfully

Steps to reproduce the issue

In webpack.config.js, have the following under "plugins":

new ForkTsCheckerWebpackPlugin({
  eslint: true,
  eslintOptions: {
    cache: true
  }
})

Issue reproduction repository

https://github.com/PopeFelix/typescript-serverless

Environment

• fork-ts-checker-webpack-plugin: ^4.1.3
• typescript: 3.2.4
• eslint: 6.8.0
• webpack: 4.29.0
• os: Ubuntu 18.04.2

[piotr-oles]

Hi! Do you have the same error for the sls webpack command? I didn't want to set up AWS credentials to test it.

I didn't have this issue for sls webpack commend so I'm not sure if there is some difference between these commands or its because of the environment.

[kpeters-cbsi]

I don't have an sls webpack command.

Serverless command "webpack" not found. Did you mean "deploy"? Run "serverless help" for a list of all available commands.

[piotr-oles]

Ok :) I assume that sls command is from the https://www.npmjs.com/package/serverless package. I assume also that you have installed this package globally. Could you tell me which version of this package do you use?

[kpeters-cbsi]

Serverless Framework 1.71.3, webpack 4.43.0

On Mon, May 25, 2020 at 1:38 AM Piotr Oleś notifications@github.com wrote:

Ok :) I assume that sls command is from the https://www.npmjs.com/package/serverless package. I assume also that you have installed this package globally. Could you tell me which version of this package do you use? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub , or unsubscribe .

[piotr-oles]

Ah, sorry, my mistake. I meant sls package command 😄

[kpeters-cbsi]

The whole thing is attached, but the first several lines are:

 sls package
Serverless: Bundling with Webpack...
Starting type checking service...
Using 1 worker with 2048MB memory limit
internal/modules/cjs/loader.js:983
  throw err;
  ^

Error: Cannot find module '/home/kit/work/serverless/fantasy-log-pipeline/api/--max-old-space-size=2048'
    at Function.Module._resolveFilename (internal/modules/cjs/loader.js:980:15)
    at Function.Module._resolveFilename (pkg/prelude/bootstrap.js:1346:46)
    at Function.Module._load (internal/modules/cjs/loader.js:862:27)
    at Function.Module.runMain (pkg/prelude/bootstrap.js:1375:12)
    at internal/main/run_main_module.js:18:47 {
  code: 'MODULE_NOT_FOUND',
  requireStack: []
}
Time: 4887ms
Built at: 05/26/2020 10:15:23 AM
                            Asset      Size  Chunks         Chunk Names
    src/dispatchKinesisRecords.js   407 KiB       0         src/dispatchKinesisRecords
src/dispatchKinesisRecords.js.map  1.83 MiB       0  [dev]  src/dispatchKinesisRecords
Entrypoint src/dispatchKinesisRecords = src/dispatchKinesisRecords.js src/dispatchKinesisRecords.js.map
  [0] ../node_modules/webpack/buildin/module.js 497 bytes {0} [built]
  [2] external "util" 42 bytes {0} [built]
  [3] ../node_modules/jayschema/lib/errors.js 3.32 KiB {0} [built]
  [4] external "zlib" 42 bytes {0} [built]
  [5] ../node_modules/firehoser/firehoser.js 7.51 KiB {0} [built]
  [6] ../node_modules/node-sql-parser/index.js 216 bytes {0} [built]
  [7] ../node_modules/lodash/lodash.js 528 KiB {0} [built]
  [8] ../node_modules/firehoser/node_modules/aws-sdk/lib/aws.js 159 bytes {0} [built]
  [9] ../node_modules/async/dist/async.js 181 KiB {0} [built]
 [10] ../node_modules/jayschema/lib/jayschema.js 10.4 KiB {0} [built]
 [11] external "crypto" 42 bytes {0} [built]
 [12] ../node_modules/jayschema/lib/schemaRegistry.js 7.83 KiB {0} [built]
 [15] ../node_modules/jayschema/lib/httpLoader.js 1.91 KiB {0} [built]
 [18] ../node_modules/moment/moment.js 170 KiB {0} [built]
[152] ./src/dispatchKinesisRecords.ts + 1 modules 7.4 KiB {0} [built]
      | ./src/dispatchKinesisRecords.ts 4.25 KiB [built]
      | ./src/lib/formatter/rds.ts 3.13 KiB [built]
    + 138 hidden modules

sls_package.out.txt

[piotr-oles]

That's weird. Could you try with the fork-ts-checker-webpack-plugin@alpha?

[kpeters-cbsi]

With fork-ts-checker-webpack-plugin@alpha:

$ sls package
Serverless: Bundling with Webpack...
internal/modules/cjs/loader.js:983
  throw err;
  ^

Error: Cannot find module '/home/kit/work/serverless/fantasy-log-pipeline/api/--max-old-space-size=2048'
    at Function.Module._resolveFilename (internal/modules/cjs/loader.js:980:15)
    at Function.Module._resolveFilename (pkg/prelude/bootstrap.js:1346:46)
    at Function.Module._load (internal/modules/cjs/loader.js:862:27)
    at Function.Module.runMain (pkg/prelude/bootstrap.js:1375:12)
    at internal/main/run_main_module.js:18:47 {
  code: 'MODULE_NOT_FOUND',
  requireStack: []
}
RpcIpcMessagePortClosedError: Cannot send the message - the message port has been closed for the process 5762.
Issues checking service aborted - probably out of memory. Check the `memoryLimit` option in the ForkTsCheckerWebpackPlugin configuration.
If increasing the memory doesn't solve the issue, it's most probably a bug in the TypeScript or EsLint.
Time: 784ms

[piotr-oles]

Ok... Would it be possible to create a Dockerfile or image to reproduce this issue? It works on my machine, so it seems like an issue with the environment or different configuration of the serverless framework.

[piotr-oles]

Will you be able to provide such a reproduction environment? :)

[kpeters-cbsi]

Not for a little while, I'm afraid :)

On Mon, Jun 8, 2020 at 2:25 PM Piotr Oleś notifications@github.com wrote:

Will you be able to provide such a reproduction environment? :) — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub , or unsubscribe .

[piotr-oles]

Ok, I will close this issue for now. We can reopen it later :)

[yemi]

@piotr-oles Could we please reopen this as I'm having the exact same issue? I'm using Serverless framework

[piotr-oles]

No problem - I just need some reproduction environment to work on it or some help from your side to debug it :) I wasn't able to reproduce it on my machine :/

[hixus]

I have the same issue with global sls package. Although if I put sls as npm script and run npm run sls -- package there are no errors. Difference is:

sls --version Framework Core: 1.78.1 (standalone) Plugin: 3.7.0 SDK: 2.3.1 Components: 2.33.4

npm run sls -- --version

Framework Core: 1.78.1 Plugin: 3.7.0 SDK: 2.3.1 Components: 2.34.1

If I try to update the global one it still says "Serverless: Already at latest version" even though the local Components are newer

[piotr-oles]

@hixus Could you provide a reproduction repository?

[deldrid1]

I don't have too much helpful to add here, but I am running into the same issue (new to serveless, webpack, etc. - following https://github.com/mtimbs/typescript-serverless and tracing the bug down to here) on macOS.

I have tracked down this line as a place where I can alter the behavior (i.e. by changing the memory limit the log changes its value). https://github.com/TypeStrong/fork-ts-checker-webpack-plugin/blob/5ba683960e871d387685379d5a78943e4a5dc679/src/rpc/rpc-ipc/RpcIpcMessagePort.ts#L112

This gets packaged up as

        this.service = childProcess.fork(path.resolve(__dirname, this.workersNumber > 1 ? './cluster.js' : './service.js'), [], {
            env,
            execArgv: (this.workersNumber > 1
                ? []
                : ['--max-old-space-size=' + this.memoryLimit]).concat(this.nodeArgs),
            stdio: ['inherit', 'inherit', 'inherit', 'ipc']
        });

Looks very similar to https://github.com/vercel/pkg/issues/909.

[jnis23]

Had the same issue using standalone serverless binaries(1.80.0) and serverless-bundle(^3.0.0 using fork-ts-checker-webpack-plugin: ^4.0.1 ). Based on @hixus 's comment, I uninstalled the standalone version (originally installed via choco) and switched to the npm package npm install serverless -g and sure enough, that fixed it.

[deldrid1]

I had installed standalone on macOS. Moving to the npm version fixed this for me as well.

[kpeters-cbsi]

Installing Serverless Framework via yarn global add serverless fixed the problem for me.

[gcphost]

Happening to me today using serverless 2.0 installed locally and globally. with global sls commands it fails, with the command in package.json it works (as with @hixus).

Using macos. Node 12.

Thanks.

[davidboom95]

Does not work for me even with local sls execution. I'm also using serverless-bundle

Serverless: Running "serverless" installed locally (in service node_modules)
^R
Serverless: DOTENV: Loading environment variables from :
Serverless: Bundling with Webpack...
Starting type checking service...
internal/modules/cjs/loader.js:967
  throw err;
  ^

Error: Cannot find module '/Users/david/Documents/GitHub/my-project/--max-old-space-size=2048'
    at Function.Module._resolveFilename (internal/modules/cjs/loader.js:964:15)
    at Function.Module._resolveFilename (pkg/prelude/bootstrap.js:1346:46)
    at Function.Module._load (internal/modules/cjs/loader.js:840:27)
    at Function.Module.runMain (pkg/prelude/bootstrap.js:1375:12)
    at internal/main/run_main_module.js:17:47 {
  code: 'MODULE_NOT_FOUND',
  requireStack: []
}```

[piotr-oles]

I really would like to help and solve this issue, but I can't reproduce it. If someone could create a docker file that reproduces it, I would be grateful :)

[davidboom95]

I cannot make a docker container to reproduce it :( But maybe a clue is that --max-old-space-size=2048 is a flag and is being appended as if its a directory name. Maybe a space is missing somewhere?

[piotr-oles]

This plugin runs a separate process for the type check with this flag. It uses childProcess.fork which should not affect the main process and anything else. In the child process, we have only typescript - no webpack. I have no idea why it happens but it might be a bug in serverless or node.

[jppereyra]

@piotr-oles I'm experiencing the same issues described above. Here's a somewhat minimal repo that reproduces the issue: https://github.com/jppereyra/webpack-fails

[feliperugai]

Any thoughts on this? Getting the same problem, my project is just like @jppereyra's.

Edit: Just uninstalled the standalone version and installed via yarn make it work again. Thumbs up for those comments.

[gcphost]

I just run mine with npx sls to avoid this issue.

[nubunto]

Could be related to serverless being installed as a dependency instead of a devDependency. Also, I'm having zero issues running sls globally under NodeJS 12.8.1, but a colleague ran into this with NodeJS 14.

[maximgeerinck]

Had the exact same issue all of a sudden.

I was using serverless-bundle and used the following to fix it:

custom:
  bundle:
    disableForkTsChecker: true

as per https://www.npmjs.com/package/serverless-bundle

By default serverless-bundle uses the ForkTsCheckerWebpackPlugin to speed up builds by running type checking in a separate process. However, this combined with Serverless Framework's package: individually: true option means that to packages each Lambda function, a separate type checking process is started. Concurrently, starting many such processes can cause your build process to run out of memory.

My serverless is added as a devDependency

also had to set memory limit to 4gb (memory hog nodejs)

NODE_OPTIONS=--max_old_space_size=4096 yarn deploy

[gcphost]

I just tried your settings @maximgeerinck to no avail. Also have sls as a dev dep, disabled ts checker in bundle, upped limit, and it still errors out @ 2048.

[maximgeerinck]

I just tried your settings @maximgeerinck to no avail. Also have sls as a dev dep, disabled ts checker in bundle, upped limit, and it still errors out @ 2048.

Are you running it in a CI? I had it again in a different monorepo that had more lambdas. The only way i could fix this was by using the large resource class (8gb) and using NODE_OPTIONS=--max_old_space_size=8196 yarn deploy

[gcphost]

I run it in a CI/D on AWS Code Deploy and no issues, I only have the problem locally.

[thelpbogeyman]

I just run mine with npx sls to avoid this issue.

This worked for me too.

[vertgo]

I don't have too much helpful to add here, but I am running into the same issue (new to serveless, webpack, etc. - following https://github.com/mtimbs/typescript-serverless and tracing the bug down to here) on macOS.

I have tracked down this line as a place where I can alter the behavior (i.e. by changing the memory limit the log changes its value).

https://github.com/TypeStrong/fork-ts-checker-webpack-plugin/blob/5ba683960e871d387685379d5a78943e4a5dc679/src/rpc/rpc-ipc/RpcIpcMessagePort.ts#L112

This gets packaged up as

        this.service = childProcess.fork(path.resolve(__dirname, this.workersNumber > 1 ? './cluster.js' : './service.js'), [], {
            env,
            execArgv: (this.workersNumber > 1
                ? []
                : ['--max-old-space-size=' + this.memoryLimit]).concat(this.nodeArgs),
            stdio: ['inherit', 'inherit', 'inherit', 'ipc']
        });

Looks very similar to vercel/pkg#909.

In a commit that fixes the problem in pkg, they change the way the argument is called. It seems that the splicing of the argv[] is somehow wrong, and so it passes the argument (max_old_space_size) in the wrong place, treating it like a module rather than argument. (or the first argument rather than the second?).

https://github.com/bergheim/pkg/commit/9c6dc845e4dae69cc0a58ed557f1057f6533e9cf

this might be fixable by changing the way the argvs are handled in this line: https://github.com/TypeStrong/fork-ts-checker-webpack-plugin/blob/5ba683960e871d387685379d5a78943e4a5dc679/src/rpc/rpc-ipc/RpcIpcMessagePort.ts#L112

[piotr-oles]

hmm... the comment that you cite is not correct, because RpcIpcMessagePort.ts file is from version >= 5, but

this.service = childProcess.fork(path.resolve(__dirname, this.workersNumber > 1 ? './cluster.js' : './service.js'), [], {
            env,
            execArgv: (this.workersNumber > 1
                ? []
                : ['--max-old-space-size=' + this.memoryLimit]).concat(this.nodeArgs),
            stdio: ['inherit', 'inherit', 'inherit', 'ipc']
        });

is a code from version 4. If serverless framework messes up with exec arguments, then I think it should be fixed there - not in this plugin. And based on the comments, it seems that the issue was fixed in newer versions of serverless so I would suggest upgrading :)

[vertgo]

I can look deeper into it, but the serverless binary that was installed was just installed the day before I dropped that comment, across 2 different platforms (mac and linux). The workarounds mentioned here works (installing from npm or yarn rather than installing according to serverless's method).

to reproduce, one could install serverless via command line according to their getting started page curl -o- -L https://slss.io/install | bash

There isn't an uninstall script, but deleting ~/.serverless before installing via npm helped me. I think the method the arguments are handled via node or yarn's installation doesn't cause the same error as installing it via the binary. (reading through the install shell script, it downloads the latest from here )

[piotr-oles]

So it seems like a bug in their binary - something to report in their repo. It may be a bug in a tool that builds this binary, or directly in their code.

[VictorMachimana]

I have experienced the same thing, there were conflicting versions of serverless installed in two different locations. to figure out if you have the same issue run whereis serverless, if you are using nvm, remove the other one (~/.serverless/bin).