ChildProcessProxy OutOfMemoryError test fails

[Tamir-M]

Summary

When running a fresh clone of the repository on the master branch it seems as if the ChildProcessProxy test for should throw an OutOfMemoryError if the process went out-of-memory: fails.

To recreate:

• clone the project
• run npm install
• run npm run build
• run npm test

The error of the failing test exactly:

@stryker-mutator/core:   61 passing (23s)
@stryker-mutator/core:   1 failing
@stryker-mutator/core:   1) ChildProcessProxy
@stryker-mutator/core:        should throw an OutOfMemoryError if the process went out-of-memory:
@stryker-mutator/core:       AssertionError: expected promise to be rejected with 'OutOfMemoryError' but it was rejected with 'Error: Child process [pid 12133] exit…'
@stryker-mutator/core:       + expected - actual
@stryker-mutator/core:       -Error: Child process [pid 12133] exited unexpectedly with exit code null (SIGABRT). Last part of stdout and stderr was:
@stryker-mutator/core:       -       26: 0x1005c3104 node::AsyncWrap::MakeCallback(v8::Local<v8::Function>, int, v8::Local<v8::Value>*) [/Users/tamir/.local/share/fnm/node-versions/v20.13.1/installation/bin/node]
@stryker-mutator/core:       -       27: 0x10074c60c node::StreamBase::CallJSOnreadMethod(long, v8::Local<v8::ArrayBuffer>, unsigned long, node::StreamBase::StreamBaseJSChecks) [/Users/tamir/.local/share/fnm/node-versions/v20.13.1/installation/bin/node]
@stryker-mutator/core:       -       28: 0x10074dc98 node::EmitToJSStreamListener::OnStreamRead(long, uv_buf_t const&) [/Users/tamir/.local/share/fnm/node-versions/v20.13.1/installation/bin/node]
@stryker-mutator/core:       -       29: 0x100751f84 node::LibuvStreamWrap::OnUvRead(long, uv_buf_t const*) [/Users/tamir/.local/share/fnm/node-versions/v20.13.1/installation/bin/node]
@stryker-mutator/core:       -       30: 0x1007526b8 node::LibuvStreamWrap::ReadStart()::$_1::__invoke(uv_stream_s*, long, uv_buf_t const*) [/Users/tamir/.local/share/fnm/node-versions/v20.13.1/installation/bin/node]
@stryker-mutator/core:       -       31: 0x10105db4c uv__stream_io [/Users/tamir/.local/share/fnm/node-versions/v20.13.1/installation/bin/node]
@stryker-mutator/core:       -       32: 0x101065450 uv__io_poll [/Users/tamir/.local/share/fnm/node-versions/v20.13.1/installation/bin/node]
@stryker-mutator/core:       -       33: 0x10105393c uv_run [/Users/tamir/.local/share/fnm/node-versions/v20.13.1/installation/bin/node]
@stryker-mutator/core:       -       34: 0x1005ad6f0 node::SpinEventLoopInternal(node::Environment*) [/Users/tamir/.local/share/fnm/node-versions/v20.13.1/installation/bin/node]
@stryker-mutator/core:       -       35: 0x1006c02e4 node::NodeMainInstance::Run(node::ExitCode*, node::Environment*) [/Users/tamir/.local/share/fnm/node-versions/v20.13.1/installation/bin/node]
@stryker-mutator/core:       -       36: 0x1006bfff8 node::NodeMainInstance::Run() [/Users/tamir/.local/share/fnm/node-versions/v20.13.1/installation/bin/node]
@stryker-mutator/core:       -       37: 0x1006487ac node::Start(int, char**) [/Users/tamir/.local/share/fnm/node-versions/v20.13.1/installation/bin/node]
@stryker-mutator/core:       -       38: 0x1804fa0e0 start [/usr/lib/dyld]
@stryker-mutator/core:       -       
@stryker-mutator/core:       +OutOfMemoryError
@stryker-mutator/core:       
@stryker-mutator/core:   
@stryker-mutator/core: 

After the coverage report it also says:

 Lerna (powered by Nx)   Running target test for 12 projects failed

Tasks not run because their dependencies failed or --nx-bail=true:

- @stryker-mutator/cucumber-runner:test
- @stryker-mutator/jasmine-runner:test
- @stryker-mutator/jest-runner:test
- @stryker-mutator/karma-runner:test
- @stryker-mutator/mocha-runner:test
- @stryker-mutator/tap-runner:test
- @stryker-mutator/typescript-checker:test
- @stryker-mutator/vitest-runner:test

Failed tasks:

- @stryker-mutator/core:test

Stryker config

not relevant.

Test runner config

Not relevant.

Stryker environment

Not relevant.

Test runner environment

using the npm test of the repository:

    "test": "lerna run test --stream --concurrency 2",

Your Environment

software	version(s)
node	v20.13.1
npm	10.5.2
Operating System	macOS Sonoma 14.5

Add stryker.log

Not relevant.

[nicojs]

Hi @Tamir-M 🙋‍♂️. It's awesome to see you're running our tests. I know that our implementation to detect if a process exited with an out-of-memory (OOM) or not is flaky. We're scraping the last output from the process to see if the text "JavaScript heap out of memory" or "FatalProcessOutOfMemory" is present.

This is very flaky:

• Sometimes the stderr buffer isn't fully flushed, and we didn't get the output.
• Sometimes, the stderr output is flushed, but more output comes after, and our round-robin approach to log capturing already flushed this relevant text.
• Or you might have a different language of nodejs installed, and instead of "JavaScript heap out of memory" it says "JavaScript heap had geen geheugen meer".

Can one of these reasons be what causes this error to fail? If so, can you think of a better way to detect OOM errors?

We handle out-of-memory errors the same way as other unexpected process crashes, so it isn't that big of a deal. However, when we detect an OOM, we log a specific warning, telling the user to look for memory leaks in their tests. See https://github.com/stryker-mutator/stryker-js/blob/chore%2Ffix-audit/packages/core/src/test-runner/retry-rejected-decorator.ts#L53