search

emscripten-core / emscripten

Emscripten: An LLVM-to-WebAssembly Compiler
Other
25.82k stars 3.31k forks source link

posixtestsuite hangs whole Windows OS when run on system with large number of cores #13267

Open juj opened 3 years ago

juj commented 3 years ago

Trying to run posixtestsuite on Windows to figure out why PR #12833 does not pass there, I notice that the suite spawns #numcores threads for each #numcores thread on the system, leading to 64*64=4096 python subprocesses running on my Threadripper system. This "fork bomb" causes Windows OS to become unstable.

Sometimes taskkill /f /im python.exe is able to recover, other times it can partially recover, and sometimes it just hangs as well. All other processes become unresponsive, and one has to hard reset the system.

sbc100 commented 3 years ago

Interesting. I wonder why I've not see this on linux.

It makes sense that it would start 64 threads so it can run one test on each code. I can't think why it would then create 64 threads within each of those. Can you tell if the second level of nesting is caused by:

a. the test runner. (runner.py) b. emcc.py compiling system libraries in parallel c. The individual tests in posixtestsuite forking worker

(The latter would show up as node/browser processes of course).

You can debug this without bringing down you machine by setting EMCC_CORES to a reasonable number (e.g. EMCC_CORES=4).

Are you running the browser version (posixtest_browser) or the node version (posixtest)?

juj commented 3 years ago

Did not have time to look too much into it, but did verify that EMCC_CORES=1 did avoid the issue.

juj commented 3 years ago

I was running the node version python tests/runner.py posixtest

stale[bot] commented 2 years ago

This issue has been automatically marked as stale because there has been no activity in the past year. It will be closed automatically if no further activity occurs in the next 30 days. Feel free to re-open at any time if this issue is still relevant.