node-test-commit-arm compiler ICE debian9-docker-armv7

[lundibundi]

Recently I've seen quite a few GCC ICE on that configuration. Can someone look into it?

https://github.com/nodejs/node/pull/30666 https://ci.nodejs.org/job/node-test-commit-arm/nodes=debian9-docker-armv7/27678/console

22:44:49 ../src/node_native_module_env.cc: In static member function 'static void node::native_module::NativeModuleEnv::GetModuleCategories(v8::Local<v8::Name>, const v8::PropertyCallbackInfo<v8::Value>&)':
22:44:49 ../src/node_native_module_env.cc:75:18: internal compiler error: Segmentation fault
22:44:49        .FromJust();
22:44:49                   ^
22:44:49 Please submit a full bug report,
22:44:49 with preprocessed source if appropriate.
22:44:49 See <file:///usr/share/doc/gcc-6/README.Bugs> for instructions.
22:44:49 libnode.target.mk:330: recipe for target '/home/iojs/build/workspace/node-test-commit-arm/nodes/debian9-docker-armv7/out/Release/obj.target/libnode/src/node_native_module_env.o' failed

https://github.com/nodejs/node/pull/30613 (3 ICE in a row) https://ci.nodejs.org/job/node-test-commit-arm/27602/nodes=debian9-docker-armv7/console https://ci.nodejs.org/job/node-test-commit-arm/nodes=debian9-docker-armv7/27606/console https://ci.nodejs.org/job/node-test-commit-arm/nodes=debian9-docker-armv7/27612/console (I've copy-pasted the ICE lines from logs to the PR)

@addaleax / @bnoordhuis suggested that the machine may be lacking memory, could we somehow check the load logs?

@lundibundi Somewhere @bnoordhuis suggested that those ICEs might be the result of an out-of-memory situation … I think it’s a machine that doesn’t have a ton of memory, so that might make sense?

[rvagg]

and https://github.com/nodejs/node/pull/30232#issuecomment-559074074 / https://ci.nodejs.org/job/node-test-commit-arm/27691/nodes=debian9-docker-armv7/, same machine

I can't see anything special with this machine. They've all got 20G of memory with no swap and they do run with JOBS=4. So I guess if we're doing big chunks of work in our compiles they might have trouble. Is new stuff in V8 starting to push boundaries for us in some special way? Do we need to reduce parallelism or introduce a swap file?

[addaleax]

That doesn’t really sound like an OOM then, and more like a real compiler bug… :disappointed:

Any chance the GCC on that machine is out of date/can be updated?

[rvagg]

there was a job on test-scaleway-ubuntu1804-armv7l-3 nearly two days old, right near the completion of the job, so we're hitting a bottleneck yet again. I've taken node-test-commit-arm offline and will do a full refresh of those machines. The Debain Stretch images are using gcc-6 but there may be updates. It wasn't super long ago that these machines were updated though.

[rvagg]

I can't read numbers, they have 2G, not 20.. they're 32-bit systems so 20 wouldn't make sense anyway.

I'm reducing JOBS to 2 from 4 and have put a 4G swap file on each of them. They're all doing a master build now to prime the cache but node-test-commit-arm is reenabled.