search

rumpkernel / rumprun-packages

Ready-made packages of software for running on the Rumprun unikernel
Other
202 stars 80 forks source link

Compiling NodeJS memory leak #119

Open miha-plesko opened 7 years ago

miha-plesko commented 7 years ago

So I'm trying to run this NodeJS application with Rumprun. I've git-cloned it to the rumprun-packages/nodejs/examples and run following commands:

zip -r ./word-finder.zip word-finder
./zipload.sh /word-finder/start.js < ./word-finder.zip > aplikacija_word-finder.js
ln -sf examples/aplikacija_word-finder.js ../rumpmain.js
make -C ..   # <---------------- CONSUMES ALL THE MEMORY (14GB)

as suggested in NodeJS readme. BUT the make command fails after some time saying:

virtual memory exhausted: Cannot allocate memory
node.target.mk:151: recipe for target '/.../rumprun-packages/nodejs/build-4.3.0/out/Release/obj.target/node/src/node_javascript.o' failed
make[2]: *** [/.../rumprun-packages/nodejs/build-4.3.0/out/Release/obj.target/node/src/node_javascript.o] Error 1
make[2]: Leaving directory '/.../rumprun-packages/nodejs/build-4.3.0/out'
Makefile:45: recipe for target 'node' failed
make[1]: *** [node] Error 2
make[1]: Leaving directory '/.../rumprun-packages/nodejs/build-4.3.0'
Makefile:35: recipe for target 'build-4.3.0/out/Release/node' failed
make: *** [build-4.3.0/out/Release/node] Error 2
make: Leaving directory '/.../rumprun-packages/nodejs'

I run this on a machine with 10GB RAM plus 4GB swap. With htop I can see that all the memmory is consumed indeed. Now, the NodeJS application is around 100MB big. Why would make consume 14GB of memory? :smile:

miha-plesko commented 7 years ago

I went crazy and used 14GB of memory + 4GB swap. This time it compiled successfully, but only barely. So it takes slightly less than 18GB of memory to compile unikernel - is this expected?

The result in rumprun-packages/nodejs/build-4.3.0/out/Release/node-aplikacija_word-finder is 108MB big. I hardly imagine what was make doing to consume 18 gigs.

Any idea?

anttikantee commented 7 years ago

We don't supply the toolchain, so it's tough to say why it's using so much memory. Maybe you can play around with different version of cc/make/ld (whichever was consuming the memory). My poor development laptop has nowhere near 18GB of memory, so I'm sure it didn't consume that much when I tried node.js out.