imyller / meta-nodejs

OpenEmbedded layer for latest Node.js releases
MIT License
79 stars 87 forks source link

Enable multi-core build #10

Closed gsikorski closed 9 years ago

gsikorski commented 9 years ago

Node-js tarball includes option to build using standard make or ninja. By default make is used and it does not support multiple job switch. On highly multicore machines this slows the build down a lot. Simple passing --ninja switch to configure script should enable ninja build and use all available cores.

imyller commented 9 years ago

I'll look into this.

I expect this to require presence of ninja build system - in addition to default make. The simple addition of --ninja switch to OE recipe would add another package dependency for building node.js.

For example, in my Yocto 1.8 test environment I do not have ninja build system (ninja-native) present.

This could be added as an option (or maybe detected if ninja-native package is available) but not as default choice.

gsikorski commented 9 years ago

Yes, this makes sense. We have python-ninja present on our system, since we are also building chromium, which uses it by default. I tested this solution and it works fine on our setup. We will patch our system anyway and can send you a pull request anyway.

gsikorski commented 9 years ago

I was wrong in my last comment, the package needed is called "ninja-native" and is available under meta-browser (https://github.com/OSSystems/meta-browser) if you enable it in layers.conf.

imyller commented 9 years ago

PR for this is welcome.

I'd still look into option making this optional or auto-detected feature; if right dependencies are available.

meta-browser layer is common, but when building for headless embedded systems it is often unnecessary and left out.

gsikorski commented 9 years ago

Yep. I wonder if auto-detect is not done automatically by nodejs configure task. It could be smart enough to detect missing ninja command and revert to default make. Unless it simply fails, then ugly hack can be done in bitbake do_configure to detect this and use fallback. Unfortunately, I do not know the way to specify optional dependency on bitbake level, which would be useful here.

gsikorski commented 9 years ago

I will soon provide a patch with fixes to make build which will enable multi-core build.

imyller commented 9 years ago

Merged PR #12

Multi-core builds are now enabled.

Thank you @gsikorski for your efforts.

thomasantony commented 7 years ago

Was this broken at some point? I am building my image on a 64-core EC2 instance and nodejs holds up the entire build, using just a single core. It take 2x-3x longer to build nodejs than the entire rest of the image. I am tried with both Yocto Poky "fido" as well as "jethro".

I see that the current recipes use oe_runmake instead of make. Is there someway I can make it utilize multiple cores?

This was mentioned in an issue made by @gsikorski here as well (which I assume led to this issue being created): https://github.com/nodejs/node/issues/4036