build node.js using clangcl when under Windows

[lemire]

Description

Chrome and Firefox are built with Clang under Windows.

@targos made it possible to build Node.js using clang under Windows, see https://github.com/nodejs/node/pull/35433

At least in some tests, this seems to result in better performance:

                                                          confidence improvement accuracy (*)    (**)    (***)
v8\get-stats.js n=1000000 method='getHeapSpaceStatistics'                 3.36 %      ±10.99% ±14.62%  ±19.03%
v8\get-stats.js n=1000000 method='getHeapStatistics'                     10.35 %      ±10.90% ±14.50%  ±18.87%
v8\serialize.js n=1000000 len=16384                                       1.97 %       ±2.21%  ±3.14%   ±4.53%
v8\serialize.js n=1000000 len=256                                ***      5.59 %       ±1.61%  ±2.30%   ±3.36%
v8\serialize.js n=1000000 len=524288                                     19.28 %      ±48.92% ±76.74% ±130.72%

It is only preliminary, but it is not very difficult to conduct a full examination. There are some maintenance benefits to switching to LLVM under Windows as well. I stress that Node works right now under Windows when compiled with LLVM (credit to @targos).

There is at least one minor issue with ICU: https://github.com/nodejs/node/issues/34201 A tiny patch might be required. I actually expect that it might be the only problem.

I am opening this issue following a request by @mcollina

Further reading: Should Node.js be built with ClangCL under Windows?

Checklist:

• [x] Add a configuration option in vcbuild.bat to pick from MSVC and CLangCL - Landed in https://github.com/nodejs/node/pull/52870
• [x] harmonize Clang checks and update V8 warning flags - Landed in https://github.com/nodejs/node/pull/52873
• [x] Upstream changes in gyp-next to enable setting language standard keys in msvs_settings - Landed upstream in https://github.com/nodejs/gyp-next/pull/252, will eventually get updated in the main project
• [x] Install ClangCL on Windows hosts in CI - Landed in https://github.com/nodejs/build/pull/3714
• [x] Upstream fix for ICU issue for cross-compilation to ARM64 - @StefanStojanovic is looking into this, ideally, we'll fix it upstream and pull it to Node.js as a dependency. PR awaiting one more reviewer https://github.com/unicode-org/icu/pull/3023
• [x] Upstream fix for V8 ARM64 cross-compilation - Landed upstream in FP16 https://github.com/Maratyszcza/FP16/pull/26 and pulled in V8 in https://chromium-review.googlesource.com/c/v8/v8/+/5553996, will eventually get updated in the main project
• [x] Implement logic for detecting installed ClangCL version - This is currently hardcoded to the version that can be installed via Visual Studio as a component. @StefanStojanovic has discussed this with @huseyinacacak-janea and they are currently investigating it.
• [x] Configure Windows CI machines to be able to compile and test ClangCL - This is done, there were 2 build repo PRS: https://github.com/nodejs/build/pull/3714 and https://github.com/nodejs/build/pull/3786
• [ ] Configure Windows CI jobs to build and test ClangCL - @StefanStojanovic will work on this once most other things are done so we can start gaining confidence for ClangCL builds
• [ ] Run benchmarks to ensure performances are not worsened by moving to ClangCL from MSVC - This is self-explanatory.
• [ ] Make sure that the Node.js ecosystem is not facing issues because of the migration - Make sure node-gyp, popular npm packages, etc. are all still working well
• [ ] Investigate the compilation times between MSVC and ClangCL - This should be taken into serious consideration as we'll be compiling with both MSVC and ClangCL in the CI for at least some time. We should investigate the possibility of using ccache or similar tools with ClangCL.
• [ ] If there are no reasons not to, switch to using ClangCL as a default compiler for future releases - Potentially Node.js v23 would be a nice time to do this. The same can also be said for dropping x86 support based on https://github.com/nodejs/node/issues/42543#issuecomment-2066905073

[mcollina]

I recall that it was recommended by MS folks to build using clang on Windows.

[lemire]

One issue to consider is that Google switched over so they are developing V8 under ClangCL.

[StefanStojanovic]

I was thinking about opening an umbrella issue with a checklist or something like that for all things related to enabling ClangCL compilation on Windows, but since there is already this issue if you agree @lemire we can use it for that (if not, I'll create a separate issue with the content of this message).

Anyway, the following is (not sorted according to priority) a list of things done, being worked on, or that should be worked on before stating ClangCL is officially supported for compiling Node.js on Windows:

• [x] Add a configuration option in vcbuild.bat to pick from MSVC and CLangCL - This landed in https://github.com/nodejs/node/pull/52870
• [x] harmonize Clang checks and update V8 warning flags - This landed in https://github.com/nodejs/node/pull/52873
• [x] Upstream changes in gyp-next to enable setting language standard keys in msvs_settings - This landed upstream in https://github.com/nodejs/gyp-next/pull/252, will eventually get updated in the main project
• [x] Install ClangCL on Windows hosts in CI - This landed in https://github.com/nodejs/build/pull/3714
• [ ] Fix ICU issue for cross-compilation to ARM64 - I'm looking into this, ideally, we'll fix it upstream and pull it to Node.js as a dependency. EDIT Opened a draft PR in ICU https://github.com/unicode-org/icu/pull/3013
• [ ] Fix ARM64 cross-compilation - WIth the latest V8 updates (v12.2 and v12.4) V8 has a new third-party dependency which is causing issues in Node.js. I'm looking into this, ideally, we'll fix it upstream and pull it to Node.js as a dependency.
  • EDIT: Opened a PR upstream in a V8 dependency project FP16 https://github.com/Maratyszcza/FP16/pull/26. It landed. Currently working on updating it in V8.
• [ ] Implement logic for detecting installed ClangCL version - This is currently hardcoded to the version that can be installed via Visual Studio as a component. I've discussed this with @huseyinacacak-janea and he is currently investigating it
• [ ] Configure Windows CI jobs to build and test ClangCL - I will work on this once most other things are done so we can start gaining confidence for ClangCL builds
• [ ] Run benchmarks to ensure performances are not worsened by moving to ClangCL from MSVC - This is self-explanatory.
• [ ] Make sure that the Node.js ecosystem is not facing issues because of the migration - Make sure node-gyp, popular npm packages, etc. are all still working well
• [ ] Investigate the compilation times between MSVC and ClangCL - This should be taken into serious consideration as we'll be compiling with both MSVC and ClangCL in the CI for at least some time. We should investigate the possibility of using ccache or similar tools with ClangCL.
• [ ] If there are no reasons not to, switch to using ClangCL as a default compiler for future releases - Potentially Node.js v23 would be a nice time to do this. The same can also be said for dropping x86 support based on https://github.com/nodejs/node/issues/42543#issuecomment-2066905073

I might have missed something, but most things should be covered here. If anyone notices something missing or wrong, please let me know.

cc: @targos @nodejs/platform-windows

[lemire]

@StefanStojanovic It definitively makes sense to leverage the current issue.

[StefanStojanovic]

@StefanStojanovic It definitively makes sense to leverage the current issue.

Great! One issue I see with it is that if the comment gets edited often (eg. I edited it just now to add a PR I opened) it will probably not be sent as a notification, plus I'm probably the only one who can edit it. If anyone has a better idea eg. a GitHub Project or something like that, I'm all for it.

[lemire]

@StefanStojanovic I can edit your comment, but another take on it would be to edit the issue first post (that is, the text I have written). Do you have edit access?

I think you should be able to edit the issue... I think you have access... don't you?

That is, I don't own this issue, it should be a collective one.

[StefanStojanovic]

@StefanStojanovic I can edit your comment, but another take on it would be to edit the issue first post (that is, the text I have written). Do you have edit access?

I think you should be able to edit the issue... I think you have access... don't you?

That is, I don't own this issue, it should be a collective one.

Nope, I do not see edit. This is what I see. Not sure GH supports collective issues, would be great if it did...

[lemire]

@joyeecheung Can you help? I think we would like @StefanStojanovic to have the ability to edit this issue.

[H4ad]

@lemire He will have the ability to edit once he becomes a nodejs collaborator (probably next week)

[lemire]

@StefanStojanovic Given @H4ad's answer, I recommend just waiting a week or so. ;-)

[hocheung-chromium]

https://issues.chromium.org/issues/341803745

fp16 in v8 has been updated to the latest version, I think the blocker on the v8 side can be removed.

[joyeecheung]

If there are no reasons not to, switch to using ClangCL as a default compiler for future releases - Potentially Node.js v23 would be a nice time to do this.

We should probably update this point since V8 announced that they will drop MSVC support in September: https://groups.google.com/g/v8-dev/c/8mSxb2UriSU

[StefanStojanovic]

Hey everyone, while we're waiting for the ICU fix to get ported to Node, I just wanted to share with you the results of performance benchmarks I ran with ClangCL compiled binaries compared to MSVS ones (both compiled and ran locally). I used 10 runs to limit the execution to roughly a weekend. Once we can compile with ClangCL in CI I will use one of the CI machines to run even more benchmarks there.