search

nodejs / build

Better build and test infra for Node.
506 stars 165 forks source link

vs2019 failing #3442

Closed RafaelGSS closed 1 year ago

RafaelGSS commented 1 year ago

Hey, I've been running the CI and vs2019 keeps failing without apparent error:

Examples:

targos commented 1 year ago

@StefanStojanovic

richardlau commented 1 year ago

Examples:

These failed on test-rackspace-win2019_vs2019-x64-3 -- that machine (or any test-rackspace-win2019* machine) isn't in our inventory (yet?): https://github.com/nodejs/build/blob/f7e0f90a55b0c866041668e4285a16274f1e36f7/ansible/inventory.yml#L316-L330

I assume the win2019* machines are part of the work going on under https://github.com/nodejs/build/issues/3373

Looking at the machines with win-vs2019 label it looks like there are still two win2012r2-vs2019 machines available, so we could try temporarily disabling test-rackspace-win2019_vs2019-x64-3 and/or the other win2019-vs2019 machines to force the builds onto the win2012r2 ones.

richardlau commented 1 year ago

Looking at the machines with win-vs2019 label it looks like there are still two win2012r2-vs2019 machines available, so we could try temporarily disabling test-rackspace-win2019_vs2019-x64-3 and/or the other win2019-vs2019 machines to force the builds onto the win2012r2 ones.

In the interests of not blocking the security release, I've marked all the win2019-vs2019 machines offline in Jenkins to force builds onto the win2012r2-vs2019 machines.

cc @StefanStojanovic

richardlau commented 1 year ago

It looks like win2019-vs2019 is needed for Node.js 20 builds, so I reenabled -1 and -2 (-3 and -4 remain offline as they were the ones erroring).

richardlau commented 1 year ago

Examples:

* https://ci.nodejs.org/job/node-compile-windows/51805/nodes=win-vs2019/console

* https://ci.nodejs.org/job/node-compile-windows/51804/nodes=win-vs2019/console

These are Node.js 18 and 16 builds. Here are more failures on win2019-vs2019 for Node.js 16.x:

It looks like the win2019 VMs do not have WiX 3 installed: e.g. https://ci.nodejs.org/job/node-compile-windows/51831/nodes=win-vs2019/console

11:03:32 > if defined msi (
11:03:32 echo Looking for WiX installation for Visual Studio 2019...  
11:03:32  if not exist "C:\Program Files (x86)\WiX Toolset v3.11\\SDK\VS2017" (
11:03:32 echo Failed to find WiX install for Visual Studio 2019  
11:03:32  echo VS2019 support for WiX is only present starting at version 3.11  
11:03:32  goto msbuild-not-found 
11:03:32 )  
11:03:32  if not exist "C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\\..\MSBuild\Microsoft\WiX" (
11:03:32 echo Failed to find the WiX Toolset Visual Studio 2019 Extension  
11:03:32  goto msbuild-not-found 
11:03:32 ) 
11:03:32 ) 
11:03:32 Looking for WiX installation for Visual Studio 2019...
11:03:32 Failed to find the WiX Toolset Visual Studio 2019 Extension
11:03:32 
11:03:32 > echo Failed to find a suitable Visual Studio installation. 
11:03:32 Failed to find a suitable Visual Studio installation.
11:03:32 
11:03:32 > echo Try to run in a "Developer Command Prompt" or consult 
11:03:32 Try to run in a "Developer Command Prompt" or consult
11:03:32 
11:03:32 > echo https://github.com/nodejs/node/blob/HEAD/BUILDING.md#windows 
11:03:32 https://github.com/nodejs/node/blob/HEAD/BUILDING.md#windows
11:03:32

There's a further weird looking error but I can't tell if it's a separate problem or a follow on from WiX not being found causing things to not be built:

11:03:32 > tar cavf binary/binary.tar.gz config.gypi icu_config.gypi Release/node.exe Release/node.lib Release/openssl-cli.exe Release/cctest.exe Release/node.pdb 
11:03:32 tar: hocolatey\bin;C:\Pro???? Files\Git\cmd;C:\Pr????m Files\Git\mingw64\????C:\Program Files\Git????\bin;C:\Program File??????????????: Couldn't visit directory: No such file or directory
11:03:32 tar: : Couldn't visit directory: No such file or directory
11:03:32 tar: : Couldn't visit directory: No such file or directory
11:03:32 tar: : Couldn't visit directory: No such file or directory
11:03:32 tar: : Couldn't visit directory: No such file or directory
11:03:32 tar: : Couldn't visit directory: No such file or directory
11:03:32 tar: : Couldn't visit directory: No such file or directory
11:03:32 tar: Error exit delayed from previous errors.
11:03:32

The migration to WiX 4 (https://github.com/nodejs/node/pull/45943) (semver-minor) hasn't been backported to Node.js 18 or 16. I'm hesitant to backport to Node.js 16 this close to its End-of-Life -- thoughts @nodejs/lts?

Anyway I've reenabled all the win2019-vs2019 VMs in Jenkins -- I believe that they are fine for builds for Node.js 20 and main and having more VMs online should help the current Jenkins backlog.

richardlau commented 1 year ago

Made a temporary change to the agent config for the win2019 VMs -- I've removed the labels

win-vs2019 win-vs2019-x86 win-vs2019-arm64

so the node-compile-windows CI job runs on the win2012r2-vs2019 VMs. If we can get WiX 3 set up on the win2109 VMs we can put those labels back.

cc @StefanStojanovic @joaocgreis

The migration to WiX 4 (https://github.com/nodejs/node/pull/45943) (semver-minor) hasn't been backported to Node.js 18 or 16. I'm hesitant to backport to Node.js 16 this close to its End-of-Life -- thoughts https://github.com/orgs/nodejs/teams/lts?

FWIW I tried a quick cherry-pick of https://github.com/nodejs/node/pull/45943 locally onto v18.x-staging and v16.x-staging and there were merge conflicts (both branches) which reinforces my hesitancy to backport that to v16.x this close to Node.js 16 End-of-Life next month.

StefanStojanovic commented 1 year ago

Hello everyone, sorry for replying just now. I was on vacation and I'm currently not working full-time on Node.js. The issue was not WiX 3, but rather its VS extension which was missing on some machines, although an ansible playbook should install them. Anyway, I fixed that and added 2 machines back to the compilation, the plan is to add the others in a few days too. Also, I made a PR updating inventory with the new machines https://github.com/nodejs/build/pull/3460.

StefanStojanovic commented 1 year ago

Since this issue was fixed manually I didn't see it repeating afterward. I also made an issue for myself to fix it in Ansible. With all this in mind, I'll close the issue and if needed it can be reopened.