Weird error MSB6006: "cmd.exe" exited with code 1 on some packages

[jviotti]

We've been facing a very strange issue at Etcher (https://github.com/resin-io/etcher). We have node-sass as a development dependency, and lzma-native as a production dependency.

When node-gyp was updated to v3.6.0, node-sass picked up this change, causing lzma-native to fail with the following error on Windows:

C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\V140\Microsoft.CppCommon.targets(171,5): error MSB6006: "cmd.exe" exited with code 1. [C:\projects\etcher\node_modules\lzma-native\build\liblzma.vcxproj]

So to make it clear, after the node-gyp upgrade on a development dependency (node-sass), a totally unrelated production dependency, lzma-native, suddenly starts to fail on Windows. What's even more strange is that node-sass compiles just fine, and removing node-sass from the development dependencies causes lzma-native to start working again. As a fix, we've started shrinkwrapping development dependencies as well, and locking node-gyp to v3.5.0, which makes everything work again.

The problem seems to have been introduced by https://github.com/nodejs/node-gyp/commit/ae141e19060ecb1c596a96acf9510d66b7376c18, however I don't understand why.

See https://github.com/resin-io/etcher/pull/1191. You can probably reproduce by cloning Etcher, checking out a commit before such fix, and running make electron-develop (which is a Makefile target that we use to pass some node-gyp options to npm, run bower, etc).

[lurch]

The problem seems to have been introduced by ae141e1, however I don't understand why.

Hmm, I wonder if the problem might be that this part of the change:

  function findMsbuild () {
    if (config.variables.msbuild_path) {
      command = config.variables.msbuild_path

means that the MSBuild command is now run with its full-path "C:\Program Files (x86)\MSBuild\...\MSBuild.exe", whereas previously it was just run as MSBuild.exe, and this additional "C:\Program Files (x86)\MSBuild\..." is now just enough to push the lzma-native compilation command over the cmd.exe command-line length limit on Windows? http://stackoverflow.com/questions/3205027/maximum-length-of-command-line-string

[bnoordhuis]

cc @refack @joaocgreis

[refack]

hmmm... giving this some thought...

[refack]

~It's not a "length of command line" issue, it's a "not run from VS console" issue. The lzma-native build calls lib.exe which is not in the path unless vsvarsall.bat (or it's derivatives) is run. Up till node-gyp:v3.5.x the whole thing would fail. Now we can find & call msbuild.exe in the general case, but than we fail if the build is complicated ;(~

[refack]

Scratch that! It's probably a bug in GYP. It erroneously escapes the action command... from:

'action': [
  'lib -def:"<(module_root_dir)\\deps\\doc\\liblzma.def" -out:"<(module_root_dir)\\deps\\bin_x86-64\\lzma.lib" -machine:x64 && if not exist <(dlldir) mkdir <(dlldir) && copy "<(module_root_dir)\\deps\\bin_x86-64\\liblzma.dll" "<(dlldir)\\liblzma.dll"'
]

to: (extra " breaks cmd syntax)

call call "lib -def:"C:\code\3party\lzma-native\deps\doc\liblzma.def" -out:"C:\code\3party\lzma-native\deps\bin_x86-64\lzma.lib" -machine:x64 && if not exist C:\code\3party\lzma-native\binding-v2.0.0-node-v51-win32-x64 mkdir C:\code\3party\lzma-native\binding-v2.0.0-node-v51-win32-x64 && copy "C:\code\3party\lzma-native\deps\bin_x86-64\liblzma.dll" "C:\code\3party\lzma-native\binding-v2.0.0-node-v51-win32-x64\liblzma.dll""
if %errorlevel% neq 0 exit /b %errorlevel%

Instead of

call call lib -def:"C:\code\3party\lzma-native\deps\doc\liblzma.def" -out:"C:\code\3party\lzma-native\deps\bin_x86-64\lzma.lib" -machine:x64 && if not exist C:\code\3party\lzma-native\binding-v2.0.0-node-v51-win32-x64 mkdir C:\code\3party\lzma-native\binding-v2.0.0-node-v51-win32-x64 && copy "C:\code\3party\lzma-native\deps\bin_x86-64\liblzma.dll" "C:\code\3party\lzma-native\binding-v2.0.0-node-v51-win32-x64\liblzma.dll"
if %errorlevel% neq 0 exit /b %errorlevel%

[jviotti]

Awesome, great to hear you found the root cause of this issue! BTW, could you explain why in our case lzma-native started failing after node-gyp was upgraded by node-sass? I'm assuming that such upgrade caused lzma-native to start re-using node-sass's node-gyp version, which built node-sass fine, but caused errors on lzma-native because lzma-native uses lib.exe during build.

Is this correct?

[refack]

node-gyp come with npm. It's a tool that compiles "native bindings" for some modules. Unfortunately native bindings need to be re-compiled for every version of node (or in this case electron). lzma-native has an online cache of precompiled bindings, seems like the new version couldn't find a precompiled pack for your platform. In that case it tries to compile it with node-gyp which triggered the bug (bug might have been there for a while)

[addaleax]

@refack Yeah, but it’s still a curious thing how the node-sass upgrade made the difference?

[lurch]

I'm afraid I'll have to disagree with the (bug might have been there for a while) comment. Up until 5 days ago all our Windows CI builds were working fine on Appveyor (including building lzma-native from source) but then all of a sudden (after the release of node-gyp 3.6.0) all our Windows CI builds started failing.

And our workaround for the problem in https://github.com/resin-io/etcher/pull/1191 was to basically version-lock node-gyp to 3.5.0 which got everything working again.

[refack]

Up until 5 days ago all our Windows CI builds were working fine on Appveyor (including building lzma-native from source) but then all of a sudden (after the release of node-gyp 3.6.0) all our Windows CI builds started failing.

I said might... 🤷 I'm still digging...

[refack]

Found it Added escaping of custom actions, which may lead to broken commands /cc @kunalspathak

[kunalspathak]

Thanks @refack for the investigation. I went back and checked and it seems the quote_cmd issue was fixed in node-chakracore but https://github.com/nodejs/node-gyp/pull/873 was not updated with the fix. I will send another PR to fix this.

[rvagg]

resolved in @3.6.1

[marcelfalliere]

Having the same problem in 3.6.2

$ npm install

> ref@1.3.5 install C:\Users\xavier\Documents\Carestream\server\node_modules\ref
> node-gyp rebuild

C:\Users\xavier\Documents\Carestream\server\node_modules\ref>if not defined npm_config_node_gyp (node "C:\Program Files (x86)\nodejs\node_modules\npm\node_modules\npm-lifecycle\node-gyp-bin\\..\..\node_modules\node-gyp\bin\node-gyp.js" rebuild )  else (node "C:\Program Files (x86)\nodejs\node_modules\npm\node_modules\node-gyp\bin\node-gyp.js" rebuild )
Building the projects in this solution one at a time. To enable parallel build, please add the "/m" switch.
C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\V140\Microsoft.CppCommon.targets(356,5): error MSB6006: "CL.exe" exited with code -1073741515. [C:\Users\xavier\Documents\Carestream\server\node_modules\ref\build\binding.vcxproj]
gyp ERR! build error
gyp ERR! stack Error: `C:\Program Files (x86)\MSBuild\14.0\bin\msbuild.exe` failed with exit code: 1
gyp ERR! stack     at ChildProcess.onExit (C:\Program Files (x86)\nodejs\node_modules\npm\node_modules\node-gyp\lib\build.js:258:23)
gyp ERR! stack     at emitTwo (events.js:126:13)
gyp ERR! stack     at ChildProcess.emit (events.js:214:7)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:198:12)
gyp ERR! System Windows_NT 6.3.9600
gyp ERR! command "C:\\Program Files (x86)\\nodejs\\node.exe" "C:\\Program Files (x86)\\nodejs\\node_modules\\npm\\node_modules\\node-gyp\\bin\\node-gyp.js" "rebuild"
gyp ERR! cwd C:\Users\xavier\Documents\Carestream\server\node_modules\ref
gyp ERR! node -v v8.9.4
gyp ERR! node-gyp -v v3.6.2
gyp ERR! not ok
npm WARN @nestjs/common@4.6.6 requires a peer of reflect-metadata@0.1.10 but none is installed. You must install peer dependencies yourself.
npm WARN @nestjs/core@4.6.6 requires a peer of reflect-metadata@0.1.10 but none is installed. You must install peer dependencies yourself.
npm WARN @nestjs/microservices@4.6.6 requires a peer of reflect-metadata@0.1.10 but none is installed. You must install peer dependencies yourself.
npm WARN @nestjs/swagger@1.1.4 requires a peer of reflect-metadata@0.1.10 but none is installed. You must install peer dependencies yourself.
npm WARN @nestjs/websockets@4.6.6 requires a peer of reflect-metadata@0.1.10 but none is installed. You must install peer dependencies yourself.
npm WARN nest-typescript-starter@1.0.0 No repository field.
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@1.1.3 (node_modules\fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for fsevents@1.1.3: wanted {"os":"darwin","arch":"any"} (current: {"os":"win32","arch":"ia32"})