sqlite3 build fails

[mayfield]

Similar to #42 I'm unable to build sqlite3 because one of the files treated as non-binary is actually a generated build artifact.

I've posted some details in the comments of https://github.com/electron-userland/electron-builder/issues/6751 as well.

Here is the build output...

  • packaging       platform=darwin arch=universal electron=18.0.3 appOutDir=dist/mac-universal
  ⨯ Expected all non-binary files to have identical SHAs when creating a universal build but "Contents/Resources/app/node_modules/sqlite3/build-tmp-napi-v6/Makefile" did not  failedTask=build stackTrace=Error: Expected all non-binary files to have identical SHAs when creating a universal build but "Contents/Resources/app/node_modules/sqlite3/build-tmp-napi-v6/Makefile" did not
    at exports.makeUniversalApp (/Users/mayfield/project/sauce4zwift/node_modules/@electron/universal/src/index.ts:123:15)

Inspection of the two Makefiles referenced shows they are generated during the sqlite build process and are non deterministic in content; most likely because of a parallel make system. The exact differences are just the order of some files in a make target dep list.

The fix employed in #42 is a hard coded exclusion of the offending nib file, but perhaps a more generic exclusion list that a user can configure in the top level package.json would address these issues better? That or a relaxation of the SHA comparison when applied to native dependencies? Just food for thought as I'm sure you don't want to be in the business of having to update and release new code every time a user wants to use some off the wall native dep.

If any of those approaches appeals to you, I can probably take a swag at implementing one and submitting a PR if you like.

[mayfield]

I patched my universal to use singleArchFiles as an exclusion list for testing, which worked but only allowed the process to get to the next error while trying to do lipo on the sqlite MachO bins. This is probably outside the scope of this issue, and perhaps something node-sqlite3 needs to address, but I wanted to mention it.

  • packaging       platform=darwin arch=universal electron=17.1.0 appOutDir=dist/mac-universal
  ⨯ Command failed with a non-zero return code (1):
lipo /private/var/folders/71/4_w74cxj79s28vhsr869b7tc0000gn/T/electron-universal-oOT5d3/Tmp.app/Contents/Resources/app/node_modules/sqlite3/build-tmp-napi-v6/Release/node_sqlite3.node /Users/mayfield/project/sauce4zwift/dist/mac-universal--arm64/Sauce for Zwift™.app/Contents/Resources/app/node_modules/sqlite3/build-tmp-napi-v6/Release/node_sqlite3.node -create -output /private/var/folders/71/4_w74cxj79s28vhsr869b7tc0000gn/T/electron-universal-oOT5d3/Tmp.app/Contents/Resources/app/node_modules/sqlite3/build-tmp-napi-v6/Release/node_sqlite3.node

fatal error: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/lipo: /private/var/folders/71/4_w74cxj79s28vhsr869b7tc0000gn/T/electron-universal-oOT5d3/Tmp.app/Contents/Resources/app/node_modules/sqlite3/build-tmp-napi-v6/Release/node_sqlite3.node and /Users/mayfield/project/sauce4zwift/dist/mac-universal--arm64/Sauce for Zwift™.app/Contents/Resources/app/node_modules/sqlite3/build-tmp-napi-v6/Release/node_sqlite3.node have the same architectures (arm64) and can't be in the same fat output file  failedTask=build stackTrace=Error: Command failed with a non-zero return code (1):
lipo /private/var/folders/71/4_w74cxj79s28vhsr869b7tc0000gn/T/electron-universal-oOT5d3/Tmp.app/Contents/Resources/app/node_modules/sqlite3/build-tmp-napi-v6/Release/node_sqlite3.node /Users/mayfield/project/sauce4zwift/dist/mac-universal--arm64/Sauce for Zwift™.app/Contents/Resources/app/node_modules/sqlite3/build-tmp-napi-v6/Release/node_sqlite3.node -create -output /private/var/folders/71/4_w74cxj79s28vhsr869b7tc0000gn/T/electron-universal-oOT5d3/Tmp.app/Contents/Resources/app/node_modules/sqlite3/build-tmp-napi-v6/Release/node_sqlite3.node
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       fatal error: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/lipo: /private/var/folders/71/4_w74cxj79s28vhsr869b7tc0000gn/T/electron-universal-oOT5d3/Tmp.app/Contents/Resources/app/node_modules/sqlite3/build-tmp-napi-v6/Release/node_sqlite3.node and /Users/mayfield/project/sauce4zwift/dist/mac-universal--arm64/Sauce for Zwift™.app/Contents/Resources/app/node_modules/sqlite3/build-tmp-napi-v6/Release/node_sqlite3.node have the same architectures (arm64) and can't be in the same fat output file
    at ChildProcess.<anonymous> (/Users/mayfield/project/sauce4zwift/node_modules/@malept/cross-spawn-promise/src/index.ts:172:16)
    at ChildProcess.emit (node:events:390:28)
    at maybeClose (node:internal/child_process:1064:16)
    at Socket.<anonymous> (node:internal/child_process:450:11)
    at Socket.emit (node:events:390:28)
    at Pipe.<anonymous> (node:net:687:12)
make: *** [build] Error 1

[mayfield]

Upon further review, it looks like the sqlite3 build process produces libs for both arm and x86 archs. So electron-builder is building both archs for both application targets and then the packaging step is trying to merge all those libs which means it's attempting to merge arm with arm and x86 with x86.

Perhaps this is a node-gyp compatibility issue in electron-builder? I'm not familiar with node-gyp enough to grok what's going on and where.

[MarshallOfSound]

That or a relaxation of the SHA comparison when applied to native dependencies? Just food for thought as I'm sure you don't want to be in the business of having to update and release new code every time a user wants to use some off the wall native dep.

The SHA comparison is there for a reason and the error you are seeing is expected. You need to choose whether to either fix the SHA mismatch (unlikely in this case) or exclude the file from your application package (this should be configured in builder / packager).

The lipo error you are seeing is because both your apps you are trying to stitch together contain an arm64 build of sqlite3. This is probably a bug in electron-builder (which we don't actively support).

[mayfield]

The lipo error you are seeing is because both your apps you are trying to stitch together contain an arm64 build of sqlite3. This is probably a bug in electron-builder (which we don't actively support).

It's how sqlite ships with and builds its binaries. It ships with both arches prebuilt and builds artifacts into dirs with the arch name in them. electron-universal expects the artifacts to be identical.

[jjacobson95]

Has there been any solution / work-around to this issue?

[mayfield]

Has there been any solution / work-around to this issue?

I don't think so. I ended up switching to better-sqlite3 for other reasons (it's very good) which incidentally doesn't have a build layout affected by this bug.