Open WrathofthePast opened 2 years ago
Hello!
This is a feature I worked on. It looks like build/Makefile
is different when you are building for x64 than it is for arm64. Any idea why?
The easiest fix might be to block it from being put into asar from package.json
's by putting "!build/Makefile"
into build.files[]
array.
The easiest fix might be to block it from being put into asar from
package.json
's by putting"!build/Makefile"
intobuild.files[]
array.
I tried that and got a different error:
⨯ Can't reconcile two non-macho files build/addon.target.mk failedTask=build stackTrace=Error: Can't reconcile two non-macho files build/addon.target.mk
at Object.exports.mergeASARs (/Users/BobWhite/razer-macos-backup/node_modules/@electron/universal/src/asar-utils.ts:151:13)
at exports.makeUniversalApp (/Users/BobWhite/razer-macos-backup/node_modules/@electron/universal/src/index.ts:200:13)
at processTicksAndRejections (node:internal/process/task_queues:96:5)
at MacPackager.doPack (/Users/BobWhite/razer-macos-backup/node_modules/app-builder-lib/src/macPackager.ts:125:9)
at MacPackager.pack (/Users/BobWhite/razer-macos-backup/node_modules/app-builder-lib/src/macPackager.ts:179:7)
at Packager.doBuild (/Users/BobWhite/razer-macos-backup/node_modules/app-builder-lib/src/packager.ts:441:9)
at Object.executeFinally (/Users/BobWhite/razer-macos-backup/node_modules/builder-util/src/promise.ts:12:14)
at Packager._build (/Users/BobWhite/razer-macos-backup/node_modules/app-builder-lib/src/packager.ts:376:31)
at Packager.build (/Users/BobWhite/razer-macos-backup/node_modules/app-builder-lib/src/packager.ts:337:12)
at Object.executeFinally (/Users/BobWhite/razer-macos-backup/node_modules/builder-util/src/promise.ts:12:14)
error Command failed with exit code 1.
Interestingly, this is a similar error to the one I got when I used electron-universal v1.0.5 with electron-builder v23.0.0. However, the first two at ...
s are different. In that electron-builder version, they used your mergeASARs API, so electron-universal v1.0.5 didn't satisfy the dependency.
Well, you can ignore that other files as well, or even all files in build
except build/Release
? Not sure how it is used in your app, but the basic idea here is that you shouldn't bundle platform-specific files into your app unless they:
Well, you can ignore that other files as well, or even all files in
build
exceptbuild/Release
?
That's what I've trying the past few minutes. After adding `"!build/addon.target.mk", the same error occurred with build/config.gypi, so added that too. The next error is with build/Release/addon.node. What do I do now since you're saying I can't ignore files in build/Release?
Not sure how it is used in your app, but the basic idea here is that you shouldn't bundle platform-specific files into your app unless they:
- have different names on each platform
They do not.
- are binary .node files (binding for native addons)
Do you mean like node-addon-api
or native-ext-loader
?
Could you run file build/Release/addon.node
? I'm curious if this is an addon built for macOS at all.
Could you run
file build/Release/addon.node
? I'm curious if this is an addon built for macOS at all.
What do you mean? Run it in the command line or add it to package.json?
By the way, I added "!build/Release/addon.node"
. Then it errored on build/Release/.deps/Release/obj.target/addon/src/driver/addon.o.d
. After adding that it errored on each driver in build/Release/.deps/Release/obj.target/addon/librazermacos/src/lib
. Finally, after adding that it, it successfully packaged the universal app and it works.
What do you mean? Run it in the command line or add it to package.json?
command line.
What do you mean? Run it in the command line or add it to package.json?
command line.
Here's the output:
macbook-pro:razer-macos-backup me$ file build/Release/addon.node
build/Release/addon.node: Mach-O universal binary with 2 architectures: [x86_64:Mach-O 64-bit bundle x86_64Mach-O 64-bit bundle x86_64] [arm64:Mach-O 64-bit bundle arm64Mach-O 64-bit bundle arm64]
build/Release/addon.node (for architecture x86_64): Mach-O 64-bit bundle x86_64
build/Release/addon.node (for architecture arm64): Mach-O 64-bit bundle arm64
I didn't need to exclude build/binding.Makefile, build/gyp-mac-tool, or build/Release/obj.target to make it build. I had to exclude build/Release/.deps, build/Release/addon.node, build/Makefile, build/addon.target.mk, and build/config.gypi.
Ah gosh, it is already universal. I wonder why it was different between x64 and arm64 bundles then...
Ah gosh, it is already universal. I wonder why it was different between x64 and arm64 bundles then...
I have an idea: in my build.files[]
array, I've always included build
. Is that redundant? Also, I noticed that build/Release/obj.target is identical to build/Release/.deps/Release/obj.target. Why are they duplicated?
This is just the way the node.js addons are built. You can definitely include whole folder if this makes sense for your app.
I noticed in addon.target.mk that -arch x86_64
is not being passed to CFLAGS_Release
while -arch arm64
is. Furthermore, I noticed config.gypi contains "msvs_configuration_platform": "ARM64"
& "xcode_configuration_platform": "arm64"
, but does not have x86_64 anywhere. Do you think my binding.gyp is misconfigured?
Unclear. I'd be more curious to learn what made this binding universal. Normally node addons are for single arch.
Unclear. I'd be more curious to learn what made this binding universal. Normally node addons are for single arch.
I attached the package.json and binding.gyp, if you wanted to look at it. Do you think all the exclusions from the build.files[]
array will any cause problems?
package.json and binding.gyp.zip
Not sure how it is used in your app, but the basic idea here is that you shouldn't bundle platform-specific files into your app unless they:
- have different names on each platform
- are binary .node files (binding for native addons)
Unclear. I'd be more curious to learn what made this binding universal. Normally node addons are for single arch.
I think I figured it out. I use native-ext-loader instead of node-addon-loader. In contrast, it "allows to build path at runtime." In my webpack.main.additions.js, native-ext-loader is set to resolve: {extensions: ['.node']}
. Doesn't this mean I have platform-specific binary .node files?
I'm having the same problem:
⨯ Can't reconcile two non-macho files node_modules/better-sqlite3/build/deps/sqlite3.target.mk failedTask=build stackTrace=Error: Can't reconcile two non-macho files node_modules/better-sqlite3/build/deps/sqlite3.target.mk
at Object.exports.mergeASARs (/Users/oltreseba/Akiflow/akido/node_modules/@electron/universal/src/asar-utils.ts:151:13)
at exports.makeUniversalApp (/Users/oltreseba/Akiflow/akido/node_modules/@electron/universal/src/index.ts:200:13)
at MacPackager.doPack (/Users/oltreseba/Akiflow/akido/node_modules/app-builder-lib/src/macPackager.ts:125:9)
at MacPackager.pack (/Users/oltreseba/Akiflow/akido/node_modules/app-builder-lib/src/macPackager.ts:179:7)
at Packager.doBuild (/Users/oltreseba/Akiflow/akido/node_modules/app-builder-lib/src/packager.ts:441:9)
at Object.executeFinally (/Users/oltreseba/Akiflow/akido/node_modules/builder-util/src/promise.ts:12:14)
at Packager._build (/Users/oltreseba/Akiflow/akido/node_modules/app-builder-lib/src/packager.ts:376:31)
at Packager.build (/Users/oltreseba/Akiflow/akido/node_modules/app-builder-lib/src/packager.ts:337:12)
at Object.executeFinally (/Users/oltreseba/Akiflow/akido/node_modules/builder-util/src/promise.ts:12:14)
Everything was working fine until i upgraded electron builder from 22.14.13 to 23.0.2.
After excluding .mk
files from the build, i'm not having problem with a .a
file: (not sure if i will be fine in excluding this file).
⨯ Can't reconcile two non-macho files node_modules/better-sqlite3/build/Release/sqlite3.a failedTask=build stackTrace=Error: Can't reconcile two non-macho files node_modules/better-sqlite3/build/Release/sqlite3.a
at Object.exports.mergeASARs (/Users/oltreseba/Akiflow/akido/node_modules/@electron/universal/src/asar-utils.ts:151:13)
at exports.makeUniversalApp (/Users/oltreseba/Akiflow/akido/node_modules/@electron/universal/src/index.ts:200:13)
at MacPackager.doPack (/Users/oltreseba/Akiflow/akido/node_modules/app-builder-lib/src/macPackager.ts:125:9)
at MacPackager.pack (/Users/oltreseba/Akiflow/akido/node_modules/app-builder-lib/src/macPackager.ts:179:7)
at Packager.doBuild (/Users/oltreseba/Akiflow/akido/node_modules/app-builder-lib/src/packager.ts:441:9)
at Object.executeFinally (/Users/oltreseba/Akiflow/akido/node_modules/builder-util/src/promise.ts:12:14)
at Packager._build (/Users/oltreseba/Akiflow/akido/node_modules/app-builder-lib/src/packager.ts:376:31)
at Packager.build (/Users/oltreseba/Akiflow/akido/node_modules/app-builder-lib/src/packager.ts:337:12)
at Object.executeFinally (/Users/oltreseba/Akiflow/akido/node_modules/builder-util/src/promise.ts:12:14)
After excluding
.mk
files from the build, i'm not having problem with a.a
file: (not sure if i will be fine in excluding this file).⨯ Can't reconcile two non-macho files node_modules/better-sqlite3/build/Release/sqlite3.a failedTask=build stackTrace=Error: Can't reconcile two non-macho files node_modules/better-sqlite3/build/Release/sqlite3.a at Object.exports.mergeASARs (/Users/oltreseba/Akiflow/akido/node_modules/@electron/universal/src/asar-utils.ts:151:13) at exports.makeUniversalApp (/Users/oltreseba/Akiflow/akido/node_modules/@electron/universal/src/index.ts:200:13) at MacPackager.doPack (/Users/oltreseba/Akiflow/akido/node_modules/app-builder-lib/src/macPackager.ts:125:9) at MacPackager.pack (/Users/oltreseba/Akiflow/akido/node_modules/app-builder-lib/src/macPackager.ts:179:7) at Packager.doBuild (/Users/oltreseba/Akiflow/akido/node_modules/app-builder-lib/src/packager.ts:441:9) at Object.executeFinally (/Users/oltreseba/Akiflow/akido/node_modules/builder-util/src/promise.ts:12:14) at Packager._build (/Users/oltreseba/Akiflow/akido/node_modules/app-builder-lib/src/packager.ts:376:31) at Packager.build (/Users/oltreseba/Akiflow/akido/node_modules/app-builder-lib/src/packager.ts:337:12) at Object.executeFinally (/Users/oltreseba/Akiflow/akido/node_modules/builder-util/src/promise.ts:12:14)
@oltreseba, @indutny-signal, I fixed it without excluding files. In my package.json, I had custom additions to "files": ...
which disables the default inclusions according to the electron-builder documentation. Apparently, the mergeASARs
API requires some of those default files that were not in my custom list, so it failed to reconcile. Since the default for "files": ...
already included my custom additions, I simply deleted it. Problem fixed. Hopefully, this fixes it for you as well. If so, I'll close this issue.
Not really.
I'm using files
only to exclude additional files in my repository from being build. Also I'm trying to find the default value for files
in electron builder, but I cannot find anything. Can you point out the list of default values? - It would be of great help.
Thank you.
@oltreseba you might need to exclude everything from "!**/node_modules/better-sqlite3/build/**" and add back the
"node_modules/better-sqlite3/build/Release/better_sqlite3.node",
. Even without merging ASARs you really don't want to have these extra files around because they could take a lot of space.
Not really. I'm using
files
only to exclude additional files in my repository from being build. Also I'm trying to find the default value forfiles
in electron builder, but I cannot find anything. Can you point out the list of default values? - It would be of great help.Thank you.
@oltreseba, here's the link to that documentation.
Having the same issue
Can't reconcile two non-macho files node_modules/@serialport/bindings-cpp/build/Release/bindings.node failedTask=build stackTrace=Error: Can't reconcile two non-macho files node_modules/@serialport/bindings-cpp/build/Release/bindings.node
running file on the bindings.node returns this
node_modules/@serialport/bindings-cpp/build/Release/bindings.node: Mach-O universal binary with 2 architectures: [x86_64:Mach-O 64-bit bundle x86_64Mach-O 64-bit bundle x86_64] [arm64:Mach-O 64-bit bundle arm64Mach-O 64-bit bundle arm64]
node_modules/@serialport/bindings-cpp/build/Release/bindings.node (for architecture x86_64): Mach-O 64-bit bundle x86_64
node_modules/@serialport/bindings-cpp/build/Release/bindings.node (for architecture arm64): Mach-O 64-bit bundle arm64
Removing custom settings for the build files doesn't seem to change anything
I've got a similar issue, however with node-mac-permissions
package:
Can't reconcile two non-macho files node_modules/node-mac-permissions/build/Release/nothing.a failedTask=build stackTrace=Error: Can't reconcile two non-macho files node_modules/node-mac-permissions/build/Release/nothing.a
I can't ignore this directory in build.files
options, because runtime error appears:
Error: Cannot find module 'node-mac-permissions'
Rolling back to older electron-builder
version is not an option after upgrading to the latest MacOS
Can confirm the same issue with node-mac-permissions
with a small difference.
First trip up is on node_modules/node-mac-permissions/node-addon-api/nothing.target.mk
Adding this to files
in electron-builder
config fixes the problem for us: '!node_modules/**/*.{mk,a,o,h}'
It basically ignores all the C/C++ build files from packaging
I've got a similar issue, however with node-mac-permissions package:
Same with usb-detection
. The binary is already universal.
For now, I have to stay with version 22.xx.xxx Python 2 (can be downloaded from here: https://www.python.org/downloads/release/python-2718/)
Don't forget to tell electron-builder where to find it using the PYTHON_PATH
Variable:
export PYTHON_PATH=/usr/local/bin/python
yarn electron-builder
Edit 1: it also works with electron-builder v23, as long as you set "mergeASARs": false
. However, I'd really like to merge asars...
Edit 2: The binary of usb-detection
is not in mach-o format. That is why it does not work.
That means, it's not a bug of electron/universal
but rather of usb-detection.
Edit 3: It seems like the binary is in Mach-O format, but already contains both architectures:
hannes@macmini node-usb-detection % file build/Release/detection.node
build/Release/detection.node: Mach-O universal binary with 2 architectures: [x86_64:Mach-O 64-bot bundle x86_64Mach-O 64 bit undle x86_64] [arm64:Mach-O 64-bit bundle arm64Mach-O 64-bit bundle arm64]
build/Release/detection.node (for architecture x86_64): Mach-O 64-bit bundle x86_64
build/Release/detection.node (for architecture arm64): Mach-O 64-bit bundle arm64
Seems like electron/universal
can't process those binaries?
i have a similar issue to this, but with this error:
⨯ Can't reconcile two non-macho files node_modules/dtrace-provider/libusdt/libusdt.a failedTask=build stackTrac
any ideas?
Adding this to
files
inelectron-builder
config fixes the problem for us:'!node_modules/**/*.{mk,a,o,h}'
It basically ignores all the C/C++ build files from packaging
Did you try this? You probably don't need .a
files in your app.asar
as they are just temporary files of the build process.
In my case adding '!node_modules/**/*.{mk,a,o,h}'
just results in it failing on some other file, and when I got to the end of those it failed in another way.
Detected unique file "node_modules/argon2/bin/darwin-arm64-113" in "/Users/me/Documents/app/release/prod/mac-universal--arm64/app.app/Contents/Resources/app.asar" not covered by allowList rule: "undefined"
and then in Electron-Builder if I add that file to singleArchFiles
then I get:
Command failed: lipo /private/var/folders/y5/pp8xq1qd1f78jth_11nsxndh0000gn/T/arm64-mw1MJx/node_modules/argon2/bin/darwin-arm64-113/argon2.node /private/var/folders/y5/pp8xq1qd1f78jth_11nsxndh0000gn/T/x64-BD57LD/node_modules/argon2/bin/darwin-arm64-113/argon2.node -create -output /private/var/folders/y5/pp8xq1qd1f78jth_11nsxndh0000gn/T/x64-BD57LD/node_modules/argon2/bin/darwin-arm64-113/argon2.node
fatal error: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/lipo: /private/var/folders/y5/pp8xq1qd1f78jth_11nsxndh0000gn/T/arm64-mw1MJx/node_modules/argon2/bin/darwin-arm64-113/argon2.node and /private/var/folders/y5/pp8xq1qd1f78jth_11nsxndh0000gn/T/x64-BD57LD/node_modules/argon2/bin/darwin-arm64-113/argon2.node have the same architectures (arm64) and can't be in the same fat output file
failedTask=build stackTrace=Error: Command failed: lipo /private/var/folders/y5/pp8xq1qd1f78jth_11nsxndh0000gn/T/arm64-mw1MJx/node_modules/argon2/bin/darwin-arm64-113/argon2.node /private/var/folders/y5/pp8xq1qd1f78jth_11nsxndh0000gn/T/x64-BD57LD/node_modules/argon2/bin/darwin-arm64-113/argon2.node -create -output /private/var/folders/y5/pp8xq1qd1f78jth_11nsxndh0000gn/T/x64-BD57LD/node_modules/argon2/bin/darwin-arm64-113/argon2.node
fatal error: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/lipo: /private/var/folders/y5/pp8xq1qd1f78jth_11nsxndh0000gn/T/arm64-mw1MJx/node_modules/argon2/bin/darwin-arm64-113/argon2.node and /private/var/folders/y5/pp8xq1qd1f78jth_11nsxndh0000gn/T/x64-BD57LD/node_modules/argon2/bin/darwin-arm64-113/argon2.node have the same architectures (arm64) and can't be in the same fat output file
In this command from the logs above ☝️
Command failed: lipo /private/var/folders/y5/pp8xq1qd1f78jth_11nsxndh0000gn/T/arm64-mw1MJx/node_modules/argon2/bin/darwin-arm64-113/argon2.node /private/var/folders/y5/pp8xq1qd1f78jth_11nsxndh0000gn/T/x64-BD57LD/node_modules/argon2/bin/darwin-arm64-113/argon2.node -create -output /private/var/folders/y5/pp8xq1qd1f78jth_11nsxndh0000gn/T/x64-BD57LD/node_modules/argon2/bin/darwin-arm64-113/argon2.node
Both paths for lipo are the same.
/private/var/folders/y5/pp8xq1qd1f78jth_11nsxndh0000gn/T/arm64-mw1MJx/node_modules/argon2/bin/darwin-arm64-113/argon2.node
Note the arm64-113/argon2.node
in both /T/arm64-mw1MJx
and /T/x64-BD57LD
dirs
This seems like a bug in electron/universal? Why is lipo
being used on a single-arch file?
In this command from the logs above point_up
Command failed: lipo /private/var/folders/y5/pp8xq1qd1f78jth_11nsxndh0000gn/T/arm64-mw1MJx/node_modules/argon2/bin/darwin-arm64-113/argon2.node /private/var/folders/y5/pp8xq1qd1f78jth_11nsxndh0000gn/T/x64-BD57LD/node_modules/argon2/bin/darwin-arm64-113/argon2.node -create -output /private/var/folders/y5/pp8xq1qd1f78jth_11nsxndh0000gn/T/x64-BD57LD/node_modules/argon2/bin/darwin-arm64-113/argon2.node
Both paths for lipo are the same.
/private/var/folders/y5/pp8xq1qd1f78jth_11nsxndh0000gn/T/arm64-mw1MJx/node_modules/argon2/bin/darwin-arm64-113/argon2.node
Note thearm64-113/argon2.node
in both/T/arm64-mw1MJx
and/T/x64-BD57LD
dirs This seems like a bug in electron/universal? Why islipo
being used on a single-arch file?
What needs to be done to fix this? We have this same problem when building a universal app on M1 with electron-builder, and we're currently working around the issue by deleting some of the native libraries in beforePack
.
@dlon did you come across this issue? https://github.com/ranisalt/node-argon2/issues/374
I'm not sure this particular issue is with @electron/universal
In macOS 12.3, Apple removed Python 2. electron-builder v23.0.0 added Python 3 support to resolve the issue. However, electron-universal now fails to package the universal app.
Full Error Message:
Does electron-universal directly utilize Python 2?