M1 support broken since v8.0.1

[Kikobeats]

Hello,

I'm using this library at https://github.com/microlinkhq/ffprobe

I noted some of the last versions are missing; also, the latest version published is broken in some way I not fully understand.

Here is the way to reproduce the problem:

npm i lzma-native@8.0.5 && npm run postinstall

up to date in 874ms

> @microlink/ffprobe@1.0.2 postinstall
> node scripts/postinstall.js

dyld[52834]: missing symbol called
fish: Job 1, 'npm run postinstall' terminated by signal SIGABRT (Abort)

~/Projects/microlink/ffprobe master*
❯ npm i lzma-native@8.0.4 && npm run postinstall
npm ERR! code ETARGET
npm ERR! notarget No matching version found for lzma-native@8.0.4.
npm ERR! notarget In most cases you or one of your dependencies are requesting
npm ERR! notarget a package version that doesn't exist.

npm ERR! A complete log of this run can be found in:
npm ERR!     /Users/kikobeats/.npm/_logs/2022-01-17T11_47_43_916Z-debug.log

~/Projects/microlink/ffprobe master*
❯ npm i lzma-native@8.0.3 && npm run postinstall
npm ERR! code ETARGET
npm ERR! notarget No matching version found for lzma-native@8.0.3.
npm ERR! notarget In most cases you or one of your dependencies are requesting
npm ERR! notarget a package version that doesn't exist.

npm ERR! A complete log of this run can be found in:
npm ERR!     /Users/kikobeats/.npm/_logs/2022-01-17T11_47_50_145Z-debug.log

~/Projects/microlink/ffprobe master*
❯ npm i lzma-native@8.0.2 && npm run postinstall
npm ERR! code ETARGET
npm ERR! notarget No matching version found for lzma-native@8.0.2.
npm ERR! notarget In most cases you or one of your dependencies are requesting
npm ERR! notarget a package version that doesn't exist.

npm ERR! A complete log of this run can be found in:
npm ERR!     /Users/kikobeats/.npm/_logs/2022-01-17T11_47_53_700Z-debug.log

~/Projects/microlink/ffprobe master*
❯ npm i lzma-native@8.0.1 && npm run postinstall

changed 1 package in 19s

> @microlink/ffprobe@1.0.2 postinstall
> node scripts/postinstall.js

# exit with 0 in this case, all fine

[addaleax]

@Kikobeats Which exact OS is this (platform + OS version + architecture)?

[Kikobeats]

npx envinfo --binaries --system

  System:
    OS: macOS 12.0.1
    CPU: (10) arm64 Apple M1 Pro
    Memory: 477.34 MB / 32.00 GB
    Shell: 3.3.1 - /opt/homebrew/bin/fish
  Binaries:
    Node: 16.13.1 - ~/.local/share/nvm/v16.13.1/bin/node
    Yarn: 1.22.17 - /opt/homebrew/bin/yarn
    npm: 8.1.2 - ~/.local/share/nvm/v16.13.1/bin/npm

[addaleax]

@beorn @tylinux @malept any ideas? this is happening specifically on Apple Silicon, and I don’t have a local setup to test things out there

[beorn]

Running yarn in ffprobe also doesn't work for me - the lzma.Decompressor() fails:

% yarn node scripts/postinstall.js
dyld[89886]: missing symbol called
error Command failed.
Exit signal: SIGABRT

Also:

% cd node_modules/lzma-native
% grep version package.json                         
  "version": "8.0.5",
% # copy test files into test
% yarn test
 yarn run v1.22.17
 $ mocha --expose-gc -s 1000 -t 5000

  Compressor/Decompressor
dyld[90388]: missing symbol called
error Command failed with signal "SIGABRT".

Maybe there's something weird about the prebuild?

[tylinux]

Run yarn node-gyp-build and copy build/Release/lzma_native.node to demo project 's node_modules/lzma-native/prebuilds/darwin-arm64/node.napi.node, works good. Good file's size is 416KB, bad's is 228KB. Maybe there's something weird about the prebuild.

[addaleax]

Looks like this might not have fixed the issue entirely? https://github.com/addaleax/lzma-native/issues/129#issuecomment-1015521980

[Kikobeats]

In case you want to replicate in your end, if you have a M1 laptop just:

1) git clone https://github.com/microlinkhq/ffprobe 2) npm install

and you can see the crash

[tylinux]

I think the problem is ARM64's binary file is build from x86_64 host...

[malept]

If cross compilation is a problem then that is extremely problematic, no CI provider I'm aware of currently has macOS/arm64 builders.

[tmacwill]

First, thanks for all of your hard work on this library! I'm also trying to use this on an M1 Mac, and here's what I'm seeing.

When I clone this repo and just run yarn, everything builds without errors. When I try to build an x64 version (in order to ship a desktop application that's compatible on non-M1 Macs) via the command:

npm_config_arch=x64 yarn

I get the below:

ld: warning: ignoring file /Users/tmac/lzma-native/build/liblzma/build/lib/liblzma.a, building for macOS-x86_64 but attempting to link with file built for macOS-arm64

The same occurs with --target_arch=x64. This suggests that the underlying liblzma build isn't using the npm_config_arch environment variable to set the architecture (maybe it's just using the host machine architecture instead?), and the errors here are due to mixing architectures.

I'm happy to help out with a fix here, just wasn't 100% sure where to start. Also happy to try things on my M1 machine if that'd be helpful as well.

Finally, here's my system:

  System:
    OS: macOS 11.4
    CPU: (8) arm64 Apple M1
    Memory: 119.73 MB / 8.00 GB
    Shell: 5.8 - /bin/zsh
  Binaries:
    Node: 16.13.1 - /opt/homebrew/Cellar/node@16/16.13.1/bin/node
    Yarn: 1.22.17 - /usr/local/bin/yarn
    npm: 8.1.2 - /opt/homebrew/Cellar/node@16/16.13.1/bin/npm

[amatosov-rbx]

I am seeing the same issue and would like to help finding the solution

~~I see the commit 8af909b0586730fa7c8ab4162f8569409f97ecf4 that added conditional patching to liblzma-config.sh However the condition in it going to pass only for native builds if [ $(uname) = "Darwin" -a $(uname -m) = "arm64" ]; then and will be skipped for cross builds~~

Edit: Tried that in a fork, it did not fix the issue

[matafonoff]

Any updates on this issue?

[H4kt]

Is there any chance of this getting fixed?

[carlocorradini]

Any update? 😥

[meichthys]

I'm also seeing this on an M1 mac running Asahi Linux Fedora.