indygreg / PyOxidizer

A modern Python application packaging and distribution tool
Mozilla Public License 2.0
5.32k stars 227 forks source link

Build failures with OSX 13.4.1 (22F82) #707

Open Cypher1 opened 1 year ago

Cypher1 commented 1 year ago

It seems there's some version mismatch between the SDK that I have (13.3) and the one that rust is linking to (11.0).

bug/deps/liboxidized_importer-b947611aed903dc4.dylib" "-Wl,-dead_strip" "-dynamiclib" "-Wl,-dylib" "-nodefaultlibs"
  = note: ld: warning: object file (/private/var/folders/j8/17r7x0c91_b50dh7l1jst1ph0000gn/T/pyoxidizeru9b3ge/build/target/aarch64-apple-darwin/debug/deps/libpyo3_ffi-b9ea064ef657ba4e.rlib(7f38341345f9c319-config.o)) was built for newer macOS version (13.0) than being linked (11.0)

Full command for cc

error: linking with `cc` failed: exit status: 1
  |
  = note: "cc" "-Wl,-exported_symbols_list,/var/folders/j8/17r7x0c91_b50dh7l1jst1ph0000gn/T/rustcYXGDD4/list" "-arch" "arm64" "/var/folders/j8/17r7x0c91_b50dh7l1jst1ph0000gn/T/rustcYXGDD4/symbols.o" "/var/folders/j8/17r7x0c91_b50dh7l1jst1ph0000gn/T/pyoxidizeru9b3ge/build/target/aarch64-apple-darwin/debug/deps/oxidized_importer-b947611aed903dc4.oxidized_importer.aa481229-cgu.0.rcgu.o" "/var/folders/j8/17r7x0c91_b50dh7l1jst1ph0000gn/T/pyoxidizeru9b3ge/build/target/aarch64-apple-darwin/debug/deps/oxidized_importer-b947611aed903dc4.oxidized_importer.aa481229-cgu.1.rcgu.o" "/var/folders/j8/17r7x0c91_b50dh7l1jst1ph0000gn/T/pyoxidizeru9b3ge/build/target/aarch64-apple-darwin/debug/deps/oxidized_importer-b947611aed903dc4.oxidized_importer.aa481229-cgu.10.rcgu.o" "/var/folders/j8/17r7x0c91_b50dh7l1jst1ph0000gn/T/pyoxidizeru9b3ge/build/target/aarch64-apple-darwin/debug/deps/oxidized_importer-b947611aed903dc4.oxidized_importer.aa481229-cgu.11.rcgu.o" "/var/folders/j8/17r7x0c91_b50dh7l1jst1ph0000gn/T/pyoxidizeru9b3ge/build/target/aarch64-apple-darwin/debug/deps/oxidized_importer-b947611aed903dc4.oxidized_importer.aa481229-cgu.12.rcgu.o" "/var/folders/j8/17r7x0c91_b50dh7l1jst1ph0000gn/T/pyoxidizeru9b3ge/build/target/aarch64-apple-darwin/debug/deps/oxidized_importer-b947611aed903dc4.oxidized_importer.aa481229-cgu.13.rcgu.o" "/var/folders/j8/17r7x0c91_b50dh7l1jst1ph0000gn/T/pyoxidizeru9b3ge/build/target/aarch64-apple-darwin/debug/deps/oxidized_importer-b947611aed903dc4.oxidized_importer.aa481229-cgu.14.rcgu.o" "/var/folders/j8/17r7x0c91_b50dh7l1jst1ph0000gn/T/pyoxidizeru9b3ge/build/target/aarch64-apple-darwin/debug/deps/oxidized_importer-b947611aed903dc4.oxidized_importer.aa481229-cgu.15.rcgu.o" "/var/folders/j8/17r7x0c91_b50dh7l1jst1ph0000gn/T/pyoxidizeru9b3ge/build/target/aarch64-apple-darwin/debug/deps/oxidized_importer-b947611aed903dc4.oxidized_importer.aa481229-cgu.2.rcgu.o" "/var/folders/j8/17r7x0c91_b50dh7l1jst1ph0000gn/T/pyoxidizeru9b3ge/build/target/aarch64-apple-darwin/debug/deps/oxidized_importer-b947611aed903dc4.oxidized_importer.aa481229-cgu.3.rcgu.o" "/var/folders/j8/17r7x0c91_b50dh7l1jst1ph0000gn/T/pyoxidizeru9b3ge/build/target/aarch64-apple-darwin/debug/deps/oxidized_importer-b947611aed903dc4.oxidized_importer.aa481229-cgu.4.rcgu.o" "/var/folders/j8/17r7x0c91_b50dh7l1jst1ph0000gn/T/pyoxidizeru9b3ge/build/target/aarch64-apple-darwin/debug/deps/oxidized_importer-b947611aed903dc4.oxidized_importer.aa481229-cgu.5.rcgu.o" "/var/folders/j8/17r7x0c91_b50dh7l1jst1ph0000gn/T/pyoxidizeru9b3ge/build/target/aarch64-apple-darwin/debug/deps/oxidized_importer-b947611aed903dc4.oxidized_importer.aa481229-cgu.6.rcgu.o" "/var/folders/j8/17r7x0c91_b50dh7l1jst1ph0000gn/T/pyoxidizeru9b3ge/build/target/aarch64-apple-darwin/debug/deps/oxidized_importer-b947611aed903dc4.oxidized_importer.aa481229-cgu.7.rcgu.o" "/var/folders/j8/17r7x0c91_b50dh7l1jst1ph0000gn/T/pyoxidizeru9b3ge/build/target/aarch64-apple-darwin/debug/deps/oxidized_importer-b947611aed903dc4.oxidized_importer.aa481229-cgu.8.rcgu.o" "/var/folders/j8/17r7x0c91_b50dh7l1jst1ph0000gn/T/pyoxidizeru9b3ge/build/target/aarch64-apple-darwin/debug/deps/oxidized_importer-b947611aed903dc4.oxidized_importer.aa481229-cgu.9.rcgu.o" "/var/folders/j8/17r7x0c91_b50dh7l1jst1ph0000gn/T/pyoxidizeru9b3ge/build/target/aarch64-apple-darwin/debug/deps/oxidized_importer-b947611aed903dc4.1dhxsd6k9deaxyoj.rcgu.o" "-L" "/var/folders/j8/17r7x0c91_b50dh7l1jst1ph0000gn/T/pyoxidizeru9b3ge/build/target/aarch64-apple-darwin/debug/deps" "-L" "/var/folders/j8/17r7x0c91_b50dh7l1jst1ph0000gn/T/pyoxidizeru9b3ge/build/target/debug/deps" "-L" "/var/folders/j8/17r7x0c91_b50dh7l1jst1ph0000gn/T/pyoxidizeru9b3ge/artifacts" "-L" "/Library/Developer/CommandLineTools/usr/lib/clang/14.0.3/lib/darwin" "-L" "/Users/jaypratt/Library/Caches/pyoxidizer/python_distributions/python.5a4a1128f6aa/python/build/lib" "-L" "/Users/jaypratt/Library/Caches/pyoxidizer/rust/1.66.0-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib" "/private/var/folders/j8/17r7x0c91_b50dh7l1jst1ph0000gn/T/pyoxidizeru9b3ge/build/target/aarch64-apple-darwin/debug/deps/libmemmap2-ed6f62524d0ecc4b.rlib" "/private/var/folders/j8/17r7x0c91_b50dh7l1jst1ph0000gn/T/pyoxidizeru9b3ge/build/target/aarch64-apple-darwin/debug/deps/libpython_packaging-79baa8f67bdb11f4.rlib" "/private/var/folders/j8/17r7x0c91_b50dh7l1jst1ph0000gn/T/pyoxidizeru9b3ge/build/target/aarch64-apple-darwin/debug/deps/libregex-8b63c8e8c4e9d7e2.rlib" "/private/var/folders/j8/17r7x0c91_b50dh7l1jst1ph0000gn/T/pyoxidizeru9b3ge/build/target/aarch64-apple-darwin/debug/deps/libaho_corasick-caef6706df54e679.rlib" "/private/var/folders/j8/17r7x0c91_b50dh7l1jst1ph0000gn/T/pyoxidizeru9b3ge/build/target/aarch64-apple-darwin/debug/deps/libmemchr-c864f76f4c086c87.rlib" "/private/var/folders/j8/17r7x0c91_b50dh7l1jst1ph0000gn/T/pyoxidizeru9b3ge/build/target/aarch64-apple-darwin/debug/deps/libregex_syntax-bf95dd28df509bc8.rlib" "/private/var/folders/j8/17r7x0c91_b50dh7l1jst1ph0000gn/T/pyoxidizeru9b3ge/build/target/aarch64-apple-darwin/debug/deps/libitertools-be943e1ef425854c.rlib" "/private/var/folders/j8/17r7x0c91_b50dh7l1jst1ph0000gn/T/pyoxidizeru9b3ge/build/target/aarch64-apple-darwin/debug/deps/libeither-90f3d4cd3781af4d.rlib" "/private/var/folders/j8/17r7x0c91_b50dh7l1jst1ph0000gn/T/pyoxidizeru9b3ge/build/target/aarch64-apple-darwin/debug/deps/libwalkdir-d062cc6129fcc2a0.rlib" "/private/var/folders/j8/17r7x0c91_b50dh7l1jst1ph0000gn/T/pyoxidizeru9b3ge/build/target/aarch64-apple-darwin/debug/deps/libsame_file-0d1003228f6da64f.rlib" "/private/var/folders/j8/17r7x0c91_b50dh7l1jst1ph0000gn/T/pyoxidizeru9b3ge/build/target/aarch64-apple-darwin/debug/deps/libpython_packed_resources-05068c704eab258f.rlib" "/private/var/folders/j8/17r7x0c91_b50dh7l1jst1ph0000gn/T/pyoxidizeru9b3ge/build/target/aarch64-apple-darwin/debug/deps/libonce_cell-72f57b509aa55247.rlib" "/private/var/folders/j8/17r7x0c91_b50dh7l1jst1ph0000gn/T/pyoxidizeru9b3ge/build/target/aarch64-apple-darwin/debug/deps/libmailparse-a9d49697a2d429d8.rlib" "/private/var/folders/j8/17r7x0c91_b50dh7l1jst1ph0000gn/T/pyoxidizeru9b3ge/build/target/aarch64-apple-darwin/debug/deps/libquoted_printable-da60606e3a2730c6.rlib" "/private/var/folders/j8/17r7x0c91_b50dh7l1jst1ph0000gn/T/pyoxidizeru9b3ge/build/target/aarch64-apple-darwin/debug/deps/libdata_encoding-47b315f0e4d1cf88.rlib" "/private/var/folders/j8/17r7x0c91_b50dh7l1jst1ph0000gn/T/pyoxidizeru9b3ge/build/target/aarch64-apple-darwin/debug/deps/libcharset-54b44ea771cb227a.rlib" "/private/var/folders/j8/17r7x0c91_b50dh7l1jst1ph0000gn/T/pyoxidizeru9b3ge/build/target/aarch64-apple-darwin/debug/deps/libencoding_rs-a827c6503a144bae.rlib" "/private/var/folders/j8/17r7x0c91_b50dh7l1jst1ph0000gn/T/pyoxidizeru9b3ge/build/target/aarch64-apple-darwin/debug/deps/libbase64-367484e5ad8172aa.rlib" "/private/var/folders/j8/17r7x0c91_b50dh7l1jst1ph0000gn/T/pyoxidizeru9b3ge/build/target/aarch64-apple-darwin/debug/deps/libspdx-fcab34d6a5fd5900.rlib" "/private/var/folders/j8/17r7x0c91_b50dh7l1jst1ph0000gn/T/pyoxidizeru9b3ge/build/target/aarch64-apple-darwin/debug/deps/libsimple_file_manifest-c9552dda4fa9359d.rlib" "/private/var/folders/j8/17r7x0c91_b50dh7l1jst1ph0000gn/T/pyoxidizeru9b3ge/build/target/aarch64-apple-darwin/debug/deps/libbyteorder-94e1b18f73c98265.rlib" "/private/var/folders/j8/17r7x0c91_b50dh7l1jst1ph0000gn/T/pyoxidizeru9b3ge/build/target/aarch64-apple-darwin/debug/deps/libanyhow-3ce6ea14ecec688b.rlib" "/private/var/folders/j8/17r7x0c91_b50dh7l1jst1ph0000gn/T/pyoxidizeru9b3ge/build/target/aarch64-apple-darwin/debug/deps/libpyo3-297cd7a87e8f6bdb.rlib" "/private/var/folders/j8/17r7x0c91_b50dh7l1jst1ph0000gn/T/pyoxidizeru9b3ge/build/target/aarch64-apple-darwin/debug/deps/libmemoffset-c7ed583fd35b5a34.rlib" "/private/var/folders/j8/17r7x0c91_b50dh7l1jst1ph0000gn/T/pyoxidizeru9b3ge/build/target/aarch64-apple-darwin/debug/deps/libparking_lot-2d035c583a202e54.rlib" "/private/var/folders/j8/17r7x0c91_b50dh7l1jst1ph0000gn/T/pyoxidizeru9b3ge/build/target/aarch64-apple-darwin/debug/deps/libparking_lot_core-8050bbb27c295dc9.rlib" "/private/var/folders/j8/17r7x0c91_b50dh7l1jst1ph0000gn/T/pyoxidizeru9b3ge/build/target/aarch64-apple-darwin/debug/deps/libcfg_if-3160d05323bb6c79.rlib" "/private/var/folders/j8/17r7x0c91_b50dh7l1jst1ph0000gn/T/pyoxidizeru9b3ge/build/target/aarch64-apple-darwin/debug/deps/libsmallvec-9a440c52bc78f3b8.rlib" "/private/var/folders/j8/17r7x0c91_b50dh7l1jst1ph0000gn/T/pyoxidizeru9b3ge/build/target/aarch64-apple-darwin/debug/deps/liblock_api-299687d350416891.rlib" "/private/var/folders/j8/17r7x0c91_b50dh7l1jst1ph0000gn/T/pyoxidizeru9b3ge/build/target/aarch64-apple-darwin/debug/deps/libscopeguard-90f94266d59b539c.rlib" "/private/var/folders/j8/17r7x0c91_b50dh7l1jst1ph0000gn/T/pyoxidizeru9b3ge/build/target/aarch64-apple-darwin/debug/deps/libpyo3_ffi-b9ea064ef657ba4e.rlib" "/private/var/folders/j8/17r7x0c91_b50dh7l1jst1ph0000gn/T/pyoxidizeru9b3ge/build/target/aarch64-apple-darwin/debug/deps/liblibc-c402e60045511443.rlib" "/private/var/folders/j8/17r7x0c91_b50dh7l1jst1ph0000gn/T/pyoxidizeru9b3ge/build/target/aarch64-apple-darwin/debug/deps/libunindent-032f2738192bf225.rlib" "/Users/jaypratt/Library/Caches/pyoxidizer/rust/1.66.0-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libstd-8b48f6cefff7b1fb.rlib" "/Users/jaypratt/Library/Caches/pyoxidizer/rust/1.66.0-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libpanic_unwind-77aeaad8f850e496.rlib" "/Users/jaypratt/Library/Caches/pyoxidizer/rust/1.66.0-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libobject-4374cc9e76b48de5.rlib" "/Users/jaypratt/Library/Caches/pyoxidizer/rust/1.66.0-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libmemchr-71ce82846aaede9b.rlib" "/Users/jaypratt/Library/Caches/pyoxidizer/rust/1.66.0-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libaddr2line-88478a43231eec41.rlib" "/Users/jaypratt/Library/Caches/pyoxidizer/rust/1.66.0-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libgimli-219a0c42f67a2534.rlib" "/Users/jaypratt/Library/Caches/pyoxidizer/rust/1.66.0-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/librustc_demangle-967edde4cff44b15.rlib" "/Users/jaypratt/Library/Caches/pyoxidizer/rust/1.66.0-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libstd_detect-5e9bad2c7ec1f53a.rlib" "/Users/jaypratt/Library/Caches/pyoxidizer/rust/1.66.0-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libcfg_if-f2d3de7828f2e1f2.rlib" "/Users/jaypratt/Library/Caches/pyoxidizer/rust/1.66.0-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libhashbrown-39feb0179b2f197f.rlib" "/Users/jaypratt/Library/Caches/pyoxidizer/rust/1.66.0-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libminiz_oxide-fe7523d51d4e2154.rlib" "/Users/jaypratt/Library/Caches/pyoxidizer/rust/1.66.0-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libadler-1fb439e2b08a54ae.rlib" "/Users/jaypratt/Library/Caches/pyoxidizer/rust/1.66.0-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/librustc_std_workspace_alloc-0ece1873dc3050cd.rlib" "/Users/jaypratt/Library/Caches/pyoxidizer/rust/1.66.0-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libunwind-1094a874bdfc7dbb.rlib" "/Users/jaypratt/Library/Caches/pyoxidizer/rust/1.66.0-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libcfg_if-8cd76113b92d6775.rlib" "/Users/jaypratt/Library/Caches/pyoxidizer/rust/1.66.0-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/liblibc-3a7cb21921429c0e.rlib" "/Users/jaypratt/Library/Caches/pyoxidizer/rust/1.66.0-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/liballoc-5b42fb3902d391c7.rlib" "/Users/jaypratt/Library/Caches/pyoxidizer/rust/1.66.0-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/librustc_std_workspace_core-b495777441600026.rlib" "/Users/jaypratt/Library/Caches/pyoxidizer/rust/1.66.0-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libcore-cba95c4386a21469.rlib" "/Users/jaypratt/Library/Caches/pyoxidizer/rust/1.66.0-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libcompiler_builtins-b8f163a13cf94b5d.rlib" "-framework" "CoreFoundation" "-ldl" "-lclang_rt.osx" "-liconv" "-lSystem" "-lc" "-lm" "-L" "/Users/jaypratt/Library/Caches/pyoxidizer/rust/1.66.0-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib" "-o" "/var/folders/j8/17r7x0c91_b50dh7l1jst1ph0000gn/T/pyoxidizeru9b3ge/build/target/aarch64-apple-darwin/debug/deps/liboxidized_importer-b947611aed903dc4.dylib" "-Wl,-dead_strip" "-dynamiclib" "-Wl,-dylib" "-nodefaultlibs"

Added full logfile as an attachment. logfile.txt

I've tried a few combinations of the following env variables with no success (and even managed to get rustc to fail to find core in some instances.

MACOSX_DEPLOYMENT_TARGET=13.3
SDKROOT=/Library/Developer/CommandLineTools/SDKs/MacOSX13.3.sdk
STATIC_DEPS=true
RUSTFLAGS="-C link-args='--verbose --sysroot /Library/Developer/CommandLineTools/SDKs/MacOSX13.3.sdk'"
time pyoxidizer build --verbose 2>&1 | tee logfile

Is my setup misconfigured? Am I holding it wrong?

Thanks in advance

dae commented 1 year ago

Tried setting the deployment target to 11?

Cypher1 commented 1 year ago

Yes, sorry I didn't mention all the env vars above that mention 13.3 I've also tried with 11.0 (in multiple, though probably not all combinations).

It looks like the cc command doesn't pick up the MACOSX_DEPLOYMENT_TARGET properly and builds for (rather than just with) 13.3 (the newest SDK I have installed).

I haven't managed to install the Rust core libraries built with 11.0 though, so setting everything to 11.0 results in core missing.

Currently have checked out the pyoxidizer source to see if I can override the TARGET for cc there to make sure that is actually the problem.

dae commented 1 year ago

Tried clearing out your Rust compilation cache after setting the target to 11? I wonder if pyo3 declares it as something that should trigger a rebuild.

We use pyoxidizer in a slightly non-standard way so we can control the Rust program that is generated, and a deployment target of 11 works there, so there's a reasonable chance it would work in a standard build too.

Cypher1 commented 1 year ago

Thanks. I tried that with the following attempts at cache clearing and

MACOSX_DEPLOYMENT_TARGET=11.0 \
SDKROOT=/Library/Developer/CommandLineTools/SDKs/MacOSX11.0.sdk \
STATIC_DEPS=true \
time pyoxidizer build --verbose 2>&1 | tee logfile

I'm using static deps because otherwise I get a bunch of errors about xslt...

Cache clearing:

Any other caches around the place?

I'm wondering if it's actually the pyoxidizer built with pyoxide. I'll try building pyoxidizer from source again with the MACOSX_DEPLOYMENT_TARGET=11.0

dae commented 1 year ago

Not sure I'm afraid. In case it's helpful, this is how we build: https://github.com/ankitects/anki/blob/main/build/runner/src/bundle/binary.rs builds https://github.com/ankitects/anki/tree/main/qt/bundle. It uses a slightly patched version of PyOxidizer for #466.

Cypher1 commented 1 year ago

Darn, still trying things. Built pyoxidizer from scratch targeting 11.0.

  = note: ld: in /private/var/folders/j8/17r7x0c91_b50dh7l1jst1ph0000gn/T/pyoxidizerJvWGUn/build/target/aarch64-apple-darwin/debug/deps/libpyo3_ffi-0c5d1fc6c366c825.rlib(libpython.127.o), archive member 'libpython.127.o' with length 1907936 is not mach-o or llvm bitcode file '/private/var/folders/j8/17r7x0c91_b50dh7l1jst1ph0000gn/T/pyoxidizerJvWGUn/build/target/aarch64-apple-darwin/debug/deps/libpyo3_ffi-0c5d1fc6c366c825.rlib' for architecture arm64

I'll try to learn from your patch. (Much appreciated)

Cypher1 commented 1 year ago

logfile.txt Progress via building pyoxidizer for 11.0 and the project for 13.3

Not sure what the pip error is but I'm hoping it's just relative paths.