search

PyO3 / maturin

Build and publish crates with pyo3, cffi and uniffi bindings as well as rust binaries as python packages
https://maturin.rs
Apache License 2.0
3.96k stars 275 forks source link

cross compile from linux to win produces a wheel that crashes silently at runtime #2296

Closed oscar6echo closed 1 week ago

oscar6echo commented 1 week ago

Bug Description

The cross compile from linux to windows of package polars_plugin_option_pricing produces a wheel.
This wheel can be pip installed on windows.
So far no apparent error or warning.

But at runtime, the program exits silently without error message when reaching the compiled module: polars_plugin_option_pricing._rust.

I tried to build the wheel on a linux host (ubuntu) and from inside a docker container (reproducible below).
I build the docker container on a ubuntu host and a redhat8 host.
Same result :-(

How to debug this ?
Is there a tested template of docker based cross compilation from linux to win somewhere in the docs ?

Note: the native and manylinux compilations work and produce working wheels.

Your maturin version (maturin --version)

1.7.4

Your Python version (python -V)

3.12

Your pip version (pip -V)

24.3.1

What bindings you're using

pyo3

Does cargo build work?

If on windows, have you checked that you aren't accidentally using unix path (those with the forward slash /)?

Steps to Reproduce

Commands on linux host:

git clone https://github.com/oscar6echo/polars-plugin-option-pricing.git
cd polars-plugin-option-pricing
docker build -t builder-win:local -f ./win.Dockerfile .
docker run --rm -v "$(pwd)":/io builder-win:local

docker run logs:

# from repo root
❯ docker run --rm -v "$(pwd)":/io builder-win:local      
    Updating crates.io index
 Downloading crates ...
  Downloaded serde v1.0.214
  Downloaded strum_macros v0.26.4
  Downloaded wasi v0.11.0+wasi-snapshot-preview1
  Downloaded compact_str v0.8.0
  Downloaded cc v1.1.31
  Downloaded castaway v0.2.3
  Downloaded bytes v1.8.0
  Downloaded bytemuck_derive v1.5.0
  Downloaded bytemuck v1.14.3
  Downloaded bitflags v2.4.2
  Downloaded memoffset v0.9.0
  Downloaded atoi_simd v0.15.6
  Downloaded array-init-cursor v0.2.0
  Downloaded argminmax v0.6.2
  Downloaded windows-targets v0.52.6
  Downloaded windows-targets v0.48.5
  Downloaded aho-corasick v1.1.2
  Downloaded windows_aarch64_gnullvm v0.48.5
  Downloaded syn v2.0.87
  Downloaded windows_aarch64_gnullvm v0.52.6
  Downloaded redox_syscall v0.4.1
  Downloaded regex v1.10.3
  Downloaded iter-read v0.3.1
  Downloaded windows_i686_gnullvm v0.52.6
  Downloaded polars-core v0.43.1
  Downloaded polars-arrow v0.43.1
  Downloaded windows_x86_64_gnullvm v0.48.5
  Downloaded libc v0.2.153
  Downloaded windows_x86_64_gnullvm v0.52.6
  Downloaded fast-float v0.2.0
  Downloaded regex-syntax v0.8.5
  Downloaded zerocopy-derive v0.7.32
  Downloaded zerocopy v0.7.32
  Downloaded version_check v0.9.4
  Downloaded scopeguard v1.2.0
  Downloaded windows_i686_gnu v0.48.5
  Downloaded windows_i686_gnu v0.52.6
  Downloaded windows_i686_msvc v0.52.6
  Downloaded windows_x86_64_gnu v0.48.5
  Downloaded windows_x86_64_msvc v0.48.5
  Downloaded windows_x86_64_msvc v0.52.6
  Downloaded xxhash-rust v0.8.10
  Downloaded wasm-bindgen-shared v0.2.91
  Downloaded wasm-bindgen-macro-support v0.2.91
  Downloaded wasm-bindgen-macro v0.2.91
  Downloaded wasm-bindgen-backend v0.2.91
  Downloaded wasm-bindgen v0.2.91
  Downloaded unindent v0.2.3
  Downloaded unicode-ident v1.0.12
  Downloaded thiserror-impl v2.0.0
  Downloaded thiserror-impl v1.0.57
  Downloaded thiserror v2.0.0
  Downloaded thiserror v1.0.57
  Downloaded target-lexicon v0.12.16
  Downloaded strength_reduce v0.2.4
  Downloaded streaming-iterator v0.1.9
  Downloaded streaming-decompression v0.1.2
  Downloaded stacker v0.1.15
  Downloaded smallvec v1.13.1
  Downloaded serde-pickle v1.1.1
  Downloaded regex-automata v0.4.5
  Downloaded simdutf8 v0.1.4
  Downloaded ryu v1.0.17
  Downloaded rustversion v1.0.14
  Downloaded windows_x86_64_gnu v0.52.6
  Downloaded windows_i686_msvc v0.48.5
  Downloaded windows_aarch64_msvc v0.52.6
  Downloaded windows_aarch64_msvc v0.48.5
  Downloaded winapi v0.3.9
  Downloaded crossbeam-utils v0.8.20
  Downloaded polars-ffi v0.43.1
  Downloaded option-pricing v0.1.4
  Downloaded multiversion-macros v0.7.3
  Downloaded recursive-proc-macro-impl v0.1.1
  Downloaded recursive v0.1.1
  Downloaded rayon v1.9.0
  Downloaded raw-cpuid v11.0.1
  Downloaded python3-dll-a v0.2.9
  Downloaded pyo3-macros-backend v0.22.5
  Downloaded pyo3-macros v0.22.5
  Downloaded pyo3-polars-derive v0.11.0
  Downloaded pyo3-polars v0.17.0
  Downloaded pyo3-ffi v0.22.5
  Downloaded pyo3-build-config v0.22.5
  Downloaded multiversion v0.7.3
  Downloaded pyo3 v0.22.5
  Downloaded psm v0.1.23
  Downloaded portable-atomic v1.9.0
  Downloaded polars-utils v0.43.1
  Downloaded polars-plan v0.43.1
  Downloaded polars-compute v0.43.1
  Downloaded itertools v0.13.0
  Downloaded hashbrown v0.14.5
  Downloaded rand_core v0.6.4
  Downloaded quote v1.0.35
  Downloaded winapi-i686-pc-windows-gnu v0.4.0
  Downloaded ppv-lite86 v0.2.17
  Downloaded polars-schema v0.43.1
  Downloaded polars-row v0.43.1
  Downloaded polars-ops v0.43.1
  Downloaded polars-mem-engine v0.43.1
  Downloaded polars-lazy v0.43.1
  Downloaded polars-io v0.43.1
  Downloaded polars-expr v0.43.1
  Downloaded bumpalo v3.15.1
  Downloaded polars-error v0.43.1
  Downloaded polars-arrow-format v0.1.0
  Downloaded planus v0.3.1
  Downloaded parquet-format-safe v0.2.4
  Downloaded parking_lot_core v0.9.9
  Downloaded parking_lot v0.12.1
  Downloaded winapi-x86_64-pc-windows-gnu v0.4.0
  Downloaded num-traits v0.2.18
  Downloaded num-integer v0.1.46
  Downloaded polars v0.43.1
  Downloaded memmap2 v0.7.1
  Downloaded memchr v2.7.1
  Downloaded log v0.4.20
  Downloaded lock_api v0.4.11
  Downloaded libm v0.2.8
  Downloaded js-sys v0.3.68
  Downloaded polars-parquet v0.43.1
  Downloaded itoa v1.0.10
  Downloaded indoc v2.0.4
  Downloaded heck v0.5.0
  Downloaded glob v0.3.1
  Downloaded windows-sys v0.52.0
  Downloaded target-features v0.1.5
  Downloaded getrandom v0.2.12
  Downloaded fallible-streaming-iterator v0.1.9
  Downloaded ethnum v1.5.0
  Downloaded either v1.13.0
  Downloaded dyn-clone v1.0.16
  Downloaded crossbeam-epoch v0.9.18
  Downloaded crossbeam-deque v0.8.5
  Downloaded chrono v0.4.34
  Downloaded byteorder v1.5.0
  Downloaded syn v1.0.109
  Downloaded bitflags v1.3.2
  Downloaded base64 v0.22.1
  Downloaded allocator-api2 v0.2.16
  Downloaded ahash v0.8.9
  Downloaded static_assertions v1.1.0
  Downloaded shlex v1.3.0
  Downloaded serde_derive v1.0.214
  Downloaded rayon-core v1.12.1
  Downloaded rand_distr v0.4.3
  Downloaded rand_chacha v0.3.1
  Downloaded rand v0.8.5
  Downloaded proc-macro2 v1.0.89
  Downloaded percent-encoding v2.3.1
  Downloaded once_cell v1.19.0
  Downloaded num-bigint v0.4.4
  Downloaded indexmap v2.2.3
  Downloaded home v0.5.9
  Downloaded equivalent v1.0.1
  Downloaded cfg-if v1.0.0
  Downloaded autocfg v1.1.0
🍹 Building a mixed python/rust project
🔗 Found pyo3 bindings with abi3 support for Python ≥ 3.8
🐍 Not using a specific python interpreter (Automatically generating windows import library)
   Compiling pyo3-build-config v0.22.5
   Compiling pyo3-ffi v0.22.5
   Compiling pyo3-macros-backend v0.22.5
   Compiling pyo3 v0.22.5
   Compiling pyo3-macros v0.22.5
   Compiling pyo3-polars v0.17.0
   Compiling polars-plugin-option-pricing v0.1.0 (/io)
    Finished `release` profile [optimized] target(s) in 1m 24s
📦 Including files matching "src/**/*.rs"
📦 Built wheel for abi3 Python ≥ 3.8 to /io/dist/docker/polars_plugin_option_pricing-0.1.0-cp38-abi3-win_amd64.whl

Commands on win host, in conda env with py3.12:

pip install polars_plugin_option_pricing-0.1.0-cp38-abi3-win_amd64.whl

# from repo root
python test-run-bs.py

This command outputs:

polars_plugin_option_pricing version: 0.1.0
START

LAP init: 0.019 s
shape: (10_000_000, 7)
┌─────────┬───────┬────────┬──────┬─────┬──────┬──────┐
│ is_call ┆ spot  ┆ strike ┆ mat  ┆ vol ┆ rate ┆ div  │
│ ---     ┆ ---   ┆ ---    ┆ ---  ┆ --- ┆ ---  ┆ ---  │
│ bool    ┆ f32   ┆ f32    ┆ f32  ┆ f32 ┆ f32  ┆ f32  │
╞═════════╪═══════╪════════╪══════╪═════╪══════╪══════╡
│ true    ┆ 80.0  ┆ 80.0   ┆ 1.0  ┆ 0.1 ┆ 0.0  ┆ 0.0  │
│ true    ┆ 80.0  ┆ 80.0   ┆ 1.0  ┆ 0.1 ┆ 0.0  ┆ 0.01 │
│ true    ┆ 80.0  ┆ 80.0   ┆ 1.0  ┆ 0.1 ┆ 0.0  ┆ 0.02 │
│ true    ┆ 80.0  ┆ 80.0   ┆ 1.0  ┆ 0.1 ┆ 0.0  ┆ 0.03 │
│ true    ┆ 80.0  ┆ 80.0   ┆ 1.0  ┆ 0.1 ┆ 0.0  ┆ 0.04 │
│ …       ┆ …     ┆ …      ┆ …    ┆ …   ┆ …    ┆ …    │
│ false   ┆ 120.0 ┆ 120.0  ┆ 10.0 ┆ 0.5 ┆ 0.04 ┆ 0.03 │
│ false   ┆ 120.0 ┆ 120.0  ┆ 10.0 ┆ 0.5 ┆ 0.04 ┆ 0.04 │
│ false   ┆ 120.0 ┆ 120.0  ┆ 10.0 ┆ 0.5 ┆ 0.04 ┆ 0.05 │
│ false   ┆ 120.0 ┆ 120.0  ┆ 10.0 ┆ 0.5 ┆ 0.04 ┆ 0.06 │
│ false   ┆ 120.0 ┆ 120.0  ┆ 10.0 ┆ 0.5 ┆ 0.04 ┆ 0.07 │
└─────────┴───────┴────────┴──────┴─────┴──────┴──────┘
10,000,000
LAP cross: 0.099 s
--------------------------------------------------
SHAPE_INPUT: cols
--------------------------------------------------

The program should continue (cf demo notebook run-bs.ipynb), but exits without showing any indication what the error was...