PyO3 / pyo3

Rust bindings for the Python interpreter
https://pyo3.rs
Other
11.5k stars 702 forks source link

Creating Module With `pymodule` Cause Segmentation Fault In Free-Threaded Python 3.13 On Mac #4310

Open FlickerSoul opened 6 days ago

FlickerSoul commented 6 days ago

Bug Description

The following code,

use pyo3::prelude::*;

#[pymodule]
fn test_lib(m: &Bound<'_, PyModule>) -> PyResult<()> {
    Ok(())
}

when built and run in the free-threaded version of Python 3.13, will cause segmentation fault.

❯ python3.13 -c 'import sysconfig; print(sysconfig.get_config_var("Py_GIL_DISABLED"))'
1
❯ python3.13 -c 'import test_lib'
[1]    20097 segmentation fault  python3.13 -c 'import test_lib'

I'm not entirely sure if it's because of PyO3's or CPython.

Steps to Reproduce

  1. Clone https://github.com/FlickerSoul/pyo3-module-bug
  2. Use free threaded version to create venv and activate it
  3. Run maturin develop --target aarch64-apple-darwin
  4. Run python3.13 -c "import test_lib"
  5. See the segmentation fault

Backtrace

❯ lldb -- python -c "import test_lib"
(lldb) target create "python"
Current executable set to 'venv/bin/python' (arm64).
(lldb) settings set -- target.run-args  "-c" "import test_lib"
(lldb) run
Process 21042 launched: 'venv/bin/python' (arm64)
Process 21042 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x0)
    frame #0: 0x0000000184f888d0 libsystem_platform.dylib`_platform_strcmp$VARIANT$Base + 64
libsystem_platform.dylib`_platform_strcmp$VARIANT$Base:
->  0x184f888d0 <+64>: ldr    q0, [x0], #0x10
    0x184f888d4 <+68>: ldr    q1, [x1], #0x10
    0x184f888d8 <+72>: cmeq.16b v1, v0, v1
    0x184f888dc <+76>: and.16b v0, v0, v1
Target 0: (python) stopped.
(lldb) bt
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x0)
  * frame #0: 0x0000000184f888d0 libsystem_platform.dylib`_platform_strcmp$VARIANT$Base + 64
    frame #1: 0x00000001008970b8 libpython3.13t.dylib`_PyImport_ResolveNameWithPackageContext + 76
    frame #2: 0x0000000100775a40 libpython3.13t.dylib`_PyModule_CreateInitialized + 284
    frame #3: 0x00000001001e6d3c test_lib.cpython-313t-darwin.so`pyo3_ffi::modsupport::PyModule_Create::ha5da682c5748e8de(module=0x0000000100240298) at modsupport.rs:140:5
    frame #4: 0x00000001001e8ec4 test_lib.cpython-313t-darwin.so`pyo3::impl_::pymodule::ModuleDef::make_module::_$u7b$$u7b$closure$u7d$$u7d$::h19ffdd4137634f86 at pymodule.rs:148:25
    frame #5: 0x0000000100220560 test_lib.cpython-313t-darwin.so`pyo3::sync::GILOnceCell$LT$T$GT$::init::h4f5a3638ae166d78(self=0x0000000100240308, (null)=(__0 = core::marker::PhantomData<(&pyo3::gil::GILGuard, pyo3::impl_::not_send::NotSend)> @ 0x000000016fdfcdd7), f={closure_env#0} @ 0x000000016fdfcdd8) at sync.rs:157:21
    frame #6: 0x00000001001e5ba0 test_lib.cpython-313t-darwin.so`pyo3::sync::GILOnceCell$LT$T$GT$::get_or_try_init::h0ef029fbba9c4847(self=0x0000000100240308, (null)=(__0 = core::marker::PhantomData<(&pyo3::gil::GILGuard, pyo3::impl_::not_send::NotSend)> @ 0x000000016fdfce87), f={closure_env#0} @ 0x000000016fdfce88) at sync.rs:146:9
    frame #7: 0x00000001001e8e20 test_lib.cpython-313t-darwin.so`pyo3::impl_::pymodule::ModuleDef::make_module::h082db6c3d1962475(self=0x0000000100240290, (null)=(__0 = core::marker::PhantomData<(&pyo3::gil::GILGuard, pyo3::impl_::not_send::NotSend)> @ 0x000000016fdfceff)) at pymodule.rs:143:9
    frame #8: 0x00000001001dd838 test_lib.cpython-313t-darwin.so`test_lib::test_lib::__pyo3_init::_$u7b$$u7b$closure$u7d$$u7d$::h9de0cf31803401d2((null)=0x000000016fdfd03e, (null)=(__0 = core::marker::PhantomData<(&pyo3::gil::GILGuard, pyo3::impl_::not_send::NotSend)> @ 0x000000016fdfd01f)) at lib.rs:3:1
    frame #9: 0x00000001001dd6c8 test_lib.cpython-313t-darwin.so`core::ops::function::FnOnce::call_once::head2ac0736f5dc62((null)={closure_env#0} @ 0x000000016fdfd03e, (null)=(pyo3::marker::Python) @ 0x000000016fdfd03f) at function.rs:250:5
    frame #10: 0x00000001001ddb34 test_lib.cpython-313t-darwin.so`pyo3::impl_::trampoline::module_init::_$u7b$$u7b$closure$u7d$$u7d$::h8a2f9709624beab0((null)=(__0 = core::marker::PhantomData<(&pyo3::gil::GILGuard, pyo3::impl_::not_send::NotSend)> @ 0x000000016fdfd0af)) at trampoline.rs:22:21
    frame #11: 0x00000001001ddb00 test_lib.cpython-313t-darwin.so`pyo3::impl_::trampoline::trampoline::_$u7b$$u7b$closure$u7d$$u7d$::h654b5c2b83d193c8 at trampoline.rs:188:54
    frame #12: 0x00000001001dd5b0 test_lib.cpython-313t-darwin.so`std::panicking::try::do_call::h7f28577fa69c1c21(data="\U00000010\xd3\xdfo\U00000001") at panicking.rs:559:40
    frame #13: 0x00000001001dd680 test_lib.cpython-313t-darwin.so`__rust_try + 32
    frame #14: 0x00000001001dd4f4 test_lib.cpython-313t-darwin.so`std::panicking::try::hfebe6c5ab5c7e583(f={closure_env#0}<pyo3::impl_::trampoline::module_init::{closure_env#0}, *mut pyo3_ffi::object::PyObject> @ 0x000000016fdfd208) at panicking.rs:523:19
    frame #15: 0x00000001001dd8ac test_lib.cpython-313t-darwin.so`std::panic::catch_unwind::ha70320dc9581b60c(f={closure_env#0}<pyo3::impl_::trampoline::module_init::{closure_env#0}, *mut pyo3_ffi::object::PyObject> @ 0x000000016fdfd248) at panic.rs:149:14
    frame #16: 0x00000001001dda7c test_lib.cpython-313t-darwin.so`pyo3::impl_::trampoline::trampoline::h91958361332b5c32(body={closure_env#0} @ 0x000000016fdfd2e0) at trampoline.rs:188:9
    frame #17: 0x00000001001dd710 test_lib.cpython-313t-darwin.so`pyo3::impl_::trampoline::module_init::h28307e976c4ec7a8(f=(test_lib.cpython-313t-darwin.so`core::ops::function::FnOnce::call_once::head2ac0736f5dc62 at function.rs:250)) at trampoline.rs:22:5
    frame #18: 0x00000001001dd730 test_lib.cpython-313t-darwin.so`PyInit_test_lib at lib.rs:3:1
    frame #19: 0x000000010089e2d8 libpython3.13t.dylib`_PyImport_RunModInitFunc + 60
    frame #20: 0x000000010089b808 libpython3.13t.dylib`import_run_extension + 116
    frame #21: 0x000000010089d2cc libpython3.13t.dylib`_imp_create_dynamic + 492
    frame #22: 0x0000000100774530 libpython3.13t.dylib`cfunction_vectorcall_FASTCALL + 268
    frame #23: 0x000000010084facc libpython3.13t.dylib`_PyEval_EvalFrameDefault + 13648
    frame #24: 0x0000000100712520 libpython3.13t.dylib`object_vacall + 304
    frame #25: 0x0000000100712320 libpython3.13t.dylib`PyObject_CallMethodObjArgs + 104
    frame #26: 0x0000000100899cf0 libpython3.13t.dylib`PyImport_ImportModuleLevelObject + 1064
    frame #27: 0x00000001008542a0 libpython3.13t.dylib`_PyEval_EvalFrameDefault + 32036
    frame #28: 0x000000010084c274 libpython3.13t.dylib`PyEval_EvalCode + 360
    frame #29: 0x000000010084834c libpython3.13t.dylib`builtin_exec + 1228
    frame #30: 0x0000000100774614 libpython3.13t.dylib`cfunction_vectorcall_FASTCALL_KEYWORDS + 164
    frame #31: 0x000000010084facc libpython3.13t.dylib`_PyEval_EvalFrameDefault + 13648
    frame #32: 0x0000000100712520 libpython3.13t.dylib`object_vacall + 304
    frame #33: 0x0000000100712320 libpython3.13t.dylib`PyObject_CallMethodObjArgs + 104
    frame #34: 0x0000000100899cf0 libpython3.13t.dylib`PyImport_ImportModuleLevelObject + 1064
    frame #35: 0x00000001008542a0 libpython3.13t.dylib`_PyEval_EvalFrameDefault + 32036
    frame #36: 0x000000010084c274 libpython3.13t.dylib`PyEval_EvalCode + 360
    frame #37: 0x00000001008c3ef4 libpython3.13t.dylib`run_eval_code_obj + 180
    frame #38: 0x00000001008c3994 libpython3.13t.dylib`run_mod + 224
    frame #39: 0x00000001008c1818 libpython3.13t.dylib`_PyRun_SimpleStringFlagsWithName + 176
    frame #40: 0x00000001008e6588 libpython3.13t.dylib`Py_RunMain + 1760
    frame #41: 0x00000001008e6d10 libpython3.13t.dylib`pymain_main + 324
    frame #42: 0x00000001008e6db0 libpython3.13t.dylib`Py_BytesMain + 40
    frame #43: 0x0000000184bcf274 dyld`start + 2840
(lldb)

Your operating system and version

macOS 15.0 24A5279h arm64

Your Python version (python --version)

Python 3.13.0b3

Your Rust version (rustc --version)

rustc 1.79.0 (129f3b996 2024-06-10)

Your PyO3 version

0.22.0

How did you install python? Did you use a virtualenv?

To install: pyenv install 3.13.0b3t To create virtualenv: python -m venv ./venv Then activate with source ./venv/bin/activate and then run maturin develop

Additional Info

The non-free-threaded version of 3.13.0b3, installed with pyenv install 3.13.0b3 works fine.

davidhewitt commented 6 days ago

This is not a huge surprise, we need to make PyO3 compatible with the free-threaded builds yet. It's probably my highest priority thing to look at now. The failure mode could be better (e.g. we could at least give you an error on build).

alex commented 6 days ago

https://github.com/PyO3/pyo3/issues/4265 is a tracking issue for free-threading