search

PyO3 / pyo3

Rust bindings for the Python interpreter
https://pyo3.rs
Apache License 2.0
12.5k stars 770 forks source link

⚠️ PyO3 0.23.0, 0.23.1, 0.23.2 have soundness bug in compile configuration #4757

Open davidhewitt opened 17 hours ago

davidhewitt commented 17 hours ago

I've seen recent segmentation faults from builds shipped in watchfiles and jiter, as well as unexpected errors in pydantic-core's test suite.

My current hypothesis is that the root cause is #4497, which mistakenly changed reading of PYO3_CONFIG_FILE env var such that we don't emit cargo:rerun-if-changed block for it.

As such, wheels built in bulk for many interpreters with maturin are not rebuilt properly and all contain code cfg-d for the first interpreter built for.

I will continue to verify that this is the cause, and work on a patch ASAP.

I think we will need to yank 0.23.0 through 0.23.2 and file a security advisory, as wheels built with these versions are likely highly unstable and will fail in unpredictable ways.

davidhewitt commented 17 hours ago

fat finger submit, details incoming...

alexdotgov2 commented 15 hours ago

This "only" impacts the situation where you're doing multiple builds in the same dir (or caching it), is that right?

ngoldbaum commented 15 hours ago

I don't think this is the root cause of issues in 0.23, but anyone who tries to set up free-threaded wheels may run into https://github.com/Quansight-Labs/setup-python/issues/5 on Windows, which also affects CI environments setting up multiple python versions simultaneously. Just sharing here since others might be experimenting with building free-threaded wheels using Maturin.

davidhewitt commented 15 hours ago

This "only" impacts the situation where you're doing multiple builds in the same dir (or caching it), is that right?

Yes, typically this would be due to using maturin build -i 3.8 3.9 3.10 3.11 etc. Projects like cryptography are likely unaffected because they're only building the one abi3 wheel.