Package pyO3

[h-vetinari]

Some packages are considering (or starting to) to want to compile Rust code into python packages - notably cryptography (see pyca/cryptography#5381). The plan is to do this by ~2021 using pyO3, which isn't packaged by conda-forge yet.

However, with existing feedstocks for rust and setuptools-rust, it seems like there's not too many pieces missing (hopefully).

CC @mingwandroid @isuruf regarding the consequences for cryptography

[h-vetinari]

@conda-forge/core @conda-forge/rust, cryptography is closing in on using rust through PyO3 in their build process for the next release (last blocker AFAICT is the new PyO3 release scheduled in 1-2 days).

I'm not much of a Rustacean, so not sure how far away we are from supporting this during the build process? Can we build the rust-parts without PyO3 (seeing that the compiler already exists), or do we need to package PyO3?

[synapticarbors]

I'm not sure that it's optimal and it looks like cryptography is using a different build procedure, but this is how we do orjson that uses PyO3 (but packaging via maturin):

https://github.com/conda-forge/orjson-feedstock/blob/master/recipe/meta.yaml

We should probably really be coordinating with the rust compiler dev branch for the nightly releases, but we just pull the nightlies we need directly from rust.

[mbargull]

I'm not much of a Rustacean, so not sure how far away we are from supporting this during the build process? Can we build the rust-parts without PyO3 (seeing that the compiler already exists), or do we need to package PyO3?

The current state with Rust packages is that conda-forge doesn't do any special-handling here, meaning you'd just have your pyo3 dep in Cargo.toml and cargo will fetch everything itself during the build. Meaning, Rust libraries like PyO3 don't have Conda packages but can be used in the builds.

We should probably really be coordinating with the rust compiler dev branch for the nightly releases, but we just pull the nightlies we need directly from rust.

https://github.com/conda-forge/rust-feedstock/tree/dev can be updated as needed. But PyO3 >=0.11.0 works with stable Rust anyway, so unless the downstream project needs nightlies, we can use the stable releases from conda-forge::rust.

[h-vetinari]

Thanks for the info @mbargull! I naively thought that PyO3 is more than a crate (seeing how it has pyproject.toml etc. in the repo), but happy to hear if it can be used in recipes already.

PS. the newest PyO3 0.13 will require rust 1.45, but that's no problem since CF is on 1.48 already.

[h-vetinari]

In-progress for cryptography: https://github.com/conda-forge/cryptography-feedstock/pull/64