search

pypa / cibuildwheel

🎡 Build Python wheels for all the platforms with minimal configuration.
https://cibuildwheel.pypa.io
Other
1.86k stars 237 forks source link

Add support for PyO3 #134

Closed thedrow closed 4 years ago

thedrow commented 5 years ago

PyO3 is a Rust library that allows you to write Cpython extensions in Rust. It'd be awesome to have an integration between it and https://github.com/PyO3/pyo3-pack

YannickJadoul commented 5 years ago

@thedrow Is there anything programming language-specific that you've encountered in cibuildwheel? All cibuildwheel is expecting is a pip wheel compatible setup.py file, I believe, so once you have that, integration with PyO3 should already be possible.

thedrow commented 5 years ago

pyo3-pack does not use setup.py. You'll need to run their CLI instead.

YannickJadoul commented 5 years ago

Then how do you make a PyPI compatible source package?

joerick commented 5 years ago

Hm. Interesting! Pip already has the ability to use different build systems (PEP 517), using pyproject.toml. I see there's an open issue to add support for this method. That would allow us to support it without any pyo3-specific additions, which would be ideal, since it would support other build systems too.

Otherwise, maybe there's a way we can add an option to change the build command. Something like CIBW_BUILD_COMMAND where the default is pip wheel {project_dir} -w {output_dir} --no-deps. My only reservation with this is that as PEP517 becomes more widely supported, pip wheel will work for every build system, reducing the need for the option.

thedrow commented 5 years ago

Let's keep this open until PyO3/pyo3-pack#2 is resolved then or close this issue if it's not something you're willing to provide a workaround for right now.

There's actually something else we need: a rust compiler. PyO3-pack does produce manylinux wheels as of PyO3/pyo3-pack#69 but the docker container does not contain a rust compiler. Since we need to install it, we need to provide an option to specify which version or rust we want to use, install rustup and install that version of rust with it.

thedrow commented 5 years ago

Then how do you make a PyPI compatible source package?

Right now source distributions are not supported.

mtreinish commented 5 years ago

I've been using both pyo3 and cibuildwheel together to upload wheels for a little toy project for several months now. It works well you can see the travis setup here: https://github.com/mtreinish/retworkx/blob/master/.travis.yml#L35 but I used setuptools-rust because personally I found it an easier model to use than pyo3-pack. The only tricky bit is that you have to install rust nightly as a before build command. Not hard to do though.

joerick commented 4 years ago

Looks like this is fixed via PEP517 support in pyo3-pack, and setuptools-rust is an alternative.

tasn commented 4 years ago

For whomever else that's interested, here's the .travis.yml I ended up writing. It's quite concise and builds the wheel on almost all platforms. I had to disable win32 and ppc64le due to bugs in some of my deps. The file: https://github.com/etesync/etebase-py/blob/master/.travis.yml