PyO3 / maturin

Build and publish crates with pyo3, cffi and uniffi bindings as well as rust binaries as python packages
https://maturin.rs
Apache License 2.0
3.86k stars 267 forks source link

Change wheel name #313

Closed Hywan closed 2 years ago

Hywan commented 4 years ago

Hello,

I use Pyo3, thanks for the great software! I want to know whether it is possible to change the wheel name. I understand the wheel name is based on the crate name defined in the Cargo.toml file. However, I want to generate multiple wheels (depending of some Cargo features), like myproject, myproject-xxx, myproject-yyy etc. Is it possible to change that from the maturin commands, or do I have to open a PR?

konstin commented 4 years ago

The wheel name is indeed based on lib.name. This is because the wheel filename should match the package/module name, which needs to match the name of your pymodule or python package directory. Pip also expects that, e.g. when using pip install myproject --find-links /vendored/wheels/dir it looks for /vendored/wheels/dir/myproject-<tags>.whl 1 .

A simple workaround would be creating a new crate in the workspace that has a unique name and only pulls in in the actual implementation crate with the desired features and exposes a pymodule with the correct name. You can then publish this as separate python package without any name clashes.

1There are exceptions such as PIL/Pillow, but imo that's against the intention of PEP 427 and leads to really nasty bugs, e.g. installing both myproject-xxx and myproject-yyy would give different results depending on the installation order.

hombit commented 4 years ago

I also found that wheel name (packages name and name I will use to pip install my package) is a crate name, while lib.name is a module name (name I will use to import my package). It could be useful to override wheel name with something. For example, I have some multi-crate repository, where my workspace contains crates myproject and myproject-python. In this case my pypi package built from myproject-python crate will be named myproject-python while it would much better to call it simply myproject.

For example, if I rename crate in /test-crates/pyo3-pure/Cargo.toml to pyo3-pure-python then maturin build outputs:

🔗 Found pyo3 bindings
🐍 Found CPython 3.7m at python3.7
📦 Built source distribution to /maturin/test-crates/pyo3-pure/target/wheels/pyo3_pure_python-2.1.0.tar.gz
   Compiling pyo3 v0.8.2
   Compiling pyo3-pure-python v2.1.0 (/maturin/test-crates/pyo3-pure)
    Finished dev [unoptimized + debuginfo] target(s) in 6.68s
📦 Built wheel for CPython 3.7m to /maturin/test-crates/pyo3-pure/target/wheels/pyo3_pure_python-2.1.0-cp37-cp37m-macosx_10_7_x86_64.whl
hombit commented 4 years ago

Another idea to support @Hywan's initial proposal: one can rename Python module using simple macros based on cargo feature, i.e. we can have the same name for module and package for given feature if maturin will allow us to specify package name

bovee commented 4 years ago

I'm having the same issue where I have a rust library crate named e.g. "my_package" with Python bindings in a separate crate named "my_package_py", but I'd like the wheels to be named "my_package". In my case, I can't temporarily rename the bindings crate itself because it has a dependency on the library crate with the same name.

My super-hacky, temporary workaround was to clone maturin, change the output name here to e.g. "my_package".clone() and then cargo install --path . it locally. Then I could build my wheels, rename them (removing the _py) and then upload them with e.g. twine dist target/wheels/*.

tasn commented 4 years ago

I have exactly the same use-case as @hombit as @bovee. I understand the issue is that Python requires the binary name to be the same as the module name, and that's fine. One way to solve it would be to just let people wrap it in a python module, and inside it just have from .lib_name import *. Though this can't currently be done because maturin uses lib.name for the package name.

TL;DR: make it possible to tell maturin to use a directory (that's a valid python module) as the wheel's name.

fjarri commented 3 years ago

Blocked by this issue as well. I have a workspace with a Rust project cratename and Python bindings cratename-python, and would like the wheel (and hence the name in PyPi) to also be just cratename. I can't rename the bindings crate because it causes a conflict in the workspace.

Would adding an optional name to [package.metadata.maturin] to override the wheel name work as a solution?

Wenzel commented 3 years ago

Hi,

I successfully built my first python bindings with PyO3 yesterday, and i was considering maturin for package and publication, however, this issue is a blocker for me as well.

:arrow_right: should i switch to setuptools-rust in the meantime ? :arrow_right: do you intent to propose a fix to specify the package name ?

thanks !

konstin commented 3 years ago

What works thanks to houqp in #373 is the following, which should solve the myproject vs myproject-python case:

[package]
authors = ["konstin <konstin@mailbox.org>"]
name = "pyo3-name1"
version = "2.1.2"
edition = "2018"

[package.metadata.maturin]
name = "pyo3_name2"

[dependencies]
pyo3 = { version = "0.13.2", features = ["abi3-py36", "extension-module"] }

[lib]
name = "pyo3_name2"
crate-type = ["cdylib"]

This will create a pyo3_name2 wheel.

However, I want to generate multiple wheels (depending of some Cargo features), like myproject, myproject-xxx, myproject-yyy etc. Is it possible to change that from the maturin commands, or do I have to open a PR?

For this specific case, I suggest creating multiple wrapper crates in the same workspace that do nothing more than depending on the implementation with the right set of features and having the appropriate name.