Pure Python fallbacks

[davidhewitt]

Recently I heard a comment pointing out that many C extension modules offer a pure Python fallbacks for users on performs without wheels and do not wish to compile code.

For Rust there's the additional case that not all hardware can run Rust code. Even if that gap is closing over time, it would be nice to make it possible to package code for these users.

I think it might not be too hard - there could be a pure-python path in the pep517 backend which is activated by a MATURIN_NO_RUST environment variable. This pathway would just install the Python code and skip any compile. Maybe we could even delegate to another backend like hatch.

Then we just need an example, to prompt users to consider it.

Of course, maybe this kind of complex configuration is better left to setuptools-rust.

[messense]

See also https://github.com/PyO3/maturin/issues/1081

[konstin]

The main problem generating different kinds of wheels based on the environment is that it breaks built source distribution caching, which is one of the main assumption a python package manager has to make. Would uploading a py3-none-any wheel next to the built wheels also work for you? That wheel could, but wouldn't even need to be built by maturin; I assume you need different python sources anyway if you don't have a rust lib to call into.

[davidhewitt]

Interesting idea to upload a pure python wheel. I think if users have set the build backend to maturin in the pyproject.toml it will be awkward for them to use a different backend without us giving them a way to delegate it. We could add config for that.

I think the Python sources can be the same ones, they could do a fairly common trick of try to import from the Rust library, and if that fails fall back to Python.

cc @mgorny who I understand would want this in Gentoo. Does giving maturin the ability to assemble a pure Python wheel help with your distribution challenges, assuming extension authors then did add pure-python fallbacks?

[mgorny]

Yes, I think having an option of building pure Python wheels would be helpful. Rather than via envvar, enabling it via maturin options (i.e. via the config_settings dict in the PEP517 backend) would also work for us.

However, it all depends on whether the Python-Rust ecosystem would actually develop a culture of providing fallbacks.

[ariebovenberg]

Thought I'd share my experience here, as I’m also working on a project with a pure-python fallback.

In general I'm really happy with setuptools-rust, but I really miss the ability to use the maturin github action:

1. It's easier to configure than cibuildwheel
2. includes more architectures (e.g. armv7l)
3. there are a lot more examples out there on github to learn from

I also wonder if it'd be a lot faster since it uses cross-compilation instead of relying on emulation 🤔

My project would easily build with maturin (minimal changes)—I only need the optionality part from setuptools-rust. If I'd want to support more architectures, I'm considering creating a second pyproject.toml with maturin as build-system just for the convenience of creating platform wheels.

edit: update: I now switched to maturin for building wheels and it is way faster (2 minutes instead of >20 minutes with cibuildwheel!)

[messense]

Thanks @ariebovenberg, I think your approach of combining setuptools-rust and maturin/maturin-action is a viable way to implement pure python fallbacks.