pip install twine fails with pypy

jbmoorhouse commented 1 year ago

1) Your operating system:

Distributor ID: Ubuntu
Description:    Ubuntu 20.04.3 LTS
Release:    20.04

2) Version of python you are running:

pypy 3.10

3) How did you install twine? Did you use your operating system's package manager or pip or something else?

Using pip

The Issue

pip install twine errors with pypy.

Steps to Reproduce

When running the following.

docker run -it pypy:3.10-7.3.12-bullseye bash
...
root@...:/# pip install twine

I get the following.

Collecting nh3>=0.2.14
  Downloading nh3-0.2.14.tar.gz (14 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Preparing metadata (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [6 lines of output]

      Cargo, the Rust package manager, is not installed or is not on PATH.
      This package requires Rust and Cargo to compile extensions. Install it through
      the system's package manager or via https://rustup.rs/

      Checking for Rust toolchain....
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

Thoughts

The issue seems to be with the package readme-renderer and the dependency change from bleach to nh3 here, which requires cargo.

Question

Since readme-renderer is used for docs, should this be a dev dependency?

sigmavirus24 commented 1 year ago

readme-renderer is not a development dependency. nh3 should be producing manylinux wheels

James-E-A commented 9 months ago

nh3 should be producing manylinux wheels

That won't help.

nh3-0.2.15-cp37-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl

The "cp" in "cp37-abi3" stands for "CPython", and "manylinux" just means you (theoretically) don't have to deal with libc compatibility hell since it's built on the oldest supported version of CentOS.

And anyway, that still leaves Windows unaccounted-for; this fails with PyPy3.10 7.3.15 64-bit on Windows, saying the file "is not a supported wheel on this platform":

python -m pip install -v https://files.pythonhosted.org/packages/6b/d2/93c8c82464d9562eae68b8e442c00d5a0d1c6a2d74478642d8bd22b45aee/nh3-0.2.15-cp37-abi3-win_amd64.whl

Quick fix for PyPy users on Windows showing up here from Google, who don't want to install rust or rustc or cargo or the rust updater or the rust updater installer or otherwise mess with their system in any other way than just installing Twine:

REM https://github.com/pypa/twine/issues/1015
REM https://github.com/pypa/readme_renderer/blob/42.0/pyproject.toml#L10
REM https://github.com/messense/nh3/blob/v0.2.15/.github/workflows/CI.yml#L58
REM https://github.com/rust-lang/rust-forge/issues/225
REM https://github.com/rust-lang/rust/issues/116028
REM https://github.com/rust-lang/cargo/issues/1734

set /p="Download MSI from https://forge.rust-lang.org/infra/other-installation-methods.html#standalone-installers then press Enter"
msiexec /a "%USERPROFILE%\Downloads\rust-1.75.0-x86_64-pc-windows-msvc.msi" /qb TARGETDIR="%cd%"
cmd /C set ^"PATH=%cd%\Rust\bin;%PATH%^" ^& set ^"CARGO_HOME=%cd%\Rust\.cargo^" ^& python -m pip install -v twine

This fix doesn't need administrative privileges
As far as I can tell, this fix doesn't seem to pollute anything other than the working directory—not the Registry, and not even the local shell's environment

Alternative fix, which is even simpler and has all the same advantages as the above fix but will make you miss out on any changes in ≥42.0 of readme-renderer (so far it looks like it was only administrative build-time changes and performance enhancements, so there should be no downsides yet):

pip install -v "readme-renderer < 42.0" && REM https://github.com/pypa/twine/issues/1015
pip install twine

sigmavirus24 commented 8 months ago

As for installing on PyPy and the need for manylinux wheels, there are manylinux wheels for pypy. See Cryptography which publishes wheels for pypy: cryptography-42.0.3-pp310-pypy310_pp73-manylinux_2_28_x86_64.whl as well as wheels for Windows on PyPy: cryptography-42.0.3-pp310-pypy310_pp73-win_amd64.whl. Some of this infrastructure isn't easy to maintain, but it is possible to build and distribute these. PyPy manylinux wheels are a matter of setting up the build in CI (and GitHub Actions, which I'm guessing nh3 uses, makes this easy and supports Linux, macOS, and Windows).

James-E-A commented 8 months ago

Just for clarification, is this closed as "completed (we found an alternative to nh3)"?

Or is it closed as "completed (nh3 now publishes PyPy wheels)"?

Or is it closed as "wontfix (not our problem / go blame nh3 for not publishing PyPy wheels)"?

sigmavirus24 commented 8 months ago

I'm not instructing you to be hateful or passive aggressive towards anyone. Twine relies on readme-renderer which is the best (and last I checked, only) library to do what we need. It's not something twine should implement itself nor will it. The ability to check the rendering of an artifact's description before uploading to PyPI is also table stakes. Further it has been a feature of twine since before readme-renderer started using nh3, removing it or making it hidden behind an extra is not an option.

nh3 is not a direct dependency for twine so it's not something we can just drop. Thus we aren't fixing this. I closed this because there's no action available for us to resolve this for you. There are many other ways to potentially resolve this though but likely none will involve the maintainers of twine

James-E-A commented 6 months ago

I'm not instructing you to be hateful or passive aggressive towards anyone.we aren't fixing this.

I closed this because there's no action available for us to resolve this for you.

Sorry for my miscommunication; I was only asking for clarification due to how the issue had been filed.

Specifically, I wasn't sure if the positive close reason meant that there had been some kind of last-minute change in Twine to accept and resolve the issue.

flying-sheep commented 1 month ago

For the record, I’d have basically asked for clarification in the same words as @James-E-A, also with completely innocent intentions.

@sigmavirus24 I think it would help to assume the best in people in situations like this instead of assuming aggression where none was intended.

pypa / twine