Empty body of METADATA results into an error when `twine check`

hzhangxyz commented 2 years ago

Your Environment

twine version is 4.0.1 (install from pip) python version is 3.10.5 (install by system package manager) linux (arch linux)

The Issue

I create a wheel file with cibuildwheel(https://github.com/pypa/cibuildwheel), in which METADATA has an empty body like

Metadata-Version: 2.1
Name: PyTAT
Version: 0.2.14
Summary: python binding for TAT(TAT is A Tensor library)
Home-page: https://github.com/hzhangxyz/TAT
Author: Hao Zhang
Author-email: zh970205@mail.ustc.edu.cn
License: GPLv3
License-File: LICENSE.md
Requires-Dist: numpy

old version cibuildwheel does not leave body empty, and add a word "UNKNOWN" like:

Metadata-Version: 2.1
Name: PyTAT
Version: 0.2.13
Summary: python binding for TAT(TAT is A Tensor library)
Home-page: https://github.com/hzhangxyz/TAT
Author: Hao Zhang
Author-email: zh970205@mail.ustc.edu.cn
License: GPLv3
Platform: UNKNOWN
License-File: LICENSE.md
Requires-Dist: numpy

UNKNOWN

With wheel file created with new version cibuildwheel, twien check failed with message:

ERROR    `long_description` has syntax errors in markup and would not be rendered on PyPI.
         No content rendered from RST source.

Steps to Reproduce

Just delete the description body of package can reproduce.

I also post an issue at https://github.com/pypa/cibuildwheel/issues/1131 but it seems it is not their fault.

because https://peps.python.org/pep-0566/#description saying

In addition to the Description header field, the distribution’s description may instead be provided in the message body (i.e., after a completely blank line following the headers, with no indentation or other special formatting necessary).

It may be provided, so not to do so should be permitted.

bhrutledge commented 2 years ago

I did some experimenting with an example package. It looks like the UNKNOWN behavior changed in setuptools v62.2.0, with the merge of https://github.com/pypa/setuptools/pull/3299.

I was able to reproduce the error using a wheel:

% rm -Rf build && python3 -m build -w
...
Successfully built example_pkg_bhrutledge-0.0.5-py3-none-any.whl

% unzip -c dist/example_pkg_bhrutledge-0.0.5-py3-none-any.whl example_pkg_bhrutledge-0.0.5.dist-info/METADATA
Metadata-Version: 2.1
Name: example-pkg-bhrutledge
Version: 0.0.5
Summary: A small example package
Home-page: https://github.com/pypa/sampleproject
Author: Example Author
Author-email: author@example.com
Project-URL: Bug Tracker, https://github.com/pypa/sampleproject/issues
Classifier: Programming Language :: Python :: 3
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Requires-Python: >=3.6
Description-Content-Type: text/rst
License-File: LICENSE

% twine check dist/example_pkg_bhrutledge-0.0.5-py3-none-any.whl
Checking dist/example_pkg_bhrutledge-0.0.5-py3-none-any.whl: FAILED
ERROR    `long_description` has syntax errors in markup and would not be rendered on PyPI.
         no output rendered

However, I'm not seeing the same behavior from the sdist:

% rm -Rf dist && python3 -m build -s
...
Successfully built example-pkg-bhrutledge-0.0.5.tar.gz

% tar Oxvf dist/example-pkg-bhrutledge-0.0.5.tar.gz 'example-pkg-bhrutledge-0.0.5/PKG-INFO'
Metadata-Version: 2.1
Name: example-pkg-bhrutledge
Version: 0.0.5
Summary: A small example package
Home-page: https://github.com/pypa/sampleproject
Author: Example Author
Author-email: author@example.com
Project-URL: Bug Tracker, https://github.com/pypa/sampleproject/issues
Classifier: Programming Language :: Python :: 3
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Requires-Python: >=3.6
License-File: LICENSE

% twine check dist/example-pkg-bhrutledge-0.0.5.tar.gz
Checking dist/example-pkg-bhrutledge-0.0.5.tar.gz: PASSED with warnings
WARNING  `long_description_content_type` missing. defaulting to `text/x-rst`.
WARNING  `long_description` missing.

Here's the relevant configuration:

https://github.com/bhrutledge/example-pkg-bhrutledge/blob/missing-description/pyproject.toml https://github.com/bhrutledge/example-pkg-bhrutledge/blob/missing-description/setup.cfg

The warnings are what I would expect, given the missing long_description in setup.cfg. It looks like METADATA file has an extra blank line at the end:

--- example-pkg-bhrutledge-0.0.5/PKG-INFO       2022-06-23 08:18:59.000000000 -0400
+++ example_pkg_bhrutledge-0.0.5.dist-info/METADATA     2022-06-23 12:22:08.000000000 -0400
@@ -12,3 +12,4 @@
 Requires-Python: >=3.6
 Description-Content-Type: text/rst
 License-File: LICENSE
+

I wonder if that's what's tripping up Twine (or readme_renderer under the hood), though I'm also seeing the same error with backends other than setuptools (e.g. hatchling and flit) that don't seem to add an extra line.

Need to investigate further.

miketheman commented 2 years ago

@bhrutledge One difference I found from reading the scenario:

Description-Content-Type appears in the built wheel METADATA and not in the sdist

~I tried to repro your findings, but couldn't find a build module - where does that come from?~ never mind, it's using https://pypi.org/project/build/ - whoops.

jaraco commented 5 months ago

+1; I encountered this issue also today when attempting to check a package with minimal metadata. According to the spec, only Metadata-Version, Name, and Version are required. Maybe check should warn when the long description is empty, but it shouldn't error.

sigmavirus24 commented 5 months ago

+1; I encountered this issue also today when attempting to check a package with minimal metadata. According to the spec, only Metadata-Version, Name, and Version are required. Maybe check should warn when the long description is empty, but it shouldn't error.

The explicit purpose of check is to verify the long description renders without errors on PyPI. It does almost nothing else.

DarkaMaul commented 1 week ago

Sorry to bump this issue. I had the same issue today in one of my runs.

Breadcrumb

twine check performs a verification on the package metadata ( here )
Notably, these verification includes that the description content is properly formatted, If no description is provided, only a warning is emitted.
The description field is parsed from the Wheel class which uses internally pkginfo which relies on Python email parser to parse the package metadata. The description is considered to be the payload of the message.
The Python email parser first parses the headers lines, then considers anything after the headers lines to be the message body (if the message type is not specified).

Bug

If the description is empty, the payload will be set to self._cur.set_payload(EMPTYSTRING.join([])). This results in an empty string if the description is empty.

If we go back at twine:

https://github.com/pypa/twine/blob/ded15b9e56db6e451a59eaa5582207f29d86dc83/twine/commands/check.py#L99-L106

description is not None so we try to render using one of the renderer (by default text/x-rst). renderer which will returns None for an empty string, generating an error

from twine.commands.check import _RENDERERS
renderer = _RENDERERS.get("text/x-rst")
assert renderer.render("") is None

pypa / twine