PyPI doesn't accept a file and suggests to use the same prefix the file has

[krasinski]

Describe the bug PyPI doesn't accept a file and suggests to use the same prefix the file has.

Expected behavior The file is uploaded.

To Reproduce twine upload h2o_pysparkling_3_2-3.42.0.4-1.tar.gz -u ********* -p *********

HTTPError: 400 Bad Request from https://upload.pypi.org/legacy/
Start filename for 'h2o-pysparkling-3-2' with 'h2o_pysparkling_3_2'.

My Platform Ubuntu 16.04 docker image in CI environment, twine was installed with conda also Ubuntu 22.04 with twine version 4.0.2 (importlib-metadata: 6.6.0, keyring: 23.13.1, pkginfo: 1.9.6, requests: 2.28.1, requests-toolbelt: 1.0.0, urllib3: 1.26.15)

Additional context Project: https://pypi.org/project/h2o-pysparkling-3.2/ Previously it didn't work with h2o_pysparkling_3.2-3.42.0.4-1.tar.gz filename, I expected the error to go away after changing, but the error stayed the same. I tried looking into the code to understand the issue myself, but I couldn't find a source for "normalize_pep426_name" function and sometimes, after many retries uploading the file works! with h2o_pysparkling_3.2-3.42.0.4-1.tar.gz style name

[di]

I'm a little confused by this:

and sometimes, after many retries uploading the file works! with h2o_pysparkling_3.2-3.42.0.4-1.tar.gz style name

I think the correct filename should be h2o_pysparkling_3_2-3.42.0.4-1.tar.gz and that h2o_pysparkling_3.2-3.42.0.4-1.tar.gz should fail, but seems like that's the opposite of what you're describing here?

I also don't see a successful upload with any of these versions, only ones with .post1 in the version.

[krasinski]

@di thanks for the response!

it looks a bit like some servers accept, and some servers don't accept, I don't know the architecture of PyPI, but is that possible you have different versions deployed and some load balancer sometimes directs to server which accepts those names, and sometimes to another, which doesn't?

either way what I described in the first post looks like a bug to me, as the error message suggests exactly the prefix the file name has

[di]

it looks a bit like some servers accept, and some servers don't accept, I don't know the architecture of PyPI, but is that possible you have different versions deployed and some load balancer sometimes directs to server which accepts those names, and sometimes to another, which doesn't?

No, I don't think this is possible.

either way what I described in the first post looks like a bug to me, as the error message suggests exactly the prefix the file name has

I agree, but without the original filename this will be hard to diagnose. Are you still able to reproduce this?

[krasinski]

ok, it turned out that my colleague had different setup tools version than me and CI but the original issue is still valid, I can reproduce it easily I think

[di]

Thanks, if you're able to reproduce and can provide us with the file that reproduces, that would be very helpful!

[edreamleo]

I have the same bug on Windows 11. It's delaying the next release of leo.

Here is a slightly edited copy of twine issue #1069 that was referred here.

Environment

Windows 11
python 3.12.0
python -m twine --version
twine version 5.0.0 (importlib-metadata: 7.0.2, keyring: 24.3.1, pkginfo: 1.10.0, requests: 2.31.0, requests-toolbelt:
1.0.0, urllib3: 2.2.1)

Which package repository are you using?

https://test.pypi.org/project/leo/

Please describe the issue that you are experiencing

The dist directory contains two files, both of which start with “leo”:

leo-6.7.8-9-py3-none-any.whl
leo-6.7.8-9.tar.gz

The suffix -9 is to ensure a unique file name.

The wheel uploads correctly, but twine complains about the .tar.gz file:

python -m twine upload -r testpypi dist/* --verbose
INFO     Using configuration from C:\Users\Dev\.pypirc
Uploading distributions to https://test.pypi.org/legacy/
INFO     dist\leo-6.7.8-9-py3-none-any.whl (14.4 MB)
INFO     dist\leo-6.7.8-9.tar.gz (12.5 MB)
INFO     password set from config file
INFO     username: __token__
INFO     password: <hidden>
Uploading leo-6.7.8-9-py3-none-any.whl
100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 15.1/15.1 MB • 00:05 • 2.8 MB/s
INFO     Response from https://test.pypi.org/legacy/:
         200 OK
INFO     <html>
          <head>
           <title>200 OK</title>
          </head>
          <body>
           <h1>200 OK</h1>
           <br/><br/>

          </body>
         </html>
Uploading leo-6.7.8-9.tar.gz
100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 13.1/13.1 MB • 00:04 • 2.9 MB/s
INFO     Response from https://test.pypi.org/legacy/:
         400 Start filename for 'leo' with 'leo'.
INFO     <html>
          <head>
           <title>400 Start filename for 'leo' with 'leo'.</title>
          </head>
          <body>
           <h1>400 Start filename for 'leo' with 'leo'.</h1>
           The server could not comply with the request since it is either malformed or otherwise incorrect.<br/><br/>
         Start filename for &#x27;leo&#x27; with &#x27;leo&#x27;.

          </body>
         </html>
ERROR    HTTPError: 400 Bad Request from https://test.pypi.org/legacy/
         Start filename for 'leo' with 'leo'.

Please include the PKG-INFO file contents from the artifact you're attempting to upload

Metadata-Version: 1.0
Name: leo
Version: 6.7.8
Summary: An IDE, PIM and Outliner. See http://leo-editor.github.io/leo-editor/preface.html.
Description: See users own descriptions of Leo at leo-editor.github.io/leo-editor/testimonials.html.
Home-page: http://leo-editor.github.io/leo-editor/
Author: Edward K. Ream
Author-email: edreamleo@gmail.com
License: MIT
Keywords: Outliner, IDE, PIM, DAG, Scripting
Platform: Windows, Linux, MacOS
Classifier: Development Status :: 6 - Mature
Classifier: Operating System :: MacOS
Classifier: Operating System :: Microsoft :: Windows
Classifier: Operating System :: POSIX :: Linux
Classifier: Programming Language :: Python
Classifier: Topic :: Software Development
Classifier: Topic :: Text Processing
Download-URL: https://github.com/leo-editor/leo-editor/releases

A redacted version of your .pypirc file

[distutils]
  index-servers =
    pypi
    testpypi
    leo

[pypi]
  username = __token__
  password = pypi-AgEIc...

[testpypi]
  username = __token__
  password = pypi-AgENdG...

Does anyone have any advice?

[di]

@edreamleo That's a separate issue, due to you using a hyphen as a post-release separator, and would technically be a duplicate of https://github.com/pypi/warehouse/issues/12245. You might want to use a post-release instead.

[edreamleo]

@di Thanks for your quick response. I'm still pretty confused.

Has anything re filenames changed recently? Do release notes for PyPi itself exist?

I also wonder why the wheel file uploads correctly with a -nnn suffix but the .tar.gz file does not.

[di]

Has anything re filenames changed recently? Do release notes for PyPi itself exist?

Not within the last few months, but we have gotten more strict within the last few years.

I also wonder why the wheel file uploads correctly with a -nnn suffix but the .tar.gz file does not.

The reason is because wheel filenames are standardized in PEP 427, which we support, but source distribution filenames are standardized in PEP 625, which we don't support yet, and which #12245 is about.

[edreamleo]

@di Thanks for your response.

For the record, I fully support the decision to become more strict. All repos such as PyPi and GitHub are under constant attack.

[di]

@edreamleo Since this error message is pretty confusing, I've proposed a temporary fix for this in #15644 which will allow PyPI to permit your original filename.

Eventually though, your builder should support PEP 625, and produce a filename with a normalized version instead:

>>> from packaging.version import Version
>>> Version("6.7.8-9")
<Version('6.7.8.post9')>

Since it looks like you're using setuptools, https://github.com/pypa/setuptools/issues/3593 is the relevant issue.

[edreamleo]

@di Thanks for the work you do! It can't be easy trying to maintain compatibility while moving forward ;-)