Cannot upload files with pkginfo==1.8.2

Lothiraldan commented 2 years ago

Your Environment

Thank you for taking the time to report an issue.

To more efficiently resolve this issue, we'd like to know some basic information about your system and setup.

1) Your operating system: Linux / Docker

2) Version of python you are running:

Python 3.9.5

3) How did you install twine? Did you use your operating system's package manager or pip or something else?

I Used pip

4) Version of twine you have installed (include complete output of):

twine version 3.7.1 (importlib_metadata: 4.9.0, pkginfo: 1.8.1, requests: 2.25.1, requests-toolbelt: 0.9.1, tqdm: 4.61.2)

5) Which package repository are you targeting?

Test Pypi. I got the issue with an internal package but I could reproduce with https://github.com/buildbot/hello-world/blob/master/setup.py after applying the following diff:

diff --git a/setup.py b/setup.py
index b46412d..de2d3a6 100644
--- a/setup.py
+++ b/setup.py
@@ -1,7 +1,4 @@
-#!/usr/bin/python
-# (c) 2005-2009 Divmod, Inc.  See LICENSE file for details
-
-from distutils.core import setup
+from setuptools import setup

 setup(
     name="hello_world",

If you're having issues uploading a specific package, you must include a copy of the following:

The package's PKG-INFO file
A redacted version of your .pypirc file (REMOVE ALL USERNAMES & PASSWORDS BEFORE UPLOADING)

The Issue

When trying to upload a wheel file, I get the following traceback:

twine upload --repository-url https://test.pypi.org/legacy/ dist/hello_world-1.0.0-py3-none-any.whl 
Uploading distributions to https://test.pypi.org/legacy/
Enter your username: Boris.Feld
/opt/conda/lib/python3.9/site-packages/twine/auth.py:75: UserWarning: No recommended backend was available. Install a recommended 3rd party backend package; or, install the keyrings.alt package if you want to use the non-recommended backends. See https://pypi.org/project/keyring for details.
  warnings.warn(str(exc))
Enter your password: 
Traceback (most recent call last):
  File "/opt/conda/bin/twine", line 8, in <module>
    sys.exit(main())
  File "/opt/conda/lib/python3.9/site-packages/twine/__main__.py", line 28, in main
    result = cli.dispatch(sys.argv[1:])
  File "/opt/conda/lib/python3.9/site-packages/twine/cli.py", line 71, in dispatch
    return main(args.args)
  File "/opt/conda/lib/python3.9/site-packages/twine/commands/upload.py", line 194, in main
    return upload(upload_settings, parsed_args.dists)
  File "/opt/conda/lib/python3.9/site-packages/twine/commands/upload.py", line 141, in upload
    resp = repository.upload(package)
  File "/opt/conda/lib/python3.9/site-packages/twine/repository.py", line 195, in upload
    resp = self._upload(package)
  File "/opt/conda/lib/python3.9/site-packages/twine/repository.py", line 150, in _upload
    data = package.metadata_dictionary()
  File "/opt/conda/lib/python3.9/site-packages/twine/package.py", line 175, in metadata_dictionary
    "dynamic": meta.dynamic,
AttributeError: 'Wheel' object has no attribute 'dynamic'

When trying to upload a tar.gz file, I get the following traceback:

twine upload --repository-url https://test.pypi.org/legacy/ dist/hello_world-1.0.0.tar.gz 
Uploading distributions to https://test.pypi.org/legacy/
Enter your username: Boris.Feld
/opt/conda/lib/python3.9/site-packages/twine/auth.py:75: UserWarning: No recommended backend was available. Install a recommended 3rd party backend package; or, install the keyrings.alt package if you want to use the non-recommended backends. See https://pypi.org/project/keyring for details.
  warnings.warn(str(exc))
Enter your password: 
Traceback (most recent call last):
  File "/opt/conda/bin/twine", line 8, in <module>
    sys.exit(main())
  File "/opt/conda/lib/python3.9/site-packages/twine/__main__.py", line 28, in main
    result = cli.dispatch(sys.argv[1:])
  File "/opt/conda/lib/python3.9/site-packages/twine/cli.py", line 71, in dispatch
    return main(args.args)
  File "/opt/conda/lib/python3.9/site-packages/twine/commands/upload.py", line 194, in main
    return upload(upload_settings, parsed_args.dists)
  File "/opt/conda/lib/python3.9/site-packages/twine/commands/upload.py", line 141, in upload
    resp = repository.upload(package)
  File "/opt/conda/lib/python3.9/site-packages/twine/repository.py", line 195, in upload
    resp = self._upload(package)
  File "/opt/conda/lib/python3.9/site-packages/twine/repository.py", line 150, in _upload
    data = package.metadata_dictionary()
  File "/opt/conda/lib/python3.9/site-packages/twine/package.py", line 175, in metadata_dictionary
    "dynamic": meta.dynamic,
AttributeError: 'SDist' object has no attribute 'dynamic'

After downgrading plginfo to 1.8.1, the upload seems to works with the exact same files (I don't have permission to this project on TestPypi so I'm getting the expected 403):

twine upload --repository-url https://test.pypi.org/legacy/ dist/hello_world-1.0.0-py3-none-any.whl 
Uploading distributions to https://test.pypi.org/legacy/
Enter your username: Boris.Feld
/opt/conda/lib/python3.9/site-packages/twine/auth.py:75: UserWarning: No recommended backend was available. Install a recommended 3rd party backend package; or, install the keyrings.alt package if you want to use the non-recommended backends. See https://pypi.org/project/keyring for details.
  warnings.warn(str(exc))
Enter your password: 
Uploading hello_world-1.0.0-py3-none-any.whl
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 5.69k/5.69k [00:00<00:00, 8.62kB/s]
Error during upload. Retry with the --verbose option for more details.
HTTPError: 403 Forbidden from https://test.pypi.org/legacy/
The user 'Boris.Feld' isn't allowed to upload to project 'hello-world'. See https://test.pypi.org/help/#project-name for more information.

Steps to Reproduce

Try to upload a file with pkginfo version 1.8.2 installed.

bhrutledge commented 2 years ago

Hm. I'm not able to reproduce that error. I cloned https://github.com/buildbot/hello-world and made the same change to setup.py.

% twine --version
twine version 3.7.1 (importlib_metadata: 4.9.0, pkginfo: 1.8.2, requests: 2.26.0, requests-toolbelt: 0.9.1, tqdm: 4.62.3)

% python3 -m build
...
Successfully built hello_world-1.0.0.tar.gz and hello_world-1.0.0-py3-none-any.whl

% twine upload -r testpypi dist/*
...
Invalid or non-existent authentication information. See https://test.pypi.org/help/#invalid-auth for more information.

I am able to reproduce it if I downgrade pkginfo:

% pip install 'pkginfo<1.8'
...
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
twine 3.7.1 requires pkginfo>=1.8.1, but you have pkginfo 1.7.1 which is incompatible.
Successfully installed pkginfo-1.7.1

% twine --version
twine version 3.7.1 (importlib_metadata: 4.9.0, pkginfo: 1.7.1, requests: 2.26.0, requests-toolbelt: 0.9.1, tqdm: 4.62.3)

% twine upload -r testpypi dist/*
...
AttributeError: 'Wheel' object has no attribute 'dynamic'

Are you sure you were using pkginfo==1.8.2 when you got the traceback? It looks like Twine was executing from a global conda environment, which makes me wonder what else is installed in that environment, and if pkginfo was somehow out of sync with Twine.

Did you try running Twine from its own virtual environment? pipx is a great way to do that.

Lothiraldan commented 2 years ago

Hi @bhrutledge, I think you are right. I have opened a bug too early, I had an older version of pkginfo installed in the conda environment that somehow was loaded instead of the up-to-date pkginfo that I installed with pip. Feel free to close this issue and sorry about the noise again.

vfdev-5 commented 2 years ago

@bhrutledge I have a similar issue but pkginfo is up to date.

twine --version
> 
twine version 3.7.1 (importlib_metadata: 4.8.2, pkginfo: 1.8.2, requests:
2.26.0, requests-toolbelt: 0.9.1, tqdm: 4.62.3)

GHA logs: https://github.com/pytorch/ignite/runs/4589104222?check_suite_focus=true#step:7:413

However I can not reproduce the issue locally neither. Any hints on how to fix the problem ? Thanks

EDIT: looks like it is again pkginfo version issue. conda package related steps reinstall pkginfo==1.7.2 (don't know why) but twine reports newer versions and certainly uses older ones...

bhrutledge commented 2 years ago

@vfdev-5 From that workflow run, it looks like "Build and Publish Conda binaries" is installing pkginfo==1.7.1 after Twine and pkginfo==1.8.2 are installed in "Install dependencies": https://github.com/pytorch/ignite/runs/4589104222?check_suite_focus=true#step:6:269

So, one workaround might be install Twine in "Build and Publish PyPi binaries". Another option might be to skip the pip install twine and use pipx to isolate its environment, e.g.

pipx run twine check dist/*
pipx run twine upload --verbose dist/*

It does seem odd that Twine is reporting 1.8.2 (via importlib_metadata.version) but apparently importing 1.7.1. @jaraco Any insight?

pypa / twine