0.28 dev version not uploaded test pypi

[proppy]

It seems that the current development version (0.28) is not uploaded to test pypi: https://test.pypi.org/project/klayout/#history despite the upload_to_test_pypi job succeeding: https://github.com/KLayout/klayout/actions/runs/3615727326/jobs/6093247182

[proppy]

looking at the log of the jobs there seems to be some authorization error:

Warning:  It looks like you are trying to use an API token to authenticate in the package index and your token value does not start with "pypi-" as it typically should. This may cause an authentication error. Please verify that you have copied your token properly if such an error occurs.

25hWARNING  Error during upload. Retry with the --verbose option for more details. 
ERROR    HTTPError: 403 Forbidden from https://test.pypi.org/legacy/            
         Invalid or non-existent authentication information. See                
         https://test.pypi.org/help/#invalid-auth for more information

[proppy]

Not this could also be due to test pypi preventing a release for a given version to be overwritten.

According to https://stackoverflow.com/a/63944201 it should be possible to workaround this by leveraging {build tag} in the standard python artefact naming scheme: https://peps.python.org/pep-0427/#file-name-convention

[klayoutmatthias]

Problem I see is that on PyPI (even test instance) you cannot overwrite files after they have been uploaded once. I had to continuously update KLAYOUT_PYPI_VERSION in version.sh and I dont't want to spam PyPI with per-commit releases.

I think PyPI should be more generous to allow rolling releases for test purposes. As long as they are not, there will be distinct snapshots and no continuous updates on PyPI.

But maybe the artefact repositories are a way to obtain a current built?

[proppy]

Apparently there is a --build-number option you can pass to bdist_wheel to override the build tag part of the wheel naming scheme and rev a pypi upload.

I wonder if there is a way to pass that flag down from cibuildwheel

[klayoutmatthias]

Yes, I think that can be done (also patching version.sh works). But does it make sense to do that on every commit?

[proppy]

I think that would be fine for test pypi as long as you keep the production pypi for regular release.

[klayoutmatthias]

Okay, but wouldn't they mind being spammed by lots of releases?

[proppy]

Maybe @di could comment if it's fine to "spam" Test PyPI with continuous integration wheels.

[di]

I would generally advise against taking a dependency on TestPyPI as a service, our long-term plan is to shut it down: https://github.com/pypi/warehouse/issues/918. This seems fine for now though.

As long as this has a legitimate use case, we wouldn't consider this spam and as long as it falls within the existing project size limit (current 10GB per project) we probably wouldn't even notice.

[thomaslima]

It would be technically possible and 'easy' to have a git-based version string for the wheels. But as Matthias said it would cause significant spam -- every commit generates ~300MB of wheels. That's only 33 commits until we reach the project size limit mentioned by @di . I just recommend leaving it as it is. It's useful to test if the twine command is working before tagging a specific commit for the main pypi.