Support publishing to test pypi

frankinspace commented 2 weeks ago

Description

The documentation suggests publishing a release from a fork to Test PyPi first to ensure the process works: https://jupyter-releaser.readthedocs.io/en/latest/how_to_guides/convert_repo_from_repo.html#initial-release-workflow

This is done by setting TWINE_REPOSITORY_URL: https://test.pypi.org/legacy/ environment variable on the finalize-release step in the workflow.

However, this fails with the following error:

Fetching PyPI OIDC token...
Fetching PyPI API token...
Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.11.10/x64/bin/jupyter-releaser", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/home/runner/work/_actions/jupyter-server/jupyter_releaser/v2/jupyter_releaser/cli.py", line 121, in invoke
    super().invoke(ctx)
  File "/opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/runner/work/_actions/jupyter-server/jupyter_releaser/v2/jupyter_releaser/cli.py", line 685, in publish_assets
    lib.publish_assets(
  File "/home/runner/work/_actions/jupyter-server/jupyter_releaser/v2/jupyter_releaser/lib.py", line 405, in publish_assets
    twine_token = python.get_pypi_token(release_url, python_package_path)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/runner/work/_actions/jupyter-server/jupyter_releaser/v2/jupyter_releaser/python.py", line 157, in get_pypi_token
    return fetch_pypi_api_token()
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/runner/work/_actions/jupyter-server/jupyter_releaser/v2/jupyter_releaser/python.py", line 140, in fetch_pypi_api_token
    r.raise_for_status()
  File "/opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/requests/models.py", line 1024, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 422 Client Error: Unprocessable Entity for url: https://pypi.org/_/oidc/github/mint-token
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/home/runner/work/_actions/jupyter-server/jupyter_releaser/v2/jupyter_releaser/actions/finalize_release.py", line 15, in <module>
    run_action("jupyter-releaser publish-assets")
  File "/home/runner/work/_actions/jupyter-server/jupyter_releaser/v2/jupyter_releaser/actions/common.py", line 25, in run_action
    _run(target, *args, **kwargs)
  File "/home/runner/work/_actions/jupyter-server/jupyter_releaser/v2/jupyter_releaser/util.py", line 94, in run
    raise e
  File "/home/runner/work/_actions/jupyter-server/jupyter_releaser/v2/jupyter_releaser/util.py", line 86, in run
    process = tee(cmd, **kwargs)
              ^^^^^^^^^^^^^^^^^^
  File "/home/runner/work/_actions/jupyter-server/jupyter_releaser/v2/jupyter_releaser/tee.py", line 159, in run
    raise subprocess.CalledProcessError(
subprocess.CalledProcessError: Command 'jupyter-releaser publish-assets' returned non-zero exit status 1.

The reason seems to be because the token url seems to be hard coded to pypi.org: https://github.com/jupyter-server/jupyter_releaser/blob/f477486b16a7297ee8d82ec8add2135885973e44/jupyter_releaser/python.py#L24

Reproduce

Fork a repo using jupyter_releaser
Setup trusted publishing on test pypi
Set TWINE_REPOSITORY_URL: https://test.pypi.org/legacy/ environment variable on the finalize-release step in the workflow
Run Prep Release then Publish Release on the fork. The publish will fail with error 422 Client Error: Unprocessable Entity for url: https://pypi.org/_/oidc/github/mint-token

Expected behavior

Package is published to test pypi

Context

Operating System and version: N/A
Browser and version: N/A
Jupyter Server version: N/A

Troubleshoot Output

Paste the output from running `jupyter troubleshoot` from the command line here.
You may want to sanitize the paths in the output.

Command Line Output

Paste the output from your command line running `jupyter lab` here, use `--debug` if possible.

Browser Output

Paste the output from your browser Javascript console here, if applicable.

frankinspace commented 2 weeks ago

I also tried working around this by not using trusted publishing and instead providing a PYPI_TOKEN env variable to finalize-release but could not get that to work either. That problem might still be user error though, not sure.

jtpio commented 2 days ago

I also tried working around this by not using trusted publishing and instead providing a PYPI_TOKEN env variable to finalize-release but could not get that to work either.

Interesting. This could have otherwise been a good enough workaround for now (not sure Test PyPI supports the trusted publishers?). What error did you encounter with this approach?

jupyter-server / jupyter_releaser