pypa / gh-action-pypi-publish

The blessed :octocat: GitHub Action, for publishing your :package: distribution files to PyPI, the tokenless way: https://github.com/marketplace/actions/pypi-publish
https://packaging.python.org/guides/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows/
BSD 3-Clause "New" or "Revised" License
947 stars 89 forks source link

v1.12 still failing with error similar to #291 #300

Open sroet opened 2 weeks ago

sroet commented 2 weeks ago

Dear developer,

Since the release of 1.12, our (self-hosted, rootless docker) workflow started failing with: /opt/conda/bin/python3: can't open file '/home/githubrunner/actions-runner/_work/_actions/pypa/gh-action-pypi-publish/release/v1.12/create-docker-action.py': [Errno 2] No such file or directory

Which seems similar to #291 (the error), but wasn't fixed with the release of 1.12.2

Here's the failing workflow run https://github.com/SBC-Utrecht/pytom-match-pick/actions/runs/11778522196/job/32805113391#step:7:58

Please let me know if we can help tracking down this issue.

xref: https://github.com/SBC-Utrecht/pytom-match-pick/issues/236

rjdbcm commented 1 week ago

Confirm 1.12.2 still fails for my nested composite action.

webknjaz commented 1 week ago

@sroet have you tried checking the contents of /home/githubrunner/actions-runner/_work/_actions/? That path is constructed using ${{ github.action_path }} which is supposed to contain a checked out copy of the action, which in turn would have create-docker-action.py in it. And since it's not there, I'd assume a bug in the GitHub Runner software.

Also, /opt/conda/bin/python3 is untypical for usual GH Runners too.

ferhatys commented 1 week ago

I'm getting the same issue for some reason.

Run pypa/gh-action-pypi-publish@release/v1
Run # Reset path if needed
Run # Set repo and ref from which to run Docker container action
Run # 🔎 Discover pre-installed Python
python-path=/usr/local/bin/python3
Run # Create Docker container action
  # Create Docker container action
  /usr/local/bin/python3 '/home/runner/work/_actions/pypa/gh-action-pypi-publish/release/v1/create-docker-action.py'
  shell: bash --noprofile --norc -e -o pipefail {0}
  env:
    REF: release/v1
    REPO: pypa/gh-action-pypi-publish
    REPO_ID: 609775869
/usr/local/bin/python3: can't open file '/home/runner/work/_actions/pypa/gh-action-pypi-publish/release/v1/create-docker-action.py': [Errno 2] No such file or directory
Error: Process completed with exit code 2.
ferhatys commented 1 week ago

Confirmed release/v1.11 is working for me as a workaround.

sroet commented 1 week ago

@sroet have you tried checking the contents of /home/githubrunner/actions-runner/_work/_actions/? That path is constructed using ${{ github.action_path }} which is supposed to contain a checked out copy of the action, which in turn would have create-docker-action.py in it. And since it's not there, I'd assume a bug in the GitHub Runner software.

Also, /opt/conda/bin/python3 is untypical for usual GH Runners too.

Hey @webknjaz, thanks for looking into this!

To repeat: we are running this on self-hosted runners inside a rootless docker, using the continuumio/miniconda3 image. That image has the default conda installation in /opt/conda, this is also where the /opt/conda/bin/python3 comes from.

Now for /home/githubrunner/actions-runner/_work/_actions/:

That directory exists on the host system of the runner, but looking at this line of the setup, specifically:

-v "/home/githubrunner/actions-runner/_work/_actions":"/__w/_actions"

it seems to get mounted at /__w/_actions instead.

I don't know enough about creating actions to know if this is a bug in the Runner software or if these variables that are not meant to be used inside actions after the container creation

webknjaz commented 1 week ago

Yeah, I understand, but can you check what's actually on dist in the runner host and within container (just add some recursive ls or something at the beginning of the job).

sroet commented 1 week ago

@webknjaz, started a debug workflow in https://github.com/SBC-Utrecht/pytom-match-pick/pull/241. Should we move any back and forth discussion about what folders / variables you want to check to there?

nikaro commented 1 week ago

@sroet have you tried checking the contents of /home/githubrunner/actions-runner/_work/_actions/? That path is constructed using ${{ github.action_path }} which is supposed to contain a checked out copy of the action, which in turn would have create-docker-action.py in it. And since it's not there, I'd assume a bug in the GitHub Runner software.

@webknjaz this is a known issue, the workaround seems to be using $GITHUB_ACTION_PATH instead of ${{ github.action_path }}.

sroet commented 1 week ago

@webknjaz, started a debug workflow in https://github.com/SBC-Utrecht/pytom-match-pick/pull/241. Should we move any back and forth discussion about what folders / variables you want to check to there?

To report back to here; After some debugging we indeed ran into the issue mentioned by @nikaro and I opened #304 to implement the proposed solution.

For my project however, I decided to restructure my release workflow to follow to current best practices around not having the id-token: write at build time and also sidestep the container issue. In case anyone else is interested this is the current workflow file.

In general; the workflow now has 4 jobs each depending on the previous one to succeed, with the type of runner in []: 1) [self-hosted] build the package and test with twine check, upload artifact 2) [github] download artifact and upload the distribution to testPyPI 3) [self-hosted] pull the latest version from testPyPI and run the test suite 4) [github] download artifact and upload the distribution to PyPI

I am fine closing this issue, but maybe it is nice to keep open until a decision has been made on #304

virtuald commented 4 days ago

This is broken on github actions runners that use containers. Using release/v1.11 as a workaround.