/app/twine-upload.sh: line 45: GITHUB_WORKFLOW_REF: unbound variable

[WilliamStam]

using this action with gitea tho. everything was working great and then suddenly it stopped.

ive tried manually pushing to gitea's pypi repo and it works fine. but whenever i try publish using this action it errors out with

im including the full workflow action here. (it includes non relevant stuff sorry)

name: Build And Test
run-name: ${{ gitea.actor }} is running PyPI build and push
on: [push]

jobs:
  build:
    runs-on: ubuntu-latest
    container: catthehacker/ubuntu:act-latest
    environment: release
    permissions:
      id-token: write
    steps:
      - name: Info
        run: |
          echo "Trigger: ${{ gitea.event_name }} "
          echo "OS: ${{ runner.os }}"
          echo "Branch: ${{ gitea.ref }}"
          echo "Repository: ${{ gitea.server_url }}/${{ gitea.repository }}.git"
          echo "Workspace: ${{ gitea.workspace }}"
          echo "-----------"
          echo "${{ toJSON(gitea) }}"

      - name: Checkout Codebase
        uses: actions/checkout@v3

      - name: Setup Python
        run: |
          apt-get update && apt-get install -y python3-venv
          pip install --upgrade pdm

      - name: Setup PDM
        uses: pdm-project/setup-pdm@v3
        with:
          python-version-file: 'pyproject.toml'

      - name: Install Dependencies
        run: cd ${{ gitea.workspace }} && pdm install

      - name: Get Meta
        id: meta
        run: |
          echo REPO_NAME=$(echo ${GITHUB_REPOSITORY} | awk -F"/" '{print $2}') >> $GITHUB_OUTPUT
          echo REPO_VERSION=$(git describe --tags --always | sed 's/^v//') >> $GITHUB_OUTPUT      
          echo PACKAGE_VERSION=$(pdm show --version) >> $GITHUB_OUTPUT      
          echo PACKAGE_NAME=$(pdm show --name) >> $GITHUB_OUTPUT           

      - name: Build
        run: cd ${{ gitea.workspace }} && pdm build

      - name: Publish release
        uses: akkuman/gitea-release-action@v1
        with:
          files: |-
            ./dist/**
          name: '${{ steps.meta.outputs.PACKAGE_VERSION }}'
          tag_name: '${{ steps.meta.outputs.PACKAGE_VERSION }}'

      - name: "Publish dists to PyPI"
        uses: "pypa/gh-action-pypi-publish@release/v1"
        with:
          verbose: true
          skip-existing: true
          repository-url: ${{ gitea.server_url }}/api/packages/${{ gitea.repository_owner }}/pypi
          user: ${{ secrets.PYPI_USER }}
          password: ${{ secrets.PYPI_PASSWORD }}

line 45 of twine-upload.sh

WORKFLOW_FILENAME="$(echo ${GITHUB_WORKFLOW_REF} | cut -d'/' -f5- | cut -d'@' -f1)"

which only gets used in the "else"

if [[ "${TRUSTED_PUBLISHING}" == true || ! "${INPUT_REPOSITORY_URL}" =~ pypi\.org || ${#PACKAGE_NAMES[@]} -eq 0 ]] ; then
    TRUSTED_PUBLISHING_MAGIC_LINK_NUDGE=""
else
    ...
    for PACKAGE_NAME in "${PACKAGE_NAMES[@]}"; do
        LINK="- ${INDEX_URL}/manage/project/${PACKAGE_NAME}/settings/publishing/?provider=github&owner=${GITHUB_REPOSITORY_OWNER}&repository=${REPOSITORY_NAME}&workflow_filename=${WORKFLOW_FILENAME}"
        ALL_LINKS+="$LINK"$'\n'
    done

where does the issue lie here? my thinking is that gitea isnt setting the GITHUB_WORKFLOW_REF env in the action runner. but is it something that gitea should even be concerned with? shouldnt this action check if it exists and if so use it else ignore?

(gitea 1.22.2)

[webknjaz]

Hey, @WilliamStam! I see you're using a custom container. That was never really considered / tested / supported. It might've worked by accident. The missing environment variable should be injected by GitHub into their worker runtimes.

This might need some research and might be GitHub's bug. I see that it's used in a new feature that is designed to help users using long-living credentials migrate to tokenless publishing that uses short-lived tokens under the hood and does not require maintaining any secrets in the GH repo, nor tokens on the PyPI side. I recommend migrating to this method regardless of this issue.

Said feature was implemented by @facutuesca in a series of PRs: #250 / #258 / #270. It was first released in v1.10.2: https://github.com/pypa/gh-action-pypi-publish/discussions/264.

So your workarounds are:

• Dropping the custom runtime container from your job
• Pinning the action to v1.10.1
• Migrating to using trusted publishing (this will also include separating the dist build job from the publishing one due to the need to separate the privilege levels, so you could use the container for building but use regular ubuntu-latest for publishing)

P.S. It's best to also link actual CI runs in a GH repo so that we could look into the logs deeper. Rerunning the job in debug mode is also known to reveal more context.

[webknjaz]

@facutuesca would you be open to researching this and perhaps adding a smoke test job to our CI with a custom container: entry?

[WilliamStam]

yeah i was taking a chance here since this action is "for github" and im working with gitea.

pinning the version to 1.10.1 worked

(again sorry i know this is out of scope since its not github)

my initial thinking was if i simply set the variable to an empty string then at least it exists so shouldn't throw the error. but i couldn't get that right... my brain seems to still be on weekend mode

[facutuesca]

@facutuesca would you be open to researching this and perhaps adding a smoke test job to our CI with a custom container: entry?

@webknjaz The issue is due to using Gitea rather than GitHub to run the action. As @WilliamStam suggested, supporting the absence of GitHub-injected variables might be out of scope since this is a GH Action. WDYT?

[WilliamStam]

a simple solution would be to check if the variable exists and if so use it else ignore it or set a empty string to it if it doesn't exist since i think this is the only thing making it not compatible

if [ -n "${GITHUB_WORKFLOW_REF}" ]; then
  WORKFLOW_FILENAME="$(echo ${GITHUB_WORKFLOW_REF} | cut -d'/' -f5- | cut -d'@' -f1)"
else
  WORKFLOW_FILENAME=""
fi

[facutuesca]

@WilliamStam I think this might be a bug on Gitea's side, since they do seem to support GITHUB_* environment variables. See for example https://github.com/go-gitea/gitea/issues/25816, where they fix one of them.

Looking at their code, it looks like they do define GITHUB_WORKFLOW, but not GITHUB_WORKFLOW_REF.

Could you try printing $GITHUB_WORKFLOW in your action to see if that's the case? If so, we can open an issue on their repo asking them to add GITHUB_WORKFLOW_REF to their runner's context.

[woodruffw]

I agree with @facutuesca's analysis here -- this looks pretty firmly like a bug in Gitea's emulation of GH actions. @webknjaz's recommendation to downgrade to v1.10.1 should fix the proximate bug here, but it wouldn't surprise me at all if Gitea's attempt to reproduce GH actions has other subtle bugs, given that there's no standard for it 🙂

[webknjaz]

Wow, I didn't even realize that running actions on another platform is a thing. Thanks for the context!

Does it support annotations/job summaries? Maybe, we could track a set of env vars expected to exist somewhere and issue a warning whenever we suspect the env isn't GH?

I'm worried that if we attempted to maintain support for all the possible environments, we'd drown in burden.

@WilliamStam if you're not using GH, do you really need the action in the first place? Especially since you're unable to use tokenless publishing. It could be replaced with a twine check --strict followed by twine upload --attestations, more or less — we aren't doing a lot of fancy things beyond OIDC.