Intermitent Caching Issues with Python and virtualenv

[motherofcoconuts]

Hi all. We are suddenly experiencing problems across our workflows that leverage caching. The issue is contained in the image below.

The relevant workflow steps are:

    - name: Install latest version of Poetry
      shell: bash
      if: steps.cache-poetry.outputs.cache-hit != 'true'
      run: |
        curl -sSL https://install.python-poetry.org | python -
        echo "$HOME/.poetry/bin" >> $GITHUB_PATH

    - name: Load cached venv
      id: cached-poetry-dependencies
      uses: actions/cache@v3
      with:
        path: .venv
        key: venv-${{ runner.os }}-${{ steps.setup-python.outputs.python-version }}-${{ hashFiles('**/poetry.lock') }}

    - name: Install dependencies
      shell: bash
      env:
        PAT_TOKEN: ${{ inputs.pat-token }}
      if: steps.cached-poetry-dependencies.outputs.cache-hit != 'true'
      run: |
        poetry config virtualenvs.create true
        poetry config virtualenvs.in-project true
        poetry install --no-interaction --no-root

      - name: Run Tests
        run: |
          . .venv/bin/activate
          pytest -sv --cov-report=xml src/${{ matrix.test-path }}

If we disable caching the tests pass fine. With cacheing enabled we are seeing failures 70-80% of the time. We also found that deleting the caches themselves did not help; only disabling cacheing as a whole. These issues surfaced about 3 days ago. No workflow or package changes were made to our repo during that time.

This is running on ubuntu-latest.

[ChristopherMacGown]

Not certain if you were able to solve the problem, but we started seeing it as well between workflow runs were we were restoring a cache for a job that conditionally ran based on the branch it was on. What I discovered while trying to figure out whether or not it was a data race between two competing matrix jobs, was that our incidence of the issue was due to the poetry virtualenv's python binary pointing to a symlink that no longer existed in /opt/hostedtoolcache. We were able to resolve it by using the built-in caching provided by action/setup-python.

jobs:
  test-python:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: snok/install-poetry@v1
      - id: setup-python
        uses: actions/setup-python@v4
        with:
          python-version: 3.9
          cache: "poetry"
      - run: poetry install ...  # install your packages
      - run: poetry run ...      # run your test suite.
  deploy:
    needs: [...],
    if: ${{ !failure() && github.ref == 'refs/heads/main' }}
    steps:
      - uses: actions/checkout@v4
      - uses: snok/install-poetry@v1
      - id: setup-python
        uses: actions/setup-python@v4
        with:
          python-version: 3.9
          cache: "poetry"
      - id: ensure-virtualenv-is-sane
        run: .venv/bin/python3 -c "print('hello from .venv/bin/python')"
      - run: |
          source .venv/bin/activate 
          # do deployment stuff

[github-actions[bot]]

This issue is stale because it has been open for 200 days with no activity. Leave a comment to avoid closing this issue in 5 days.

[github-actions[bot]]

This issue was closed because it has been inactive for 5 days since being marked as stale.