`bunder-cache: true` not getting a cache hit on most jobs

[kleinjm]

Ensure the following before filing this issue

• [X] I verified it reproduces with the latest version with - uses: ruby/setup-ruby@v1 (see Versioning policy)

• [X] I tried to reproduce the issue locally by following the workflow steps (including all commands done by ruby/setup-ruby, except for Downloading Ruby & Extracting Ruby), and it did not reproduce locally (if it does reproduce locally, it's not a ruby/setup-ruby issue)

Are you running on a GitHub-hosted runner or a self-hosted runner?

GitHub-hosted runner

The workflow code or a link to the workflow file

name: Test
on: [pull_request]
# only allow one workflow per branch
concurrency:
  group: ${{ github.ref }}
  cancel-in-progress: true
env:
  BUNDLE_GEMS__CONTRIBSYS__COM: ${{ secrets.BUNDLE_GEMS__CONTRIBSYS__COM }}
  BUNDLE_GITHUB__COM: x-access-token:${{ secrets.BUNDLE_GITHUB__COM }}
  RAILS_ENV: test

jobs:
  linters:
    # The linters should stay in sync with the linters run locally in the
    # lint-staged section of package.json (excluding any rewrite commands)
    name: Linters
    runs-on: ubuntu-22.04
    env:
      BUNDLE_WITHOUT: development:production
      RAILS_MASTER_KEY: ${{ secrets.RAILS_TEST_KEY }}

    steps:
      - name: Checkout code
        uses: actions/checkout@v3
        with:
          fetch-depth: 100 # Fix for https://github.com/danger/danger/issues/913

      - name: Setup Ruby
        uses: ruby/setup-ruby@v1
        with:
          bundler-cache: true

      - name: Setup NodeJS
        uses: actions/setup-node@v3
        with:
          node-version: 16.13.0
      - name: Find yarn cache location
        id: yarn-cache
        run: echo "yarn_cache_dir=$(yarn cache dir)" >> $GITHUB_ENV
      - name: Check yarn cache
        uses: actions/cache@v3
        with:
          path: ${{ env.yarn_cache_dir }}
          key: ${{ runner.os }}-yarn-${{ hashFiles('**/yarn.lock') }}
          restore-keys: |
            ${{ runner.os }}-yarn-
      - name: Install packages
        run: |
          yarn install --pure-lockfile

      - name: tsc
        uses: EPMatt/reviewdog-action-tsc@v1 # TODO: watch for an official action (or fork our own)
        with:
          reporter: github-pr-review
          fail_on_error: true

      # Note: this includes prettier rules for .js and .ts files
      - name: eslint
        uses: reviewdog/action-eslint@v1
        with:
          reporter: github-pr-review
          eslint_flags: 'app/javascript/ config/**/*.js --ext .js,.ts'
          fail_on_error: true

      # Run prettier for other files
      - name: prettier
        run: yarn prettier

      - name: stylelint
        uses: reviewdog/action-stylelint@v1
        with:
          reporter: github-pr-review
          stylelint_input: 'app/javascript/stylesheets/**/*.scss'
          stylelint_config: .stylelintrc.json
          fail_on_error: true

      - name: rubocop
        uses: reviewdog/action-rubocop@v2
        with:
          reporter: github-pr-review
          rubocop_version: gemfile
          rubocop_flags: --parallel --config=.rubocop_ci.yml
          fail_on_error: true

      - name: haml-lint
        uses: patch-technology/action-haml-lint@0.4
        with:
          reporter: github-pr-review
          rubocop_version: gemfile
          haml_lint_flags: -c .haml-lint_todo.yml
          fail_on_error: true

      - name: patch cops
        run: |
          bin/no_new_helpers

      - name: danger
        env:
          DANGER_GITHUB_API_TOKEN: ${{ secrets.DANGER_GITHUB_API_TOKEN }}
        run: bundle exec danger

      - name: brakeman
        uses: reviewdog/action-brakeman@v2
        with:
          brakeman_version: gemfile
          reporter: github-pr-review
          fail_on_error: false # wait until all failures have been fixed before enabling

      - name: lint_swagger
        run: bundle exec rake lint_swagger

      - name: Cancelling parallel jobs
        if: failure()
        uses: styfle/cancel-workflow-action@0.11.0

  # Checks that do not run in each parallel test worker but do require a database setup
  non_parallelized_checks:
    name: Non-parallelized Checks
    runs-on: ubuntu-22.04
    strategy:
      fail-fast: true
    env:
      PGUSER: patch
      PGHOST: localhost
      RAILS_MASTER_KEY: ${{ secrets.RAILS_TEST_KEY }}
      BUNDLE_WITHOUT: development:production
    services:
      postgres:
        image: postgres:12-alpine
        env:
          POSTGRES_USER: patch
          POSTGRES_DB: patch_test_
          POSTGRES_HOST_AUTH_METHOD: "trust"
        ports: ["5432:5432"]
        options: >-
          --health-cmd pg_isready
          --health-interval 25ms
          --health-timeout 500ms
          --health-retries 20
    steps:
      - name: Checkout code
        uses: actions/checkout@v3
      - name: Setup Ruby
        uses: ruby/setup-ruby@v1
        with:
          bundler-cache: true
      - name: Make bin/ci executable
        run: chmod +x ./bin/ci
      - name: i18n tests
        id: rspec-tests
        env:
          DD_SERVICE: patch
          DD_ENV: test
          DD_TRACING: false
          CI_NODE_TOTAL: 1
          CI_NODE_INDEX: 0
          CI_TEST_PATTERN: "spec/i18n_spec.rb"
        continue-on-error: false
        run: |
          ./bin/ci

      - name: database_consistency
        run: bundle exec database_consistency

      - name: Cancelling parallel jobs
        if: failure()
        uses: styfle/cancel-workflow-action@0.11.0

  tests-javascript:
    name: Javascript Tests
    runs-on: ubuntu-latest
    strategy:
      fail-fast: true
    steps:
      - name: Checkout code
        uses: actions/checkout@v3
      - name: Find yarn cache location
        id: yarn-cache
        run: echo "yarn_cache_dir=$(yarn cache dir)" >> $GITHUB_ENV
      - name: Check yarn cache
        uses: actions/cache@v3
        with:
          path: ${{ env.yarn_cache_dir }}
          key: ${{ runner.os }}-yarn-${{ hashFiles('**/yarn.lock') }}
          restore-keys: |
            ${{ runner.os }}-yarn-
      - name: Install packages
        run: |
          yarn install --pure-lockfile
      - name: Run jest tests
        continue-on-error: false
        run: yarn test

  tests:
    name: Tests
    runs-on: ubuntu-22.04
    strategy:
      fail-fast: false
      matrix:
        # Source: https://rubyyagi.com/how-to-run-tests-in-parallel-in-github-actions/
        ci_node_total: [4]
        ci_node_index: [0, 1, 2, 3]
    env:
      PGUSER: patch
      PGHOST: localhost
      REDIS_URL: redis://localhost:6379/0
      RAILS_MASTER_KEY: ${{ secrets.RAILS_TEST_KEY }}
      BUNDLE_WITHOUT: development:production
    services:
      postgres:
        image: postgres:12-alpine
        env:
          POSTGRES_USER: patch
          POSTGRES_DB: patch_test_
          POSTGRES_HOST_AUTH_METHOD: "trust"
        ports: ["5432:5432"]
        options: >-
          --health-cmd pg_isready
          --health-interval 25ms
          --health-timeout 500ms
          --health-retries 20
      redis:
        image: redis:alpine
        ports: ["6379:6379"]
        options: --entrypoint redis-server

    steps:
      - name: Checkout code
        uses: actions/checkout@v3

      - name: Setup Ruby
        uses: ruby/setup-ruby@v1
        with:
          bundler-cache: true

      - name: Setup Node
        uses: actions/setup-node@v3
        with:
          node-version: 16.13.0
      - name: Find yarn cache location
        id: yarn-cache
        run: echo "yarn_cache_dir=$(yarn cache dir)" >> $GITHUB_ENV
      - name: Check yarn cache
        uses: actions/cache@v3
        with:
          path: ${{ env.yarn_cache_dir }}
          key: ${{ runner.os }}-yarn-${{ hashFiles('**/yarn.lock') }}
          restore-keys: |
            ${{ runner.os }}-yarn-
      - name: Install packages
        run: |
          yarn install --pure-lockfile

      - name: Check assets cache
        uses: actions/cache@v3
        with:
          path: |
            public/assets
          key: asset-cache-${{ runner.os }}-${{ github.ref }}-${{ github.sha }}
          restore-keys: |
            asset-cache-${{ runner.os }}-${{ github.ref }}-${{ github.sha }}
            asset-cache-${{ runner.os }}-${{ github.ref }}-
            asset-cache-${{ runner.os }}-

      - name: Setup the test database and compile js/css
        run: bundle exec rails db:test:prepare --trace

      - name: Precompile assets
        run: bundle exec rails assets:precompile --trace

      - name: Make bin/ci executable
        run: chmod +x ./bin/ci
      - name: Run tests
        id: rspec-tests
        env:
          ASSET_COMPILE: false # assets are already pre-compiled
          DD_SERVICE: patch
          DD_ENV: test
          DD_TRACING: false
          CI_NODE_TOTAL: ${{ matrix.ci_node_total }}
          CI_NODE_INDEX: ${{ matrix.ci_node_index }}
          CI_TEST_PATTERN: "spec/**/*_spec.rb"
          CI_EXCLUDE_PATTERN: "spec/system/**/*,spec/i18n_spec.rb"
          CI_TEST_SUITE: "non-system"
        continue-on-error: false
        run: |
          ./bin/ci

      - name: Upload Coverage Artifact
        uses: actions/upload-artifact@v3
        if: success() # upload artifacts only when the previous step of a job succeeded
        with:
          name: coverage-result-${{ matrix.ci_node_index }}
          path: coverage/.resultset.json

  tests-system:
    name: System Tests
    runs-on: ubuntu-22.04
    strategy:
      fail-fast: false
      matrix:
        # Source: https://rubyyagi.com/how-to-run-tests-in-parallel-in-github-actions/
        ci_node_total: [4]
        ci_node_index: [0, 1, 2, 3]
    env:
      PGUSER: patch
      PGHOST: localhost
      REDIS_URL: redis://localhost:6379/0
      RAILS_MASTER_KEY: ${{ secrets.RAILS_TEST_KEY }}
    services:
      postgres:
        image: postgres:12-alpine
        env:
          POSTGRES_USER: patch
          POSTGRES_DB: patch_test_
          POSTGRES_HOST_AUTH_METHOD: "trust"
        ports: ["5432:5432"]
        options: >-
          --health-cmd pg_isready
          --health-interval 25ms
          --health-timeout 500ms
          --health-retries 20
      redis:
        image: redis:alpine
        ports: ["6379:6379"]
        options: --entrypoint redis-server

    steps:
      - name: Checkout code
        uses: actions/checkout@v3

      - name: Setup Ruby
        uses: ruby/setup-ruby@v1
        with:
          bundler-cache: true

      - name: Setup Node
        uses: actions/setup-node@v3
        with:
          node-version: 16.13.0
      - name: Find yarn cache location
        id: yarn-cache
        run: echo "yarn_cache_dir=$(yarn cache dir)" >> $GITHUB_ENV
      - name: Check yarn cache
        uses: actions/cache@v3
        with:
          path: ${{ env.yarn_cache_dir }}
          key: ${{ runner.os }}-yarn-${{ hashFiles('**/yarn.lock') }}
          restore-keys: |
            ${{ runner.os }}-yarn-
      - name: Install packages
        run: |
          yarn install --pure-lockfile

      - name: Check assets cache
        uses: actions/cache@v3
        with:
          path: |
            public/assets
          key: asset-cache-${{ runner.os }}-${{ github.ref }}-${{ github.sha }}
          restore-keys: |
            asset-cache-${{ runner.os }}-${{ github.ref }}-${{ github.sha }}
            asset-cache-${{ runner.os }}-${{ github.ref }}-
            asset-cache-${{ runner.os }}-

      - name: Setup test database and compile js:css
        run: bundle exec rails db:test:prepare --trace

      - name: Precompile assets
        run: bundle exec rails assets:precompile --trace

      - name: Make bin/ci executable
        run: chmod +x ./bin/ci
      - name: Run tests
        id: rspec-tests-system
        env:
          ASSET_COMPILE: false # assets are already pre-compiled
          DD_SERVICE: patch
          DD_ENV: test
          DD_TRACING: false
          CI_NODE_TOTAL: ${{ matrix.ci_node_total }}
          CI_NODE_INDEX: ${{ matrix.ci_node_index }}
          CI_TEST_PATTERN: "spec/system/**/*_spec.rb"
          CI_TEST_SUITE: "system"
        continue-on-error: false
        run: |
          ./bin/ci

      - name: Upload Coverage Artifact
        uses: actions/upload-artifact@v3
        if: success() # upload artifacts only when the previous step of a job succeeded
        with:
          name: coverage-result-system-${{ matrix.ci_node_index }}
          path: coverage/.resultset.json

      - name: Upload Screenshot Artifacts
        uses: actions/upload-artifact@v3
        if: failure() # upload artifacts only when the previous step of a job failed
        with:
          name: failure_screenshot
          path: |
            tmp/screenshots/*.png
            tmp/capybara/*.png

      - name: Upload Rails log
        uses: actions/upload-artifact@v3
        # upload artifacts only when the previous step of a job failed and
        # commit name starts with `[ci full]`
        if: failure() && startsWith(github.event.head_commit.message, '[ci full]')
        with:
          name: rails-log-system-${{ matrix.ci_node_index }}
          path: |
            log/*

  coverage-report:
    name: Coverage Report
    runs-on: ubuntu-22.04
    needs: [tests, tests-system]
    if: ${{ github.actor != 'dependabot[bot]' }}
    strategy:
      fail-fast: false
    env:
      RAILS_MASTER_KEY: ${{ secrets.RAILS_TEST_KEY }}

    steps:
      - name: Checkout code
        uses: actions/checkout@v3
      - name: Setup Ruby
        uses: ruby/setup-ruby@v1
        with:
          bundler-cache: true
      - name: Download Artifacts
        uses: actions/download-artifact@v3
      - name: Merge Coverage Report
        run: bundle exec rake coverage:report
      - name: Upload Coverage Report
        uses: actions/upload-artifact@v3
        with:
          name: coverage-report
          path: coverage/

      - uses: joshmfrankel/simplecov-check-action@main
        with:
          github_token: ${{ secrets.GITHUB_TOKEN }}
          minimum_suite_coverage: 95
          minimum_file_coverage: 40

Link to the log of a failed workflow job, or to a gist with the output

https://gist.github.com/kleinjm/234db05590cbd13e7b6e6a9eb75b1e9d

The command and output of the failing step

- name: Setup Ruby
        uses: ruby/setup-ruby@v1
        with:
          bundler-cache: true

Any other notes?

We run 2 ruby jobs for linting and checks and 2 jobs for tests; one unit and one system. The test jobs are parallelized 4X each.

The linting/check jobs and the unit tests never get a cache hit on the bundler cache. However, the system tests and reruns of failed unit tests do get a cache hit.

At the end of each job, the cache will either save or fail to save because it already exists.

I can see the cache key in the GH actions UI and have verified is exactly the same between jobs.

The issue is that the cache should be working on all jobs. Fixing this will shave about a minute off of each job so thanks in advance for the help.

[kleinjm]

Looks like this may be related to https://github.com/actions/cache/issues/485#issuecomment-744145040

I'm sure I'm not the only one to run into this though so I'm curious if others have workarounds.

[kleinjm]

Fixed! The issue was that my main base branch was bundling all gems and my feature branches were excluding dev & prod gems with BUNDLE_WITHOUT: development:production. Removing that env var fixed the issue by allowing the test builds to always use the same cache as the main branch.