actions / toolkit

The GitHub ToolKit for developing GitHub Actions.
https://github.com/features/actions
MIT License
4.95k stars 1.42k forks source link

Allow updating an existing cache #505

Open davidsbond opened 4 years ago

davidsbond commented 4 years ago

GitHub actions supports caching across builds which is super powerful. However, once a cache has been saved it can no longer be modified. While this is fine for most scenarios where you need the cache there are a few times when you would like to update it.

I created this PR on the actual cache action to allow updating a cache when providing it in configuration:

https://github.com/actions/cache/pull/353

I created it after reading the related issue:

https://github.com/actions/cache/issues/342

However, it seems like something either needs to change in this package or server-side on the actions cache.

My use-case for wanting to update a cache is when using GitHub actions in golang projects. Go supports a test cache which allows tests to be skipped if the code they use has not changed, which can significantly improve test times (especially in large projects). Each time I run my tests, I'd like to maintain a single cache for tests, that way, when a run completes I can update the cache and all my workflows share a single test cache

There are some extra use-cases described on the issue above.

smorimoto commented 3 years ago

I would also like to support this proposal.

half2me commented 3 years ago

I need this too

eLarocque commented 3 years ago

same

CAMOBAP commented 1 year ago

+

Blissful89 commented 1 year ago

poke

hyperxpro commented 1 year ago

+1

stephanrotolante commented 1 year ago

+1

DenizUgur commented 1 year ago

+1

emrebayramc commented 1 year ago

+1

YegorMedvedev commented 1 year ago

In the meantime, here's a workaround that I use:

- name: Get cache
  uses: actions/cache@v3
  with:
    path: ... # a list of cache paths
    key: ${{ runner.os }}-my-cache-${{ github.head_ref }}-${{ hashFiles(...for example yarn.lock...) }}

- name: Do something to change a cache
  run: ...

- name: Enforce cache update for a local NX cache
  run: |
    STATE_CACHE_KEY="${{ runner.os }}-my-cache-${{ github.head_ref }}-${{ hashFiles(...) }}"
    echo "STATE_CACHE_KEY=${STATE_CACHE_KEY}" >> $GITHUB_ENV
emrebayramc commented 1 year ago

+1

wind57 commented 1 year ago

+1

arpanrec commented 1 year ago

+1

bexsoft commented 1 year ago

+1

nnsay commented 1 year ago

+1

dsame commented 1 year ago

Updating the cache is supposed to be done with restore-keys

This input allows to restore the cache even if it is not fully matched. And to generate the new key to update the cache do one of:

  1. include the changing file into ${{ hashFiles(...) }}
  2. set output of the previous step and include it into the key
isaacbowen commented 1 year ago

@dsame This is exactly what I was missing. Thank you for sharing! Super super super helpful. ❤️

pauldraper commented 1 year ago

@dsame's workaround is the best available currently, though it does have problems.

Github hard-limits repos to 10GB of caches. So creating lots of caches can push other less-frequent-but-helpful caches to deletion. Whereas if you could update/replace an existing cache, that wouldn't happen.

lcswillems commented 3 months ago

On Gitlab or a lot of other different CI/CD tools, this is built-in :sob: Such a pain not to have this!

trallnag commented 3 weeks ago

I came across this documentation that uses artifacts across workflow runs as a workaround for this: https://github.com/renovatebot/github-action?tab=readme-ov-file#persisting-the-repository-cache

BillyTom commented 1 week ago

I have tried several workarounds to get actions/cache to update my existing test results correctly. But my conclusion is that it is much easier to use actions/upload-artifact and actions/download-artifact instead.

I have a matrix job that creates test results for a dozen different features. Each job in the matrix uploads its test results with:

  - name: "Upload test results"
    uses: actions/upload-artifact@v4
    with:
      name: test-result-${{ matrix.feature.name }}
      path: |
        frontend/cypress/screenshots
        frontend/cypress/reports

After the matrix job there comes another job that aggregates all test results into one single folder:

  - uses: actions/download-artifact@v4
    with:
      pattern: test-result-*
      merge-multiple: true
      path: frontend/cypress

After that you could just use actions/cache to create a new cache for the folder with the combined test results.

Optional: Since the individual artifacts are now obsolete you can perform a cleanup operation with:

  - name: "Delete obsolete artifacts"
    uses: geekyeggo/delete-artifact@65041433121f7239077fa20be14c0690f70569de
    with:
      name: test-result-*
      useGlob: true
Lolicchi commented 4 days ago

+8