Open jmcvetta opened 3 years ago
Does it mean caching the plugins themselves? Or does it mean caching the tools installed by asdf?
We want to cache anything that is downloaded when running asdf install
against a .tool-versions
file.
With the cache action, it would look something like this:
- uses: actions/cache@v2
with:
path: |
~/.asdf/installs
~/.asdf/plugins
key: ${{ runner.os }}-${{ hashFiles('**/.tool-versions') }}
Currently we can't use the cache action with the asdf action. Because the asdf action runs git checkout
to install asdf. If the cache action has already populated ~/.asdf/installs
, then the git checkout fails because the folder ~/.asdf/
already exists.
In my use case caching works as expected:
... checkout
... asdf setup
- name: asdf cache
uses: actions/cache@v2
with:
path: ${{ env.ASDF_DIR }}
key: ${{ runner.os }}-${{ hashFiles('**/.tool-versions') }}
restore-keys: |
${{ runner.os }}-asdf-
... asdf add plugin
... asdf install
OK. We will make some major improvements in the near future, so let's do it together.
Many thanks @0x6a68, your snippet proved very useful (I'm caching the Erlang build which takes ages get currently, as described in https://github.com/asdf-vm/asdf-erlang/issues/165).
For those interested, the full script is available at https://github.com/thbar/ex-portmidi/pull/4
Currently we can't use the cache action with the asdf action. Because the asdf action runs git checkout to install asdf. If the cache action has already populated ~/.asdf/installs, then the git checkout fails because the folder ~/.asdf/ already exists.
@jmcvetta I have seen people using continue-on-error: true
with the Cache action to avoid this issue
Additionally, you can skip the asdf install
step if you get a cache hit with: https://github.com/actions/cache#skipping-steps-based-on-cache-hit
I think you could close this @jthegedus, the cache action works well.
steps:
- name: Checkout
uses: actions/checkout@v2
- name: Install asdf
uses: asdf-vm/actions/setup@v1
- name: asdf cache
id: asdf-cache
uses: actions/cache@v2
with:
path: ~/.asdf/
key: ${{ runner.os }}-${{ hashFiles('**/.tool-versions') }}
- name: Install CD tools from .tool-versions
if: steps.asdf-cache.outputs.cache-hit != 'true'
uses: asdf-vm/actions/install@v1
Thanks for all the hints above
Some benchmark from my abstract project:
asdf-vm/actions
: 50 secactions/node
: 40 secasdf-vm/actions
with caching: 30 secLooks like cache will be very useful (since @afirth snippet is long).
Thanks for the great snippet, @afirth; I expect this will speed up our workflows considerably! I agree that caching would be an excellent improvement to bake into the action itself. Even if the snippet exists, only a small percentage of users of the action will find it, and it downloads asdf itself twice in the event of a cache hit. In case this helps others, you only need to specify asdf_branch
to the setup
action if you don't want to use master
since the setup step will be skipped by the install
action. Also, I expect that users of this snippet will encounter issues where outdated versions of files in asdf core from the cache overwrite an updated asdf core. I suggest both pinning a release using the asdf_branch
parameter to the setup
action and including that release in the cache key so that updates to asdf itself invalidate the cache.
When trying to restore the cache currently, we get issues because it seems like node
isn't restored properly. No clue why though.
@snowe2010, can you offer more details? We are using the approach I described successfully, and we do use the node plugin. See ScribeMD/pre-commit-action for an example.
I think you could close this @jthegedus, the cache action works well.
steps: - name: Checkout uses: actions/checkout@v2 - name: Install asdf uses: asdf-vm/actions/setup@v1 - name: asdf cache id: asdf-cache uses: actions/cache@v2 with: path: ~/.asdf/ key: ${{ runner.os }}-${{ hashFiles('**/.tool-versions') }} - name: Install CD tools from .tool-versions if: steps.asdf-cache.outputs.cache-hit != 'true' uses: asdf-vm/actions/install@v1
Thanks for all the hints above
Thanks @afirth, this solution works on our project as well, should we add this into the README file so other developer can follow it 😄
It can works without the if: steps.asdf-cache.outputs.cache-hit != 'true'
https://github.com/asdf-vm/actions/issues/445#issuecomment-1129738813
It would be great to document the recipe more visibly, but it would need to be corrected first as mentioned previously:
Thanks for the great snippet, @afirth; I expect this will speed up our workflows considerably! I agree that caching would be an excellent improvement to bake into the action itself. Even if the snippet exists, only a small percentage of users of the action will find it, and it downloads asdf itself twice in the event of a cache hit. In case this helps others, you only need to specify
asdf_branch
to thesetup
action if you don't want to usemaster
since the setup step will be skipped by theinstall
action. Also, I expect that users of this snippet will encounter issues where outdated versions of files in asdf core from the cache overwrite an updated asdf core. I suggest both pinning a release using theasdf_branch
parameter to thesetup
action and including that release in the cache key so that updates to asdf itself invalidate the cache.
@Kurt-von-Laven I think it is due to using a custom build machine that is shared with projects that have different node versions. The node version somehow gets mangled or not updated correctly.
@snowe2010, what “it” are you referring to? Are you asking a question, and if so, what question? What are you trying to achieve, and how are you currently going about it?
To give you more information might take a while. I pretty much just trashed the asdf caching for the projects I was having issues on because I couldn't get node to work correctly.
Makes sense. To try to help you I would need to see the relevant excerpt of your workflow and any pertinent errors, warnings, logs, etc., but obviously your call whether it's worth investigating.
Just coming here to say that I also used that and it works great. https://github.com/react-native-community/rn-diff-purge/blob/master/.github/workflows/new_release.yml
Idea: could this make use of the same RUNNER_TOOL_CACHE
mechanism that the Python ASDF action uses? See:
I realize it's adjacent to the discussion here, crosslinking it:
The action seems to pick existing data properly now, adding this to your pipeline should work
- name: Cache asdf
uses: actions/cache@v3
with:
path: |
~/.asdf
key: ${{ runner.os }}-asdf-${{ hashFiles('**/.tool-versions') }}
works like a charm:
~/.asdf/plugins
, ~/.asdf/installs
steps:
- name: Setup ASDF itself
uses: asdf-vm/actions/setup@v2
- name: Restore ASDF tools from cache
id: asdf-tools-cache
uses: actions/cache@v3
with:
key: asdf-tools-${{ hashFiles('.tool-versions') }}
restore-keys: |
asdf-tools-
path: |
${{ env.ASDF_DIR }}/plugins
${{ env.ASDF_DIR }}/installs
- name: Install ASDF tools on cache-miss
if: ${{ steps.asdf-tools-cache.outputs.cache-hit != 'true' }}
uses: asdf-vm/actions/install@v2
- name: Reshim installed ASDF tools
shell: bash
run: asdf reshim
Any reason not to simply use ${{ env.ASDF_DIR }}
as the cache path? Not sure why, but we don't reshim, and it still works at least for the plugins we are using.
Any reason not to simply use
${{ env.ASDF_DIR }}
as the cache path? Not sure why, but we don't reshim, and it still works at least for the plugins we are using.
Definitely, according to the documentation ${{ env.ASDF_DIR }}
depending on the installation either could be a ASDF git repo clone, or it might contain other artifacts not needed to be cached.
That's why install asdf first, restore previously installed plugins/tools, and reshim to match current versions from .tool-versions file.
https://github.com/actions/cache/pull/1328 adds documentation re. this to the cache action.
It would be very useful if the Github Actions cache system could be integrated with this action. It should be able to save significant time when installing a large list of tools.