actions-rs / toolchain

🛠️ GitHub Action for `rustup` commands
https://github.com/marketplace/actions/rust-toolchain
MIT License
579 stars 89 forks source link

Advice on toolchain caching #54

Open HadrienG2 opened 4 years ago

HadrienG2 commented 4 years ago

So, I have a workflow with a relatively big build matrix composed of many short jobs. As a result, toolchain downloads are one of the CI perf bottlenecks, which leads me to wonder if I couldn't speed things up by caching the toolchain.

Unfortunately, all my attempts at doing so failed so far, IIUC due to a mixture of file permission issues and my lack of knowledge on the directory layout used by the "toolchain" action. I think it would help if you either explained how to do it in the toolchain action's documentation, or clarified that it cannot be done in the current rustup installation directory layout if that's the issue.

djc commented 4 years ago

It seems we could use this: https://github.com/actions/cache/blob/master/examples.md#rust---cargo

HadrienG2 commented 4 years ago

I tried to use it in various ways, but did not quite get it to work. Unfortunately, I did not take notes at the time, and the GH Actions environment is not amenable to quick replication, but from memory...

Since the GH action environment seems to be peculiarly configured in various ways, and does not provide tools for easy experimentation like similar docker containers that you can run on your machine or interactive SSH sessions, documentation and advice on what can be cached and where it's located would be nice.

jhpratt commented 4 years ago

For what it's worth, I just tried using action/cache@v1 in combination with overriding permissions (to avoid errors), and found minimal difference in speed. It seems the network is the restricting factor, not anything else.

lcdr commented 4 years ago

So, for some reason the ~/.rustup directory itself won't be cached properly, but caching its contents works. The following setup works for me:


- name: Cache toolchain
  uses: actions/cache@v2
  with:
    path: |
      ~/.rustup/toolchains
      ~/.rustup/update-hashes
      ~/.rustup/settings.toml
    key: toolchain-${{ hashFiles('rust-toolchain') }}

This caches based on the contents of the rust-toolchain file.

HadrienG2 commented 4 years ago

I suspect that ~/.rustup might be a symlink, which would explain why caching it doesn't work while caching its content does. But indeed, I ended up using a similar strategy as yours, and with cache v2 this leads to a net speed improvement at last!

      - name: Cache toolchain
        if: github.event_name != 'schedule'
        uses: actions/cache@v2
        with:
          path: |
            ~/.rustup/settings.toml
            ~/.rustup/toolchains/${{ matrix.rust }}-*
            ~/.rustup/update-hashes/${{ matrix.rust }}-*
          key: run-toolchain-rust_${{ matrix.rust }}
remagpie commented 3 years ago

Running ls -alh on the home folder shows the following result:

Run ls -alh ~/
total 60K
drwxrwxrwx  11 runner docker 4.0K Apr 19 12:10 .
drwxr-xr-x+  5 root   root   4.0K Apr 16 17:25 ..
-rwxrwxrwx   1 runner docker  220 Apr 16 17:25 .bash_logout
-rwxrwxrwx   1 runner docker   67 Apr 16 17:25 .bash_profile
-rwxrwxrwx   1 runner docker 3.7K Apr 16 17:25 .bashrc
lrwxrwxrwx   1 runner docker   22 Apr 16 17:25 .cargo -> /usr/share/rust/.cargo
drwxrwxrwx   2 runner docker 4.0K Apr 16 17:25 .composer
drwxrwxrwx   4 runner docker 4.0K Apr 19 12:10 .config
drwx------   2 runner docker 4.0K Apr 16 17:28 .docker
lrwxrwxrwx   1 runner docker   17 Apr 16 17:25 .ghcup -> /usr/local/.ghcup
drwxrwxrwx   6 runner docker 4.0K Apr 16 17:25 .nvm
-rwxrwxrwx   1 runner docker  807 Apr 16 17:25 .profile
lrwxrwxrwx   1 runner docker   23 Apr 16 17:25 .rustup -> /usr/share/rust/.rustup
drwxr-xr-x   2 runner root   4.0K Apr 16 17:28 factory
drwxr-xr-x   2 runner docker 4.0K Apr 19 12:10 perflog
drwxr-xr-x   4 runner root   4.0K Apr 16 17:28 runners
drwxr-xr-x   2 runner root   4.0K Apr 16 17:28 warmup
drwxr-xr-x   6 runner root   4.0K Apr 19 12:10 work

Caching /usr/share/rust/.rustup(the destination folder of symlink at ~/.rustup) worked for me.

ebkalderon commented 1 year ago

Turns out that caching the ~/.rustup directory itself now works perfectly fine on actions/cache@v3, so there's no need to drill down into its contents anymore, as the above 3 comments all suggest. Caching the toolchain is pretty simple these days:

- id: cache-rustup
  name: Cache Rust toolchain
  uses: actions/cache@v3
  with:
    path: ~/.rustup
    key: toolchain-${{ matrix.os }}-${{ matrix.rust-version }}

- if: ${{ steps.cache-rustup.outputs.cache-hit != 'true' }}
  name: Install Rust toolchain
  uses: actions-rs/toolchain@v1
  with:
    profile: minimal
    toolchain: ${{ matrix.rust-version }}
    override: true
    components: clippy, rustfmt

Given a list of runner labels (matrix.os) and a list of valid Rust toolchain versions (matrix.rust-version), the above code should create an OS- and version-specific toolchain cache, e.g. toolchain-ubuntu-latest-1.67.1.

Technically, the matrix.os component above can be safely omitted, because caches cannot be reused between different runners (source).