julia-actions / cache

A shortcut action to cache Julia artifacts, packages, and registries.
MIT License
38 stars 8 forks source link

Cache datadeps #40

Closed aviks closed 8 months ago

aviks commented 1 year ago

The DataDeps.jl package is useful in many situations that Artifacts do not handle. It would be nice to cache automatically the .julia/datadeps directory that this package uses for it's storage.

rikhuijzer commented 1 year ago

Could this maybe be solved on the DataDeps.jl side by adding to the DataDeps.jl README that people can use the following:

      - uses: actions/cache@v3
        with:
          path: '~/.julia/datadeps'
          key: 'datadeps-${{ hashFiles('src') }}'
          restore-keys: |
            datadeps-

Unlike the default Julia cache action (see below), this block will update the cache every time that any file in src changes and it will restore any earlier saved cache as long as it starts with datadeps-. Note that there is a risk of running out of allowed cache space especially when DataDeps.jl is used for large files.

The default Julia cache action looked something like (source):

      - uses: actions/cache@v1
        env:
          cache-name: cache-artifacts
        with:
          path: ~/.julia/artifacts
          key: ${{ runner.os }}-test-${{ env.cache-name }}-${{ hashFiles('**/Project.toml') }}
          restore-keys: |
            ${{ runner.os }}-test-${{ env.cache-name }}-
            ${{ runner.os }}-test-
            ${{ runner.os }}-
rikhuijzer commented 1 year ago

Adding .julia/datadeps by default is probably not going to work because julia-actions/cache uses all Project.toml files for determining the cache key (source) while DataDeps.jl dependencies are specified in the source code via register(DataDep("some key", "some value")).

SaschaMann commented 1 year ago

Could this maybe be solved on the DataDeps.jl side by adding to the DataDeps.jl README that people can use the following:

You could also consider adding an action.yml to DataDeps or a subdir of it so that people can add caching with something like uses: oxinabox/DataDeps.jl@main

rikhuijzer commented 8 months ago

This issue will be fixed if DataDeps.jl moves to scratch spaces, see https://github.com/oxinabox/DataDeps.jl/issues/135. scratch spaces are cached by default by julia-actions/cache.