actions / cache

Cache dependencies and build outputs in GitHub Actions
MIT License
4.39k stars 1.17k forks source link

No cache being saved with Path Validation Errors #1177

Closed glatterf42 closed 7 months ago

glatterf42 commented 1 year ago

Hi everyone, please let me know if this is not the right place to report this issue/ask this question, but I don't quite know who else might be able to clarify this. Recently, our conda-forge CI tests started to fail (see here for the last successful run and here for the first failed one). These tests are run on the latest Windows and MacOS runner images that GitHub provides. While researching why the tests fail, I discovered that they were not using any cache despite being intended to. Here is the relevant part of our workflow file:

jobs:
  conda:
    strategy:
      matrix:
        os: [windows-latest, macos-latest]
        conda:
        - {installer: anaconda, version: 2023.03-1}
        - {installer: miniconda, version: py310_23.3.1-0}
        extra-deps: ["", "JPype1=1.2.1"]
      fail-fast: false

    runs-on: ${{ matrix.os }}
    name: ${{ matrix.os }}-${{ matrix.conda.installer }}-${{ matrix.extra-deps }}

    steps:
    - name: Cache Anaconda installer, conda packages
      uses: actions/cache@v3
      with:
        path: |
          $CONDA/pkgs
          ~/.conda/pkgs
          ~/anaconda3/pkgs
          ~/appdata/local/conda/conda/pkgs
          ${{ github.workspace }}/Anaconda*.exe
        key: ${{ matrix.os }}-${{ matrix.conda.installer }}

When running this workflow, even if the installation is successful, no cache is saved and I only receive the following error message:

Warning: Path Validation Error: Path(s) specified in the action for caching do(es) not exist, hence no cache is being saved.

Finally, here is the output that conda info produces during the run on Windows:

conda info

     active environment : None
       user config file : C:\Users\runneradmin\.condarc
 populated config files : 
          conda version : 23.3.1
    conda-build version : 3.24.0
         python version : 3.10.9.final.0
       virtual packages : __archspec=1=x86_64
                          __win=0=0
       base environment : D:\a\_actions\iiasa\actions\main\setup-conda\Anaconda3  (writable)
      conda av data dir : D:\a\_actions\iiasa\actions\main\setup-conda\Anaconda3\etc\conda
  conda av metadata url : None
           channel URLs : https://repo.anaconda.com/pkgs/main/win-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/win-64
                          https://repo.anaconda.com/pkgs/r/noarch
                          https://repo.anaconda.com/pkgs/msys2/win-64
                          https://repo.anaconda.com/pkgs/msys2/noarch
          package cache : D:\a\_actions\iiasa\actions\main\setup-conda\Anaconda3\pkgs
                          C:\Users\runneradmin\.conda\pkgs
                          C:\Users\runneradmin\AppData\Local\conda\conda\pkgs
       envs directories : D:\a\_actions\iiasa\actions\main\setup-conda\Anaconda3\envs
                          C:\Users\runneradmin\.conda\envs
                          C:\Users\runneradmin\AppData\Local\conda\conda\envs
               platform : win-64
             user-agent : conda/23.3.1 requests/2.28.1 CPython/3.10.9 Windows/10 Windows/10.0.20348
          administrator : True
             netrc file : None
           offline mode : False

And this is the one on MacOS:

Run conda info

     active environment : None
       user config file : /Users/runner/.condarc
 populated config files : 
          conda version : 23.3.1
    conda-build version : 3.24.0
         python version : 3.10.9.final.0
       virtual packages : __archspec=1=x86_64
                          __osx=10.16=0
                          __unix=0=0
       base environment : /Users/runner/work/_actions/iiasa/actions/main/setup-conda/Anaconda3  (writable)
      conda av data dir : /Users/runner/work/_actions/iiasa/actions/main/setup-conda/Anaconda3/etc/conda
  conda av metadata url : None
           channel URLs : https://repo.anaconda.com/pkgs/main/osx-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/osx-64
                          https://repo.anaconda.com/pkgs/r/noarch
          package cache : /Users/runner/work/_actions/iiasa/actions/main/setup-conda/Anaconda3/pkgs
                          /Users/runner/.conda/pkgs
       envs directories : /Users/runner/work/_actions/iiasa/actions/main/setup-conda/Anaconda3/envs
                          /Users/runner/.conda/envs
               platform : osx-64
             user-agent : conda/23.3.1 requests/2.28.1 CPython/3.10.9 Darwin/21.6.0 OSX/10.16
                UID:GID : 501:20
             netrc file : None
           offline mode : False

Please note that this output is for Anaconda, but the same happens when using Miniconda. The run on Windows that produced this conda info output failed in the end while the one on MacOS was successful.

One thing that I noted when looking at our workflow file is that the paths we provide for the cache action had not changed since they were introduced 3 years ago. Judging from the phrasing of the encountered error message, I think as soon as one of the paths provided for the cache action doesn't exist, no cache will be saved after successful execution of the workflow. Thus, I didn't expect our caching to work since the provided Anaconda paths would never be there for Miniconda and the appdata/local path would never be there on Windows. I couldn't find any documentation for the $CONDA environment variable either, but figured it would be better to remove that temporarily. This (and all subsequent tests) I did on iiasa/message_ix#715, which left me with ~/.conda/pkgs for the path provided to the cache action. Assuming that the conda info package cache output lists all directories where packages that we might want cached are stored, I thought this path should be there for both Windows and MacOS. But I still received the same error message of non-existent paths. Next, I tried other options (always removing those that didn't work before): '**/_actions/iiasa/actions/main/setup-conda/*/pkgs' and, due to #1125, '**/**/_actions/iiasa/actions/main/setup-conda/*/pkgs', but to no avail. I even tried hard-coding some paths for Windows:

                          C:\Users\runneradmin\.conda\pkgs
                          C:\Users\runneradmin\AppData\Local\conda\conda\pkgs

Still, I was told they didn't exist. So my question is: what am I missing here? Why is nothing being cached even if I provide paths to packages that conda itself tells me are used to store packages? And most importantly, how can I achieve caching for our workflow?

Please let me know if you need any further information. Many thanks in advance.

github-actions[bot] commented 7 months ago

This issue is stale because it has been open for 200 days with no activity. Leave a comment to avoid closing this issue in 5 days.

github-actions[bot] commented 7 months ago

This issue was closed because it has been inactive for 5 days since being marked as stale.