Errors saving and restoring cache

[safetypins]

I'm getting a few errors related to cache keys:

error computing cache key: template: cacheKey:1:11: executing "cacheKey" at <checksum "package.json">: error calling checksum: open /home/circleci/project/package.json: no such file or directory

Error computing cache key: template: cacheKey:1:11: executing "cacheKey" at <checksum "package.json">: error calling checksum: open /home/circleci/project/package.json: no such file or directory

Here's what the cache section of the config.yml file looks like:

# Download and cache dependencies
- restore_cache:
    keys:
      # "composer.lock" can be used if it is committed to the repo
      - v1-dependencies-{{ checksum "composer.json" }}
      # fallback to using the latest cache if no exact match is found
      - v1-dependencies-{{ checksum "package.json" }}

- run: composer install -n --prefer-dist

- save_cache:
    key: v1-dependencies-{{ checksum "composer.json" }}
    paths:
      - ./vendor
- restore_cache:
    keys:
      - node-v1-{{ checksum "package.json" }}
- run: yarn install
- save_cache:
    key: node-v1-{{ checksum "package.json" }}
    paths:
      - node_modules

I've made a few changes from the original:

# Download and cache dependencies
- restore_cache:
    keys:
      # "composer.lock" can be used if it is committed to the repo
      - v1-dependencies-{{ checksum "composer.json" }}
      # fallback to using the latest cache if no exact match is found
      - v1-dependencies-

- run: composer install -n --prefer-dist

- save_cache:
    key: v1-dependencies-{{ checksum "composer.json" }}
    paths:
      - ./vendor
- restore_cache:
    keys:
      - node-v1-{{ checksum "package.json" }}
      - node-v1-
- run: yarn install
- save_cache:
    key: node-v1-{{ checksum "package.json" }}
    paths:
      - node_modules

I don't really understand what the purpose of saving and restoring a cached version of these two files in this CircleCI testing environment is, considering the environments only live for a few minutes. Can I just get rid of these?

[safetypins]

I admit that I do not understand very much about how CircleCI works, but it looks to me like the install script is not creating a functioning block of commands. package.json is not listed in the restore_cache > keys section, and the package.json key is restored before it is saved.

[deviantintegral]

I'm not sure what's up with the errors themselves, but note that the filenames like package.json are being used as cache keys not cached files. This saves significant time by telling Circle that if package.json has been seen before, it can use the node_modules directory from a previous build.

I do wonder if these should both be lockfiles and not the package files. @juampynr ?

[juampynr]

Correct @deviantintegral. Looking at https://github.com/Lullabot/drupal8ci/blob/master/dist/circleci/.circleci/config.yml, the installer uses composer.lock to create cache keys. I see a mix of package and composer keys used above.

@safetypins how about if you start by removing cache management altogether? You can add it later once you get the workflow building correctly.

[fjgarlin]

This was removed as part of https://github.com/Lullabot/drupal9ci/pull/65

[davereid-pfg]

I'm curious why the caching was removed?

[fjgarlin]

No big reason really. As there was some refactoring needed to have it all working again, the caching was removed as part of it, trying to get the minimum code to work again, but we could defo try to add it back again.

It was also done as a way of standardization between the different platforms covered (Travis, Circle, Gitab, Github, Bitbucket), trying to make all those files "as similar as possible", but I guess once you pick a platform you can go ahead and optimize it as needed.

[fjgarlin]

@davereid-pfg @juampynr - as a follow-up from the caching. I tried to restore it here: https://github.com/fjgarlin/my-drupal9-site/commit/0385f85f530a071776985dd5e3acb1e898523104, where it generated a cache version of the vendor folder (build 7), however, when triggering another commit (ie: https://github.com/fjgarlin/my-drupal9-site/commit/8d0e07800d4a399331203c97839793739d43cef4) we can see that the jobs that were previously working no longer work due to the files not being found, despite the fact that the restoring of the cache was successful in the previous step as seen here:

(CI task url: https://app.circleci.com/pipelines/github/fjgarlin/my-drupal9-site/3/workflows/362e6aee-fc19-4e5a-afac-294009d04fd8/jobs/13)

That the issue I found back in the day and why I removed it as part of the D9 refactoring.

[fjgarlin]

More paths needed to be included when saving the cache. We can see the caching working here: https://app.circleci.com/pipelines/github/fjgarlin/my-drupal9-site/5/workflows/123b6d07-6eda-4f5c-8654-cca25ad927de/jobs/22

The PR above should bring back caching.