Enabling Plugin Cache Directory

[bflad]

Description

We have a use case that requires running Terraform in a GitHub Action across multiple, separate configurations in the same repository. Since Terraform prefers to maintain its own plugin directory per directory, this leads to re-downloading (mostly the same) plugins for each separate directory. It would be great if there was the option to enable the plugin cache directory, which can be handled in the Terraform CLI configuration file via the plugin_cache_dir option, e.g. from the documentation:

plugin_cache_dir = "$HOME/.terraform.d/plugin-cache"

Seems like it could be either a boolean where the GitHub Action maintains the directory location itself (enable_plugin_cache_dir: true) or allow configurations to provide it directly (plugin_cache_dir: /wherever/whenever), which might be useful for managed runners at the expense of being more complex. It seems like in either case, the GitHub Action would should try to create the directory, skipping errors for existing ones, as its prior existence is required.

Potential Workarounds

env:
  TF_PLUGIN_CACHE_DIR: ${{ github.workspace }}/.terraform.d/plugin-cache

jobs:
  terraform:
    runs-on: ubuntu-latest
    steps:
    # alternatively: run: mkdir -p $TF_PLUGIN_CACHE_DIR
    - uses: actions/cache@v1
      with:
        key: ${{ runner.os }}-terraform-plugin-cache
        path: ${{ env.TF_PLUGIN_CACHE_DIR }}
    - uses: actions/checkout@v2
    - uses: hashicorp/setup-terraform@v1
    - name: terraform
      run: |
        for DIR in $(find ./examples -type d); do
          pushd $DIR
          terraform init
          terraform fmt -check
          terraform validate
          popd
        done

References

• https://github.com/terraform-providers/terraform-provider-aws/issues/12861

[nantiferov]

Thank you for workaround example, just want to mention that order of cache and checkout should be opposite. In your example checkout will clean cache.

So it should look like:

    - uses: actions/checkout@v2
    - uses: actions/cache@v1
      with:
        key: ${{ runner.os }}-terraform-plugin-cache
        path: ${{ env.TF_PLUGIN_CACHE_DIR }}

[jwenz723]

It is mentioned here that the plugin cache directory will not be created by the terraform cli. So you must ensure the directory exists before trying to cache anything. Here is an example of a workaround which is functioning for me:

env:
  TF_PLUGIN_CACHE_DIR: ${{ github.workspace }}/.terraform.d/plugin-cache

jobs:
  terraform:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v2

      - name: Setup Terraform
        uses: hashicorp/setup-terraform@v1.2.1
        with:
          terraform_wrapper: false
          terraform_version: '1.0.0'

      - name: Create Terraform Plugin Cache Dir
        run: mkdir --parents $TF_PLUGIN_CACHE_DIR

      - name: Cache Terraform
        uses: actions/cache@v2
        with:
          path: ${{ env.TF_PLUGIN_CACHE_DIR }}
          key: ${{ runner.os }}-terraform-${{ hashFiles('**/.terraform.lock.hcl') }}

[ctrlok]

I tried to set up caching, but I'm not sure it's working. At least I see no difference in terraform init time with and without caching.

Did you see the difference?

[danopia]

I tried to set up caching, but I'm not sure it's working.

To confirm if caching worked, check the init's output text, it should include "from the shared cache directory".

Snip from uncached provider download:

Initializing provider plugins...
- Reusing previous version of cloudflare/cloudflare from the dependency lock file
- Installing cloudflare/cloudflare v3.7.0...
- Installed cloudflare/cloudflare v3.7.0 (signed by a HashiCorp partner, key ID DE413CEC881C3283)

Snip from successful provider reuse:

Initializing provider plugins...
- Reusing previous version of cloudflare/cloudflare from the dependency lock file
- Using cloudflare/cloudflare v3.7.0 from the shared cache directory

Also, I believe the more-recent Github Actions YAML above will only restore from cache if the lockfile has an exact match

[audunsolemdal]

I've been trying to get this working on a single .lock.hcl file without any luck. From the docs it states that hashFiles() needs the path to be under the github_workspace folder

So key: ${{ runner.os }}-terraform-${{ hashFiles('**/.terraform.lock.hcl') }}

works

but would it be possible to get something like key: ${{ runner.os }}-terraform-${{ hashFiles('${{ inputs.workingDir }}/.terraform.lock.hcl') }} working, where workingDir can be "platform/dev" or "platform/test"?

[trallnag]

Hi @audunsolemdal, yes. The following works for me:

- name: Configure Terraform plugin cache
  run: |
    echo 'plugin_cache_dir = "$HOME/.terraform.d/plugin-cache"' > ~/.terraformrc
    mkdir -p ~/.terraform.d/plugin-cache
- name: Cache Terraform
  uses: actions/cache@v3
  with:
    path: ~/.terraform.d/plugin-cache
    key: ${{ runner.os }}-terraform-${{ hashFiles(format('{0}/.terraform.lock.hcl', env.TF_DIR)) }}
    restore-keys: ${{ runner.os }}-terraform-

TF_DIR in my case is envs/dev or envs/prod.

[leonardovillela]

If I understood correctly this issue is about adding the caching capability to the setup-terraform action instead of using the actions/cache. Is that right?

[allanlewis]

setup-python and setup-node have built-in caching functionality, it would be great if setup-terraform could do the same.