Clarifiy plugin_cache_dir behavior

[sebastianreloaded]

Terraform Version

terraform -version
Terraform v1.8.5
on darwin_amd64
+ provider registry.terraform.io/hashicorp/aws v5.54.1

Affected Pages

https://developer.hashicorp.com/terraform/cli/config/config-file#provider-plugin-cache

What is the docs issue?

The docs state under section "Provider Plugin Cache":

Given that provider plugins can be quite large (on the order of hundreds of megabytes), this default behavior can be inconvenient for those with slow or metered Internet connections. Therefore Terraform optionally allows the use of a local directory as a shared plugin cache, which then allows each distinct plugin binary to be downloaded only once.

This is wrong, it is not downloaded once, it is only stored once. Everytime terraform init is executed in a different working directory, it will download the plugin again, even if it exisits in the plugin_cache_dir already.

TF_PLUGIN_CACHE_DIR="$HOME/.terraform.d/plugin-cache" terraform init
Initializing provider plugins...
- Finding hashicorp/aws versions matching "~> 5.0"...
- Installing hashicorp/aws v5.54.1...
- Installed hashicorp/aws v5.54.1 (signed by HashiCorp)

On a side note: Only using TF_PLUGIN_CACHE_MAY_BREAK_DEPENDENCY_LOCK_FILE=1 will lead to Using hashicorp/aws v5.54.1 from the shared cache directory but of course will lead to a warning:

│ Warning: Incomplete lock file information for providers
│
│ Due to your customized provider installation methods, Terraform was forced to calculate lock file checksums locally for the following providers:
│   - hashicorp/aws

Proposal

No response

References

No response

[apparentlymart]

Thanks for reporting this, @sebastianreloaded.

It is true that the documentation is currently lagging a little behind the implementation. It's describing the older behavior that TF_PLUGIN_CACHE_MAY_BREAK_DEPENDENCY_LOCK_FILE re-enables (at the expense of generating an incomplete dependency lock file), rather than the current behavior.

The current behavior is that Terraform will use plugin packages from the cache directory only if they match a preexisting entry in the dependency lock file.

This means that the first terraform init in an entirely new configuration does not benefit from the cache (so that Terraform can use the origin registry metadata to correctly populate the dependency lock file), but once the dependency lock file has been generated and checked in to version control the cache can then be used for as long as the selected version of each provider doesn't change.

This "true-up" process then repeats again each time a particular configuration switches to a new version of a provider: terraform init -upgrade will, if selecting a new version of an existing provider, contact the origin registry to gather the information required to record the new version selection in the dependency lock file. Once the dependency lock file again agrees with the plugin cache, Terraform will return to using the plugin cache.

[sebastianreloaded]

Thanks for the explanation.

If i understand correctly, i would say this is a collaboration/version control feature. Once the "dependency lock file" is generated and downloaded by another developer/myself, the plugin isn't downloaded again (given he/i already has the plugin in cache) but fetched from the plugin cache. So you don't need the ".terraform"-Folder present, but ".terraform.lock.hcl" is mandatory for this to work.

Makes sense to me, given the ".terraform.lock.hcl" is checked into version control anyways. The docu just confused me.