"terraform providers mirror" skip downloading packages that are already present in the mirror directory

[maonat]

Terraform Version

v1.4.6

Use Cases

I would like to handle all the providers in a cached path on my or the remote machine which would allow me to:

• Save space on the disk as I would have all the providers cached in a specific path
• Not break multiple terraform commands execution at the same time when a provider is being downloaded
• Keep updated the cached providers once every a while without the need to worry about it when running the CI

Attempted Solutions

I was not able to achieve this. For now the only solution would be to enter the modules I have in my repository and run terraform init for each terraform block found with a different required_providers block,

Proposal

I think that there are two possibilities: Add a simple option named -download-providers-only which would simply download the providers if not already present in the providers directory without installing them in the .tf files directory Create new command plugin-download to simply do the above

References

No response

[apparentlymart]

Hi @maonat! Thanks for sharing this use-case.

Have you seen the terraform providers mirror command? It seems like it's related to what you are asking about, though perhaps not exactly the same in the details.

[maonat]

Hi @apparentlymart, I've tried using the terraform providers mirror and is working ALMOST as I need: It is indeed doing just the download but it does not skip the download if the file already binary already exists in the directory. Is this expected or should say that it's a possible feature request for this command?

[apparentlymart]

Hi @maonat,

Having it skip re-downloading if it can detect that the mirror packages already match the checksums reported by the registry does seem like a very reasonable feature request. Shall we transform this issue into that? :grinning:

[maonat]

@apparentlymart that would be great! How do I need to proceed? Close this issue and open a new one or proceed and add an edit: below the original discussion? PS: Do you think it would be possible to skip the checksums check?

[apparentlymart]

For now I've just changed the summary to reflect what we discussed and we'll let the discussion above document how we got here. I think that'll be sufficient to give us something to use for prioritization and gathering related use-cases.

This command's purpose is to synchronize a mirror with the content it is mirroring and so the checksum part seems important to allow the tool to repair a mirror that has become corrupted somehow, so that there isn't a broken or maliciously modified package present indefinitely.

However, if you can say more about why you'd want to disable checksum verification then of course we could consider that while designing this new behavior.

Thanks!

[maonat]

@apparentlymart I understand the issue on the checksums. Let's skip this for now 👍

[crw]

Thanks @maonat! Since this is a valid feature request, we will leave it open to gather support and use cases. We appreciate your feedback!

[nathaclmpaulino]

I think I also have an use case as well.

On the company that I work for, we use Terraform (1.7.0) with Terragrunt (0.54.17) and we share the same short providers file (most related with AWS and Kubernetes) with all our terraform code.

Currently, we have 28 folders that we should interact at the same time with commands like terragrunt run-all plan and in every single folder we should init and download those providers, so to avoid spent extra time on terragrunt run-all init we created a centralized plugins cache and when it's required, we pull the plugins data from the cache.

This update will also improve the time spent downloading the same plugin twice and we also increase the number of folders that we interact at the same time.