Open deepbrook opened 2 years ago
Hi @deepbrook! Thanks for sharing this use-case.
Terraform tracks a couple different things that overlap in scope but are not equivalent:
required_providers
blocks and various other backward-compatibility-oriented implicit detection strategies. This tells us a version constraint (not an exact version) associated with each provider source address.
terraform providers
output, but Terraform in practice doesn't use that information itself and only does so to help users debug conflicting version constraints between modules..terraform.lock.hcl
file.Can you confirm which of these are important to the automation you are trying to build, and which are not? That will help us evaluate what makes sense to export as JSON, even if it doesn't necessarily exactly match what terraform providers
does.
One specific reason for this question -- although we do just generally like to understand the full requirements of a problem before trying to solve it -- is that in your proposal it seems that you already discarded the information about which version constraints belong to each module, which I assume means that your automation doesn't actually need that. But communicating that information is the primary reason terraform providers
exists as a separate command, so I have a suspicion that your use-case doesn't exactly match what terraform providers
is designed to do and so some other solution might be a better fit.
Thanks again!
Hey @apparentlymart thanks for getting back to me this fast!
Essentially, I'd like a computer-readable file I can inspect for changes to provider versions. Something like Pip's requirements.txt
, if you will.
I understand this is what the lock.hcl
does, am I right? Then I "simply" need it in a machine-friendly format (e.g. JSON)
Incidentally, I tried to run terraform provider lock -json
first - only to find out this is just for updating the lock file, not creating it (afait).
As for your question regarding what it is we're building: we keep an extensive cache of plugins, providers and modules.
We've solved caching modules by simply caching the backend (.terraform
dir), and while our caching for providers works somewhat (we're tracking each and all version.tf
files, in the repository and its checksum), it's recently got annoying, because we also have extensive documentation right in the tf files, as well as annotations in comments (including version.tf
) - changes to these then trigger the cache to be partially rebuild.
I hope that made sense. Thanks again, I appreciate the effort.
We're currently doing shell acrobatics to programmatically parse provider versions needed by our terraform configurations.
In the current version of terraform,
terraform providers
renders a neat human-readable output:..with an additional warning, in case we use experiments:
Extracting the provider versions programmatically from this is at the very least cumbersome.
Current Terraform Version
Use-cases
We cache plugins for our configurations, and would like to correctly deduce whether or not we need to refresh the cache. Using merely
version.tf
's checksum is too vague, as we also keep documentation in this and other TF files, which has quite a bit of churn.Attempted Solutions
We've used the usual shell utils to parse the output of the CLI so far. We've also used them to parse lock and
version.tf
files. The problem remains the same: the format isn't automation friendly.Proposal
Add a
-json
option toterraform provider
, which generates the output as a JSON-encoded string:In case of sub-modules with specific provider requirements: ideally, terraform merges all required providers and tells me exactly which provider versions work with the modules.