Change provider identity semantics to include signature

[stevehipwell]

OpenTofu Version

ALL

Use Cases

Currently a provider is referenced by it's source and version constraint (if it's coming from a registry). I would like to see this augmented with an optional signature constraint so the identity can be better managed and validated during a terraform init -upgrade operation. This makes particular sense with the move away from defaulting to a central registry.

If modules are able to be locked (see #586) then the same logic should be applied to them too.

Attempted Solutions

n/a

Proposal

This would likely require an additional layer over the Terraform defaults such as supporting .tofu files and a tofu block to add the constraints. As this would only be relevant when installing the providers it would be fully backwards compatible with Terraform.

versions.tf

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.6"
    }
  }
}

versions.tofu

tofu {
  required_providers {
    aws = {
      signatures = [...]
    }
  }
}

References

• 307

[cube2222]

Hey @stevehipwell!

I wonder, if you're already writing the signature verbatim, would it make sense for this to instead just be the checksum?

[stevehipwell]

@cube2222 signature might have been a bad term in the example, I think identity might be a better fit. I'm specifically thinking about linking back to a key or a Cosign keyless identity to validate the provider before the hash is taken and added to the lock file.

For example if Cosign is being used an identity might be https://github.com/my-org/opentf-provider-example/.github/workflows/release.yaml@refs/tags/v* (note the *).

[cube2222]

Ah, makes sense!

I wonder if in that case it would make sense to store it in the lockfile? Right now it only stores checksums, which change with each provider update. But maybe it would make sense to also store an identity for each provider there, which should generally stay immutable?

You could of course hand edit it, but by default it would be stored the first time you add the provider.

[stevehipwell]

@cube2222 the identity would be a constraint not a result so it would need to be outside the lock file which can be removed and re-generated. The lock file doesn't need to capture the identity as it has the hash which would have been validated, plus modifying the lock file might break compatibility.

[Hefeweizen]

I want to link to this issue: https://github.com/opentofu/opentofu/issues/428.

In the discussion above, it was clarified that the title's use of signature might be better referenced as identity; and that changes the need. The linked issue is specifically looking for gpg signature verification which is not what this is asking for. This issue is asking on providers; the link is asking for modules. However, behind both, I see a request for "ways to authenticate resources". Can a general mechanism provide for both needs?

[apparentlymart]

I'm not really familiar enough with Cosign to fully follow exactly what it means to specify a "signature" or "identity" as described here, but I do want to note that the provider source address (as specified in source) is primarily a unique identifier and only secondarily a default location to fetch packages from.

This unique identifier is used both to understand whether two modules in the same configuration are trying to use the same provider -- in which case they can potentially share a single configuration for that provider -- and to keep track of which provider is responsible for each resource instance between plan/apply rounds in state snapshots.

For example, configuring a filesystem mirror in the Provider Installation settings means that hashicorp/aws is now purely an identifier for talking about that provider, and so the implied reference to registry.opentofu.org is now purely DNS-as-a-namespace, similar to the reverse-DNS naming scheme conventionally used for Java packages, and not describing a network service to contact. OpenTofu would still use registry.opentofu.org/hashicorp/aws as the identifier for that provider even though it was "installed" just by creating a symlink to a directory already on the local filesystem.

Unless we want to make significant changes to how OpenTofu's tracks the identity of providers I think we'll need to retain the source argument in its "unique identifier" role, even if we're introducing a new way to find the plugin package to install for that provider.

It isn't clear to me whether version is also still required, or if the new argument effectively subsumes the "version constraint" idea. But thankfully the version constraints are relevant only to the plugin installer, so the consequences of changing the modelling of those are probably far less significant. I think it would only require deciding what makes sense as the value for "remembering" the single selected version in the dependency lock file.