`provider_meta` supports provider alias

magodo commented 1 year ago

Terraform Version

v1.3.5

Use Cases

provider_meta is used to set some provider scoped metadata, which can modify the provider behaviors (e.g. the PlanModifier has access to that metadata). Since a same typed provider can have multiple instances via alias, that makes sense that the provider_meta can distinguish them.

Attempted Solutions

I've tried to define an alias to my provider (named foo), and use it as below:

terraform {
  required_providers {
    restful = {
      source = "magodo/restful"
    }
  }
  provider_meta "restful.foo" {
    #...
  }
}

Terraform panics:

!!!!!!!!!!!!!!!!!!!!!!!!!!! TERRAFORM CRASH !!!!!!!!!!!!!!!!!!!!!!!!!!!!

Terraform crashed! This is always indicative of a bug within Terraform.
Please report the crash with Terraform[1] so that we can fix this.

When reporting bugs, please include your terraform version, the stack trace
shown below, and any additional information which may help replicate the issue.

[1]: https://github.com/hashicorp/terraform/issues

!!!!!!!!!!!!!!!!!!!!!!!!!!! TERRAFORM CRASH !!!!!!!!!!!!!!!!!!!!!!!!!!!!

dots are not allowed
goroutine 1 [running]:
runtime/debug.Stack()
        /usr/local/go/src/runtime/debug/stack.go:24 +0x65
runtime/debug.PrintStack()
        /usr/local/go/src/runtime/debug/stack.go:16 +0x19
github.com/hashicorp/terraform/internal/logging.PanicHandler()
        /home/circleci/project/project/internal/logging/panic.go:55 +0x153
panic({0x221a920, 0xc000b899e0})
        /usr/local/go/src/runtime/panic.go:844 +0x258
github.com/hashicorp/terraform/internal/addrs.MustParseProviderPart({0xc000430240?, 0x0?})
        /home/circleci/project/project/internal/addrs/provider.go:449 +0x4c
github.com/hashicorp/terraform/internal/addrs.NewDefaultProvider(...)
        /home/circleci/project/project/internal/addrs/provider.go:122
github.com/hashicorp/terraform/internal/addrs.ImpliedProviderForUnqualifiedType({0xc000430240?, 0x47b19e?})
        /home/circleci/project/project/internal/addrs/provider.go:114 +0x11b
github.com/hashicorp/terraform/internal/configs.(*Module).ImpliedProviderForUnqualifiedType(...)
        /home/circleci/project/project/internal/configs/module.go:590
github.com/hashicorp/terraform/internal/configs.(*Module).ProviderForLocalConfig(0x22f1480?, {{0xc000430240?, 0xc000430bd0?}, {0x0?, 0x2?}})
        /home/circleci/project/project/internal/configs/module.go:576 +0xbd
github.com/hashicorp/terraform/internal/configs.(*Module).appendFile(0xc000bc9340, 0xc0006bc8c0)
        /home/circleci/project/project/internal/configs/module.go:242 +0xde8
github.com/hashicorp/terraform/internal/configs.NewModule({0xc0006a4130, 0x1, 0x1?}, {0x0, 0x0, 0x7fae64fe3328?})
        /home/circleci/project/project/internal/configs/module.go:142 +0x407
github.com/hashicorp/terraform/internal/configs.(*Parser).LoadConfigDir(0xc000603520?, {0x26b3c8e, 0x1})
        /home/circleci/project/project/internal/configs/parser_config_dir.go:42 +0x2a5
github.com/hashicorp/terraform/internal/command.(*Meta).loadSingleModule(0x17762fa7000002be?, {0x26b3c8e?, 0xc000814481?})
        /home/circleci/project/project/internal/command/meta_config.go:70 +0x9b
github.com/hashicorp/terraform/internal/command.(*Meta).loadBackendConfig(0x0?, {0x26b3c8e?, 0x6?})
        /home/circleci/project/project/internal/command/meta_config.go:132 +0x25
github.com/hashicorp/terraform/internal/command.(*PlanCommand).PrepareBackend(0xc000603520, 0x1?)
        /home/circleci/project/project/internal/command/plan.go:120 +0xde
github.com/hashicorp/terraform/internal/command.(*PlanCommand).Run(0xc000603520, {0xc00004c260?, 0x4?, 0x4?})
        /home/circleci/project/project/internal/command/plan.go:67 +0x285
github.com/mitchellh/cli.(*CLI).Run(0xc0006bc3c0)
        /home/circleci/go/pkg/mod/github.com/mitchellh/cli@v1.1.4/cli.go:262 +0x5f8
main.realMain()
        /home/circleci/project/project/main.go:312 +0x15b4
main.main()
        /home/circleci/project/project/main.go:58 +0x19

Proposal

No response

References

https://github.com/magodo/terraform-provider-restful/pull/22

apparentlymart commented 1 year ago

Hi @magodo! Thanks for reporting this.

The intention for provider_meta is a very narrow use-case: whenever a module is written by the same organization as the provider it uses, it allows the author of both to use the provider to gather usage metrics about their modules by including metadata in requests made to the provider.

This was implemented to meet a very specific partner request and is not a feature we plan to expand to other use-cases. I believe the current behavior is that all configurations for the named provider should be associated with that metadata, because the content of the provider_meta block is supposed to be describing characteristics of the module it's embedded into rather than settings for the provider. Settings that control the provider itself belong in a provider block, not in a provider_meta block.

However, it is definitely not intended that it should crash when you specify an invalid provider local name in the label, so there is a configuration decoding bug to be fixed here: the configuration decoder should notice that the label has invalid syntax and return an explicit error about that, rather than crashing.

magodo commented 1 year ago

@apparentlymart I understand my use case is not the expected scenario of the provider_meta. Whilst, I think it is a potential valid case that sometimes we want a dynamic default value for a certain resource. Specifically, we have a same named properties (e.g. the create_method) at both the provider and the resource level, where the provider level one is the fallback value, which is used by the resouce when that property is not set at the resource level. Currently the implementation is tricky and brings unnecessary noise.

We've found that the provider_meta is an ideal way to be used to define dynamic default value, as we can access that during plan (as is defined in the protocol). So what I really want to ask is can we have some mechanism similar to provider_meta, but can work for provider aliases, so taht we can define dynamic default values for resources.

apparentlymart commented 1 year ago

Hi @magodo,

The existing concept of provider configurations is the intended way to meet the use-case you described. You can describe the same arguments inside the provider's own configuration schema to allow configuring them on a per-provider-configuration basis:

provider "restful" {
  header = {
    foo = "bar"
  }
  resource = {
    create_method = "PUT"
  }
}

I see that your provider is using the new Terraform Plugin Framework. I believe the intended way to use the above in the Framework is for your provider's Configure function to save these settings in fields of your provider type. You can then refer to those saved settings when handling the plan request.

provider_meta is for describing properties of the module that it's written in rather than describing properties of the provider it's related to. If we made a version of that feature which was associated with provider configurations rather than with modules then it would be functionally equivalent to provider configurations, and so it would be a redundant protocol feature.

If you find it inconvenient to use the provider configuration settings from inside the plan implementation in your provider then I think that would be good feedback to share in the plugin framework repository; the team which develops the framework could change the design to make what you are trying to do more convenient without any changes to the Terraform language or to the plugin protocol.

hashicorp / terraform