Existing registered resource providers clash with automatic provider registration

jakubslonxlab commented 7 months ago

Is there an existing issue for this?

[X] I have searched the existing issues

Community Note

Please vote on this issue by adding a :thumbsup: reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment and review the contribution guide to help.

Terraform Version

1.5.7

AzureRM Provider Version

3.9.0

Affected Resource(s)/Data Source(s)

Microsoft.AppConfiguration

Terraform Configuration Files

azurerm = {
      source  = "hashicorp/azurerm"
      version = "3.90.0"
}

Debug Output/Panic Output

Error: The Resource Provider "Microsoft.AppConfiguration" is automatically registered by Terraform. 
To manage this Resource Provider Registration with Terraform you need to opt-out 
of Automatic Resource Provider Registration (by setting 'skip_provider_registration' 
to 'true' in the Provider block) to avoid conflicting with Terraform. 
  with azurerm_resource_provider_registration.provider["Microsoft.AppConfiguration"], 
  on resource_providers.tf line 1, in resource "azurerm_resource_provider_registration" "provider": 
   1: resource "azurerm_resource_provider_registration" "provider" { 
The Resource Provider "Microsoft.AppConfiguration" is automatically 
registered by Terraform. 
To manage this Resource Provider Registration with Terraform you need to 
opt-out 
of Automatic Resource Provider Registration (by setting 
'skip_provider_registration' 
to 'true' in the Provider block) to avoid conflicting with Terraform.

Expected Behaviour

Terraform should acknowledge already registered resource providers.

Actual Behaviour

Terraform does not handle the registered providers, just breaks with a notice to skip the provider registration.

This is a breaking change in the scenario where providers have been manually registered and have just been added to automatic provider registration.

Steps to Reproduce

Register a provider in via Azure Portal.
Upgrade azurerm provider to 3.90.0
Run terraform

Important Factoids

No response

References

https://github.com/hashicorp/terraform-provider-azurerm/pull/24645

katbyte commented 7 months ago

@jakubslonxlab,

Apologies for breaking your configuration in 3.90 - I want to confirm where that error is coming from. By manually registered do you mean you have used the azurerm_resource_provider_registration resource to register it via terraform and that resource is now failing? or that you manually registered it via the portal/cli/elsewhere and the provider is throwing that error during the automatic registration?

MasterJuan commented 7 months ago

I have the same issue I suppose: Error: The Resource Provider "Microsoft.AppConfiguration" is automatically registered by Terraform. To manage this Resource Provider Registration with Terraform you need to opt-out of Automatic Resource Provider Registration (by setting 'skip_provider_registration' to 'true' in the Provider block) to avoid conflicting with Terraform. with module.portal_environment.module.app_configuration.azurerm_resource_provider_registration.app_configuration_registration on ../../../../modules/app-config/main.tf line 39, in resource "azurerm_resource_provider_registration" "app_configuration_registration": resource "azurerm_resource_provider_registration" "app_configuration_registration" { The Resource Provider "Microsoft.AppConfiguration" is automatically registered by Terraform.

To manage this Resource Provider Registration with Terraform you need to opt-out of Automatic Resource Provider Registration (by setting 'skip_provider_registration' to 'true' in the Provider block) to avoid conflicting with Terraform.

I tried the following, without luck:

provider "azurerm" {
  client_id       = my_client_id
  client_secret   = my_client_secret
  subscription_id = var.subscription
  features {
    resource_provider_registration {
      skip_provider_registration = true
    }
  }
}

I tried to use explicitly the v.3.89, again without luck:

terraform {
  required_version = "~>1.5"

  required_providers {
    azurerm = {
      source  = "hashicorp/azurerm"
      version = "3.89"
    }
  }
}

jakubslonxlab commented 7 months ago

@jakubslonxlab,

Apologies for breaking your configuration in 3.90 - I want to confirm where that error is coming from. By manually registered do you mean you have used the azurerm_resource_provider_registration resource to register it via terraform and that resource is now failing? or that you manually registered it via the portal/cli/elsewhere and the provider is throwing that error during the automatic registration?

Hello,

We registered it via the portal, as there was no automatic registration for the said resource in place ("Microsoft.AppConfiguration"). On 3.89 it works okay, the 3.90 is breaking a lot of builds because of this.

@MasterJuan rolling back to 3.89 has worked for us.

katbyte commented 7 months ago

@MasterJuan - the complete error you have resource "azurerm_resource_provider_registration" "app_configuration_registration" { seems to indicate that you need to remove the resource at modules/app-config/main.tf line 39 as it is trying to register the RP which is now automatically done so by the provider again.

@jakubslonxlab - are you certain there is no azurerm_resource_provider_registration resource in your code? could you please share the entire error like @MasterJuan did? which should indicate where the error came from

jakubslonxlab commented 7 months ago

@jakubslonxlab - are you certain there is no azurerm_resource_provider_registration resource in your code? could you please share the entire error like @MasterJuan did? which should indicate where the error came from

@katbyte We have azurerm_resource_provider_registration in the code, but we would like to use the automatic registration functionality going forward - we would prefer not to skip the automatic provider registration.

resource "azurerm_resource_provider_registration" "provider" {
  for_each = toset(local.resource_providers)
  name     = each.value
}

locals {
  resource_providers = [
    "Microsoft.SaaS",
    "Microsoft.Dashboard",
    "Microsoft.AppConfiguration"
  ]}

We are uncertain on how to go forward with this, due to the following note (can be found here) from Microsoft on their kb:

You can't unregister a resource provider when you still have resource types from that resource provider in your subscription.

Can you advise on how to make this change without destroying the resources in subscriptions where providers have already been registered?

jakubslonxlab commented 7 months ago

We attempted to remove azurerm_resource_provider_registration for the "Microsoft.AppConfiguration" provider, and we get the same error message:

Error: The Resource Provider "Microsoft.AppConfiguration" is automatically registered by Terraform. 
To manage this Resource Provider Registration with Terraform you need to opt-out 
of Automatic Resource Provider Registration (by setting 'skip_provider_registration' 
to 'true' in the Provider block) to avoid conflicting with Terraform. 
  with azurerm_resource_provider_registration.provider["Microsoft.AppConfiguration"], 
  on resource_providers.tf line 1, in resource "azurerm_resource_provider_registration" "provider": 
   1: resource "azurerm_resource_provider_registration" "provider" { 
The Resource Provider "Microsoft.AppConfiguration" is automatically 
registered by Terraform. 
To manage this Resource Provider Registration with Terraform you need to 
opt-out 
of Automatic Resource Provider Registration (by setting 
'skip_provider_registration' 
to 'true' in the Provider block) to avoid conflicting with Terraform.

katbyte commented 7 months ago

Automatic registration is done unless you tell the provider to skip registration.

the error you are seeing is because in the file resource_providers.tf on line 1 you have a resource azurerm_resource_provider_registration" "provider which is trying to register Microsoft.AppConfiguration - remove that and you should be good to go

tombuildsstuff commented 7 months ago

@jakubslonxlab

Can you advise on how to make this change without destroying the resources in subscriptions where providers have already been registered?

In this instance you should be able to remove the existing registration resource from your Terraform Statefile, so that the Resource Provider remains registered - but is no longer explicitly managing it in Terraform. You can do this via:

terraform state rm azurerm_resource_provider_registration.provider

Once that's done you should be able to able to remove this from your Terraform Configuration.

jakubslonxlab commented 7 months ago

@tombuildsstuff thank you for your helpful comment - we found a way to apply this to multiple states that we manage 👍

I think it would be a good idea to potentially improve the error message on this, as currently it is not clear how to manage the scenario that we experienced with the update to 3.90. Adding skip_provider_registration is not always suitable, but is displayed as a potential fix by terraform.

paul-hugill commented 4 months ago

Similar issue upgrading from 3.61.0 to 3.99.0, we have a for_each on a list of providers that were not previously included in the automatic registration so that they are registered (only Microsoft.DataFactory and Microsoft.Synapse currently).

Ideally I would just need to do the state remove but on 200+ dynamic API Driven workspaces, that is not going to be fun. And the removed block option in TF 1.7, doesn't support removing instances of resources.

The only workaround I can see would be to use skip_provider_registration and specify the providers we want to use but of course this adds more issues down the line related to having to keep that list updated. We would have to do the import for each of them.

3.89.0 doesn't cause me issues for Microsoft.DataFactory but 3.99.0 does, so for now we might be stuck on that until a better solution is available.

hashicorp / terraform-provider-azurerm