search

hashicorp / terraform-provider-azurerm

Terraform provider for Azure Resource Manager
https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs
Mozilla Public License 2.0
4.51k stars 4.6k forks source link

When destroying azurerm_spring_cloud_api_portal -- Error: Please unassign public endpoint before deleting API Portal. #19949

Open luisfeliz79 opened 1 year ago

luisfeliz79 commented 1 year ago

Is there an existing issue for this?

Community Note

Terraform Version

1.3.7

AzureRM Provider Version

3.38.0

Affected Resource(s)/Data Source(s)

azurerm_spring_cloud_api_portal

Terraform Configuration Files

terraform {
  required_providers {
    azurerm = {
      source = "hashicorp/azurerm"
      version = "= 3.38.0"
    }

  }

}

provider "azurerm" {
    features {
    } 
}

# Create the SpringApps Resource group 
resource "azurerm_resource_group" "springapps_rg" {
    name                        = "testapiportal"
    location                    = "eastus"
}

resource "azurerm_spring_cloud_service" "sc_enterprise" {

  name                = "testspringappsxyz"
  resource_group_name = azurerm_resource_group.springapps_rg.name
  location            = "eastus"

  sku_name = "E0"

  # Tanzu service registry - Set to true if Enterprise Tier
  service_registry_enabled = true
  build_agent_pool_size    = "S1"

  timeouts {
      create = "60m"
      delete = "2h"
  }

}

resource "azurerm_spring_cloud_api_portal" "apiportal" {
  name                          = "default"
  spring_cloud_service_id       = azurerm_spring_cloud_service.sc_enterprise.id
  https_only_enabled            = false
  public_network_access_enabled = true
  instance_count                = 1
}

Debug Output/Panic Output

https://learn.microsoft.com/en-us/azure/active-directory/fundamentals/certificate-authorities

azurerm_spring_cloud_api_portal.apiportal: Destroying... [id=/subscriptions/xxxxx/resourceGroups/testapiportal/providers/Microsoft.AppPlatform/spring/testspringappsxyz/apiPortals/default]
2023-01-10T11:26:46.979-0500 [INFO]  Starting apply for azurerm_spring_cloud_api_portal.apiportal
2023-01-10T11:26:46.979-0500 [DEBUG] azurerm_spring_cloud_api_portal.apiportal: applying the planned Delete change
2023-01-10T11:26:46.980-0500 [DEBUG] provider.terraform-provider-azurerm_v3.38.0_x5.exe: AzureRM Request:
DELETE /subscriptions/xxxx/resourceGroups/testapiportal/providers/Microsoft.AppPlatform/Spring/testspringappsxyz/apiPortals/default?api-version=2022-11-01-preview HTTP/1.1
Host: management.azure.com
User-Agent: Go/go1.19.3 (amd64-windows) go-autorest/v14.2.1 Azure-SDK-For-Go/v66.0.0 appplatform/2022-11-01-preview HashiCorp Terraform/1.3.7 (+https://www.terraform.io) Terraform Plugin SDK/2.10.1 terraform-provider-azurerm/dev pid-222c6c49-1b0a-5959-a213-6608f9eb8820
X-Ms-Correlation-Request-Id: c7bbaec9-8f7f-9eec-7b67-0fb9a8aed85c
Accept-Encoding: gzip: timestamp=2023-01-10T11:26:46.980-0500
2023-01-10T11:26:47.759-0500 [DEBUG] provider.terraform-provider-azurerm_v3.38.0_x5.exe: AzureRM Response for https://management.azure.com/subscriptions/xxxxx/resourceGroups/testapiportal/providers/Microsoft.AppPlatform/Spring/testspringappsxyz/apiPortals/default?api-version=2022-11-01-preview:
HTTP/2.0 400 Bad Request
Content-Length: 132
Cache-Control: no-cache
Content-Type: application/json; charset=utf-8
Date: Tue, 10 Jan 2023 16:26:45 GMT
Expires: -1
Pragma: no-cache
Request-Context: appId=cid-v1:797d7e4e-8180-497e-a254-780fbd39ba4d
Strict-Transport-Security: max-age=31536000; includeSubDomains
X-Content-Type-Options: nosniff
X-Ms-Client-Request-Id: f7e2e91e-3762-4fdc-844d-5ae32285ecb5
X-Ms-Correlation-Request-Id: c7bbaec9-8f7f-9eec-7b67-0fb9a8aed85c
X-Ms-Ratelimit-Remaining-Subscription-Deletes: 14999
X-Ms-Request-Id: c4300e3b-69b4-4846-ad36-61936f090a28
X-Ms-Routing-Request-Id: CANADACENTRAL:20230110T162646Z:c4300e3b-69b4-4846-ad36-61936f090a28
X-Rp-Server-Mvid: 602e8535-9334-4b2f-9945-fcf7db9ba99e

{"error":{"code":"BadRequest","message":"Please unassign public endpoint before deleting API Portal.","target":null,"details":null}}: timestamp=2023-01-10T11:26:47.759-0500
2023-01-10T11:26:47.760-0500 [ERROR] provider.terraform-provider-azurerm_v3.38.0_x5.exe: Response contains error diagnostic: tf_proto_version=5.3 tf_req_id=e13c9731-001b-c78e-8431-67159b33f723 tf_resource_type=azurerm_spring_cloud_api_portal @caller=github.com/hashicorp/terraform-plugin-go@v0.14.1/tfprotov5/internal/diag/diagnostics.go:55 diagnostic_severity=ERROR diagnostic_detail= diagnostic_summary="deleting Spring Cloud A P I Portal: (Api Portal Name "default" / Spring Name "testspringappsxyz" / Resource Group "testapiportal"): appplatform.APIPortalsClient#Delete: Failure sending request: StatusCode=400 -- Original Error: Code="BadRequest" Message="Please unassign public endpoint before deleting API Portal."" tf_provider_addr=provider tf_rpc=ApplyResourceChange @module=sdk.proto timestamp=2023-01-10T11:26:47.759-0500
2023-01-10T11:26:47.762-0500 [ERROR] vertex "azurerm_spring_cloud_api_portal.apiportal (destroy)" error: deleting Spring Cloud A P I Portal: (Api Portal Name "default" / Spring Name "testspringappsxyz" / Resource Group "testapiportal"): appplatform.APIPortalsClient#Delete: Failure sending request: StatusCode=400 -- Original Error: Code="BadRequest" Message="Please unassign public endpoint before deleting API Portal."
╷
│ Error: deleting Spring Cloud A P I Portal: (Api Portal Name "default" / Spring Name "testspringappsxyz" / Resource Group "testapiportal"): appplatform.APIPortalsClient#Delete: Failure sending request: StatusCode=400 -- Original Error: Code="BadRequest" Message="Please unassign public endpoint before deleting API Portal."

Expected Behaviour

I expect the apiportal resource to be destroyed successfully

Actual Behaviour

The destruction of the plan is interrupted with the error "Please unassign public endpoint before deleting API Portal."

Steps to Reproduce

Note: You may need to prepare your subscription first, see here:

https://github.com/Azure/azure-spring-apps-reference-architecture/tree/main/terraform#:~:text=az%20provider%20register

terraform init terraform plan -out my.plan terraform apply my.plan

terraform plan -destroy -out my.plan terraform apply my.plan

Note: same behavior when using terraform destroy -auto-approve

Important Factoids

No response

References

No response

ms-henglu commented 1 year ago

Hi @luisfeliz79 ,

Thank you for taking time to report this issue!

Would you please try to set public_network_access_enabled = false then apply it before delete the resource?

luisfeliz79 commented 1 year ago

HI @ms-henglu , this works. If I turn off the public endpoint via the portal or by changing the terraform plan+applying, followed by a destroy, then the destroy operation completes successfully.

luisfeliz79 commented 1 year ago

Hi @ms-henglu, checking in on the status for this thread?

This is still an issue as doing a destroy operation always fails unless the public network access is disabled as manual step.

Ideally, the module should disable the endpoint before attempting destruction of azurerm_spring_cloud_api_portal.

taoxu0903 commented 1 year ago

@luisfeliz79 my name is Ken, PM from Azure Spring Apps. Let me try to clarify this further here. in general, disabling/deleting a managed Tanzu component like API portal here is considered as a serious action which will destroy Tanzu components away from your enterprise tier instances. So to avoid to disable them by mistake on customer side, we put some prerequisites for each kind of Tanzu component to help you/customers take this action carefully. Like for API portal here, to execute a deletion of API portal can be pretty safe if no public endpoint assigned by you or customers since there surely will be no traffic to it. Let me know if you still have comments on it.

katbyte commented 1 year ago

Hi @taoxu0903 - the way terraform typically works, and our users expect it to work is that it "declares intended infra state". So when the user adds/modifies/removes a resource from their terraform config they expect terraform to do everything required to get to that state. As such there should be no manual steps required for deletion, and if there is users will unlikely know about it making for a poor ux. If the resource is removed from config then it should be deleted and removed.

as it stands users will remove it, try to apply and it will fail to delete, know they have to restore the config and change this flag, apply, and then remove it from config, and apply again.

I agree with @luisfeliz79 that this is a bug that should be fixed by having the provider update this property before deletion.

luisfeliz79 commented 4 months ago

Hi @taoxu0903, Ken, completely understand your logic, however, in this case the public endpoint is being enabled automatically, not through a post deployment action. Perhaps a good compromise would be to surface a feature or provider flag that can allow the deletion to complete even if the public endpoint is enabled. This is way is safe by default, but those who need the automation to work, have an option. Thoughts?

taoxu0903 commented 4 months ago

about "the public endpoint is being enabled automatically", this is not true actually. for API portal, to expose either the default public endpoint owned by ASA, or binding a custom domain, you need manually do it. By default after enabling API portal only, no endpoint would be exposed. Please double check if you see same behavior on your side.

luisfeliz79 commented 4 months ago

Got it.... So then to fix this, would the action item be this?

If "public_network_access_enabled" = true as part of an "azurerm_spring_cloud_api_portal" resource then during a destroy operation for "azurerm_spring_cloud_api_portal" the code should automatically disable the public endpoint first, then attempt deletion of the API portal.

This would successfully allow terraform destroy operations to complete successfully and no user interaction or workarounds are needed.

tombuildsstuff commented 4 months ago

@taoxu0903 FWIW other APIs within Azure will handle this as part of the Delete process, so I'd consider this an API bug since this is configured via a property within the Spring Cloud API Portal resource, rather than something the Provider should need to do?

From the Providers side, users have confirmed they want to destroy the resource via the Terraform Plan/Apply workflow, so from our side this should be part of the regular deletion flow - but (as mentioned above) I'd consider this an API bug that needs to be fixed there, rather than worked around in different clients.

taoxu0903 commented 4 months ago

Got your points, guys. I just synced internally with our engineer team and generally we feel it makes sense to add a force deletion flag to allow to delete without removing pre-conditions. I will add it into backlog for overall planning for next sprints.