Adding an API Management API policy to an existing API Management API fails with message that the API already exists

gercograndia commented 9 months ago

Is there an existing issue for this?

[X] I have searched the existing issues

Community Note

Please vote on this issue by adding a :thumbsup: reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment and review the contribution guide to help.

Terraform Version

v1.5.4

AzureRM Provider Version

v3.80.0

Affected Resource(s)/Data Source(s)

azurerm_api_management_api_policy

Terraform Configuration Files

data "azurerm_api_management_api" "example" {
  name                = "<api_name>"
  api_management_name = "<api_mngt_name>"
  resource_group_name = "<rg_name>"
  revision            = "<rev_id>"
}

resource "azurerm_api_management_api_policy" "example" {
  api_name            = data.azurerm_api_management_api.example.name
  api_management_name = data.azurerm_api_management_api.example.api_management_name
  resource_group_name = data.azurerm_api_management_api.example.resource_group_name

  xml_content = <<XML
<policies>
  <inbound>
    <find-and-replace from="xyz" to="abc" />
  </inbound>
</policies>
XML
}

Debug Output/Panic Output

module.api_backend[0].azurerm_api_management_api_policy.example: Creating...
╷
│ Error: A resource with the ID "/subscriptions/658e58dd-8725-4413-b1ae-115e6b54d9a9/resourceGroups/rg-smz-euw-run-api-management/providers/Microsoft.ApiManagement/service/apimsmzeuwrunapimanagement/apis/dev_dbx_reg_api" already exists - to be managed via Terraform this resource needs to be imported into the State. Please see the resource documentation for "azurerm_api_management_api_policy" for more information.
│
│   with module.api_backend[0].azurerm_api_management_api_policy.example,
│   on ../../core/api-management/modules/api/main.tf line 74, in resource "azurerm_api_management_api_policy" "example":
│   74: resource "azurerm_api_management_api_policy" "example" {
│
╵
ERRO[0043] Terraform invocation failed in /Users/gerco/git/eon/daai/data-platform/landing-zones/next-gen/terragrunt/smz/run/apps/.terragrunt-cache/Wvn--6PuxeX3JkhpVYFHXLobvLE/B61D95J0T3rzXIlP7yHoEtOZtEc/composite/app  prefix=[/Users/gerco/git/eon/daai/data-platform/landing-zones/next-gen/terragrunt/smz/run/apps]
ERRO[0043] 1 error occurred:
    * [/Users/gerco/git/eon/daai/data-platform/landing-zones/next-gen/terragrunt/smz/run/apps/.terragrunt-cache/Wvn--6PuxeX3JkhpVYFHXLobvLE/B61D95J0T3rzXIlP7yHoEtOZtEc/composite/app] exit status 1

Expected Behaviour

it should create the policy

Actual Behaviour

it raises an error that the API already exists, which is logical as I look it up via a data source.

Steps to Reproduce

just an apply raises the issue

Important Factoids

No response

References

No response

sinbai commented 9 months ago

Hi @gercograndia thanks for opening this issue. Based on the information provided, it appears that the policy resource for the specified API already exists. In this case, if a new policy needs to be appended, please run terraform import command to import the policy first, for example: terraform import azurerm_api_management_api_policy.example /subscriptions/658e58dd-8725-4413-b1ae-115e6b54d9a9/resourceGroups/rg-smz-euw-run-api-management/providers/Microsoft.ApiManagement/service/apimsmzeuwrunapimanagement/apis/dev_dbx_reg_api . Then update the xml_content to append the new policy(assuming ip-filter is the one that exists), like:

  xml_content = <<XML
<policies>
  <inbound>
   <ip-filter action="allow" />
    <find-and-replace from="xyz" to="abc" />
  </inbound>
</policies>
XML

gercograndia commented 9 months ago

Hi,

Thanks for your (quick) answer.

You definitely helped me a bit further, but I still feel this is not working properly.

Let me start the things that helped:

Indeed I could import the existing api into the declared api_policy
After that it came back to me with a proposed update which I indeed could apply.

However, then the things that are still breaking things, or are debatable:

When you create an API, there seems to be an base api being provisioned automatically, which is just some kind of boiler plate code. Which means that you always have to (manually) import, or delete this before it actually works.
The UI of APIM shows that you can add a policy, but apparently you are not adding a new policy, just adding something to an existing (the base) policy.

So my way of working is now that I have to delete the base policy, after the API is created, and then I can add it. I hope I am missing something though :-)

Again thanks for your support, appreciated!

OctavioCore commented 8 months ago

Also encountered this issue, and just as well, I do believe the behaviour of this resource is quite misleading. Based on the docs, I would expect either of those two scenarios:

The resource represents just the expected new contents of the base policy - the xmlcontent is "pasted" into the base policy. This is clearly not the case.
The resource represents an entire product policy, separate from the base one, created independently of it. Not the case, either.

In my scenario, a condensed module provides capabilities to provision an APIM instance, and allows the developers to configure additional elements such as product policies, backends, api policies, et cetera when they see fit. Importing the base policy to make changes to its contents would be a dirty manual action. They didn't create it, have no idea what is in there, how to pick it for import, and don't care about its fate; All they do care about, is the content of the new policy, which they want to be binding.

IMHO, if we're declaring a product policy resource without a previous import, the intent is clear - the operator wishes to replace the base policy with their custom one, regardless of the contents of the base policy previously applied by Azure. Either the azurerm_api_management_product resource should not create a base policy at all (Which would probably be quite difficult to achieve, I would assume Azure wouldn't be too happy to cope with this) or the azurerm_api_management_product_policy resource should remove the default policy, when you specify a custom one - perhaps if a certain option is not specified, or set to true?

gercograndia commented 8 months ago

IMHO, if we're declaring a product policy resource without a previous import, the intent is clear - the operator wishes to replace the base policy with their custom one, regardless of the contents of the base policy previously applied by Azure. Either the azurerm_api_management_product resource should not create a base policy at all (Which would probably be quite difficult to achieve, I would assume Azure wouldn't be too happy to cope with this) or the azurerm_api_management_product_policy resource should remove the default policy, when you specify a custom one - perhaps if a certain option is not specified, or set to true?

I do agree with this. Strange thing though is that I ran into it and had to import the (still default) policy in order to proceed, but later I haven't ran into the issue anymore and could create them without issues.

Quite some impredictable behaviour, which unfortunately happens more often on Azure.

davejhahn commented 5 months ago

I am running into this, and consistently. The dev teams are in the process of making changes that require only changing policies to the APIs, and even though the Terraform build plan clearly shows that it will destroy and create only the policy, it fails that the resource id for the API already exists, even though it was previously deployed, in the state fail, and should not be deploying.

The workaround they have been using is deleting the version and re-deploying, which is pretty destructive.

ruimaciel commented 3 weeks ago

I'm also running into this problem. I'm using AzureRM v3.95.0, Terraform v1.8.5.

It would be helpful if there was any official guidance on how users can unblock themselves once they stumble upon this problem.

hashicorp / terraform-provider-azurerm