Error reading azurerm_backup_protected_vm - BMSUserErrorCrossRegionRestoreNotEnabledInThisVault

[UntiIted]

Is there an existing issue for this?

• [X] I have searched the existing issues

Community Note

• Please vote on this issue by adding a :thumbsup: reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform Version

1.3.9

AzureRM Provider Version

3.55.0

Affected Resource(s)/Data Source(s)

azurerm_backup_protected_vm

Terraform Configuration Files

resource "azurerm_backup_protected_vm" "this" {
  recovery_vault_name = "example-vault"
  resource_group_name = "example"
  source_vm_id        = "/subscriptions/xxx/resourceGroups/example/providers/Microsoft.Compute/virtualMachines/example-dev"
  backup_policy_id    = "/subscriptions/xxx/resourceGroups/example/providers/Microsoft.RecoveryServices/vaults/example-vault/backupPolicies/TestEnvPolicy"
}

Debug Output/Panic Output

Error: checking for presence of existing Protected Item (Subscription: "xxx"
│ Resource Group Name: "example"
│ Vault Name: "example-vault"
│ Backup Fabric Name: "Azure"
│ Protection Container Name: "iaasvmcontainer;iaasvmcontainerv2;example;example-dev"
│ Protected Item Name: "VM;iaasvmcontainerv2;example;example-dev"): protecteditems.ProtectedItemsClient#Get: Failure responding to request: StatusCode=400 -- Original Error: autorest/azure: Service returned an error. Status=400 Code="BMSUserErrorCrossRegionRestoreNotEnabledInThisVault" Message="Cross Region Restore is not enabled in this vault. Please enable Cross Region Restore in this vault."

Expected Behaviour

Resource read request should pass successfully, azurerm_backup_protected_vm created.

Actual Behaviour

Resource read request fails with 400.

Steps to Reproduce

Create any valid configuration for a backup protected vm and try to run terraform apply. Terraform plan will also fail for existing resources.

This can also be reproduced with a simple rest method GET request using api version 2023-02-01. It will give the following output for both existing and non-existing resources: Bad Request({"error":{"code":"BMSUserErrorCrossRegionRestoreNotEnabledInThisVault","message":"Cross Region Restore is not enabled in this vault. Please enable Cross Region Restore in this vault."}})

Meanwhile api version 2021-02-10 returns as expected: Not Found({"error":{"code":"BMSUserErrorDataSourceObjectNotFound","message":"Item not found. Selected item might have been deleted. Please check if item is registered to the vault."}})

API reference: https://learn.microsoft.com/en-us/rest/api/backup/protected-items/get?tabs=HTTP

Important Factoids

No response

References

No response

[Misterjohnson87]

Also experiencing this same issue

[mpjtaylor]

Also seeing this issue is Release v3.55.0 Rolling back to v3.54.0 removed the issue

[ElPrincidente]

+1 on the issue. Reverting to 3.54.0 bypasses the error for us.

[bcline760]

May have something to do with #21575 as I am experiencing this issue as well.

Potential work around is in your azurerm_recovery_services_vault resource, explicitly set the cross_region_restore_enabled parameter to true. I did a quick test with this, and it checks out.

[ziyeqf]

This is a known issue of Azure services on API version 2023-02-01 and the fix is being deployed. Will update here once it resolved

[nickchristie]

I actually have storage_mode_type = "ZoneRedundant" in my azurerm_recovery_services_vault but get the error Code="BMSUserErrorCrossRegionRestoreZRSVaultNotPossible" Message="Cross Region Restore cannot be enabled in ZRS vault. Cross region Restore cannot be enabled in ZRS vault but only in GRS vault." whether I omit cross_region_restore_enabled (default is false) or set it explicitly to false. Hoping that will be resolved by the same fix.

[myc2h6o]

@nickchristie the error code you mentioned is a different one, have you tried with version <=3.54.0?

[nickchristie]

@myc2h6o With 3.54.0 I get no error; my azurerm_recovery_services_vault is created with Zone Redundancy. I have:

resource "azurerm_recovery_services_vault" "recovery_services_vault" {
  name                = "${var.customer_name}FileShareBackupVault"
  location            = var.location
  resource_group_name = var.resource_group_name
  sku                 = "Standard"
  storage_mode_type   = "ZoneRedundant"
  cross_region_restore_enabled = false

  tags = {
    ManagedByTerraform = true
  }
}

which results in

 # module.customer.module.rsv.azurerm_recovery_services_vault.recovery_services_vault will be created
  + resource "azurerm_recovery_services_vault" "recovery_services_vault" {
      + classic_vmware_replication_enabled = (known after apply)
      + cross_region_restore_enabled       = false
      + id                                 = (known after apply)
      + immutability                       = (known after apply)
      + location                           = "uksouth"
      + name                               = "testFileShareBackupVault"
      + public_network_access_enabled      = true
      + resource_group_name                = "test"
      + sku                                = "Standard"
      + soft_delete_enabled                = true
      + storage_mode_type                  = "ZoneRedundant"
      + tags                               = {
          + "ManagedByTerraform" = "true"
        }
    }

which works with 3.54.0. Before this week I had neither storage_mode_type nor cross_region_restore_enabled in my Terraform code and it was working, though I am uncertain if the result was ZRS or GRS mode. I don't know at what point I started using AzureRM 3.55; that was automatic. Sorry for any vagueness; I have only just started using Terraform and I cannot tell if my issue has the same underlying cause as the original post.

[ketansiddhapura]

yes, same issue on resource "azurerm_backup_protected_vm" when used azurerm = {     source  = "hashicorp/azurerm"     version = "> 3..0"   }

Yes I had to use < 3.54 and it solved the issue

[avooca]

Any idea when we might see this fix..?

[ziyeqf]

Hi @avooca , this is going to be fixed in days.

[Bluffkin]

To confirm, downgrading to 3.54.0 solved the problem for me.

[avooca]

Yep, just not in the mood. A minor version shouldn't be introducing API-breaking changes.

[saurabh-sethi]

Facing same issue with 3.56.0

[paulsteer-STW]

downgrading to 3.54.0 solved the problem for me too

[marciobarbato]

3.56 is reporting the same

[khankf1]

yup, downgrading from 3.56.0 to 3.54.0 solved the issue

[avooca]

@ziyeqf I see 21686 is closed now, saying its no longer needed due to fix on Azure side, but behavior hasn't changed. Is there a regional rollout occuring?

[ziyeqf]

Hi @avooca , just gave it a try without any error. Could you please give it another try? And if it's still error out could you please provide which region you are using?

Thanks

[ziyeqf]

Hi all, this should have been fixed on Azure, so 3.55.0 and 3.56.0 should work without this error. Please give it a try.

[avooca]

Hi @avooca , just gave it a try without any error. Could you please give it another try? And if it's still error out could you please provide which region you are using?

Thanks

All good in West Europe now, this morning not so much. Appreciate the follow up.

[mybayern1974]

I'm closing this issue by seeing this issue is resolved. Feel free to suggest re-activate if you see further issues.

[github-actions[bot]]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.