Open kimyen opened 2 months ago
Hi @kimyen ,
Thank you for taking time open this issue and apologize for late response.
Thanks for the details and I could reproduce this issue. It seems that this API only works if the outbound rules are created one by one.
The azapi_resource
supports locks
field which allows user to specify a list of ARM resource IDs which are used to avoid create/modify/delete azapi resources at the same time.
But I also noticed that there is an API bug(https://github.com/Azure/azure-rest-api-specs/issues/28982) which will make the azapi v1.13.x crash. I have two workarounds for this case, hope it could help.
Workaround 1. (Recommended)
Use azapi v1.12.1 to deploy the following config, and you could upgrade to the latest once the bug fix is released.
resource "azurerm_machine_learning_workspace" "example" {
name = "acctesthenglu562"
location = azurerm_resource_group.example.location
resource_group_name = azurerm_resource_group.example.name
application_insights_id = azurerm_application_insights.example.id
key_vault_id = azurerm_key_vault.example.id
storage_account_id = azurerm_storage_account.example.id
identity {
type = "SystemAssigned"
}
public_network_access_enabled = true
managed_network {
isolation_mode = "AllowOnlyApprovedOutbound"
}
}
resource "azapi_resource" "example" { count = 3 type = "Microsoft.MachineLearningServices/workspaces/outboundRules@2023-10-01" name = "test2${count.index}" parent_id = azurerm_machine_learning_workspace.example.id body = jsonencode({ properties = { category = "UserDefined" status = "Active" type = "FQDN" destination = "conda.anaconda${count.index}.org" } }) locks = [azurerm_machine_learning_workspace.example.id] }
Workaround 2.
If you prefer the dynamic properties that v1.13.x provides, you could use the `azapi_resource_action` to bypass the bug, however the action resource doesn't monitor the resource's state.
```hcl
data "azapi_resource_id" "outboundRules" {
count = 3
type = "Microsoft.MachineLearningServices/workspaces/outboundRules@2023-10-01"
name = "test2${count.index}"
parent_id = azurerm_machine_learning_workspace.example.id
}
resource "azapi_resource_action" "outboundRules" {
count = 3
type = "Microsoft.MachineLearningServices/workspaces/outboundRules@2023-10-01"
resource_id = data.azapi_resource_id.outboundRules[count.index].id
method = "PUT"
locks = [azurerm_machine_learning_workspace.example.id]
body = {
properties = {
category = "UserDefined"
status = "Active"
type = "FQDN"
destination = "repo.anaconda.org${count.index}"
}
}
}
@ms-henglu Do you have any additional updates on this?
I did try the first workaround you listed (the second won't work for us as we need Terraform to monitor the state). It worked, partially. I was able to create the FQDNs and PEs outbound, but I had to force them to be created only one at a time.
In addition, if you later add/modify the azapi rules, terraform still sometimes deletes previously created outbound rules. This makes it extremely difficult to add new rules.
@ms-henglu Do you have any additional updates on this?
I did try the first workaround you listed (the second won't work for us as we need Terraform to monitor the state). It worked, partially. I was able to create the FQDNs and PEs outbound, but I had to force them to be created only one at a time.
In addition, if you later add/modify the azapi rules, terraform still sometimes deletes previously created outbound rules. This makes it extremely difficult to add new rules.
@Chaseshak how did you force to be created only one at a time
Hi @krupakar1329 - You could use the lock
field to force the rules to be created only once at a time. Please see above comment.
Hi @Chaseshak,
In addition, if you later add/modify the azapi rules, terraform still sometimes deletes previously created outbound rules. This makes it extremely difficult to add new rules.
Would you please provide some details so I could reproduce it? Thanks!
Brief description of the problem
Microsoft.MachineLearningServices/workspaces/outboundRules@2023-10-01
as documented here to create multiple outbound rules, the behaviors is non deterministic and destructive.terraform apply
will report successfully created X rules.terraform state list
&terraform state show
can correctly show to the rules created. However, the Azure portal shows only the one that has not been deleted. The followingterraform plan
without code change will show that the rules that are deleted need to be re-created.How to reproduce
Step 1: Add the following to your workspace
.tf
file:terraform plan
, there should be 2 FQDN rules to be createdStep 2: Run
terraform apply
, after ~20 minutes, it should succeedterraform state list
, the state should be presentterraform state shown
, each state should have full detailsStep 3: Run
terraform plan
, 1/2 rules need to be createdOther setup
Allow only approved outbound
Desired resolution
terraform apply
and it runs to completion, all outbound rules are created and visible in Azure portal. The followingterraform plan
without code change results in no changes.