azurerm_machine_learning_datastore_datalake_gen2 can not find storage account from a different subscription after creation

ltutar commented 2 months ago

Is there an existing issue for this?

[X] I have searched the existing issues

Community Note

Please vote on this issue by adding a :thumbsup: reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave comments along the lines of "+1", "me too" or "any updates", they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment and review the contribution guide to help.

Terraform Version

1.9.5

AzureRM Provider Version

4.0.1

Affected Resource(s)/Data Source(s)

azurerm_machine_learning_datastore_datalake_gen2

Terraform Configuration Files

resource "azurerm_machine_learning_datastore_datalake_gen2" "dsleventdev" {
  name                 = "levent"
  description          = "dsleventdev"
  workspace_id         = module.mlwsleventdev.id
  storage_container_id = data.azurerm_storage_containers.raw_layer_containers.containers[index(data.azurerm_storage_containers.raw_layer_containers.containers[*].name, var.levent_storage_account_container_name)].resource_manager_id
  tags                 = module.foundation.tags
}

# raw_layer_containers and the data storage are existing in a different Azure subscription.

Debug Output/Panic Output

Planning failed. Terraform encountered an error while generating this plan.
╷
│ Error: Unable to locate Storage Account "stvocmpprdweu002"!
│ 
│   with azurerm_machine_learning_datastore_datalake_gen2.dsleventdev,
│   on main.tf line 122, in resource "azurerm_machine_learning_datastore_datalake_gen2" "dsleventdev":
│  122: resource "azurerm_machine_learning_datastore_datalake_gen2" "dsleventdev" {
│ 
│ Unable to locate Storage Account "stvocmpprdweu002"!
╵
Operation failed: failed running terraform plan (exit 1)
ltutar@Levents-Laptop terraform %

Expected Behaviour

I create azurerm_storage_container resources in a different azure subscription. I can create the resource azurerm_machine_learning_datastore_datalake_gen2 for the first time through terraform by referring to these container resources. I can also see the result in Azure Machine Learning Workspace and the datastore is working properly. If there is no change in the implementation, I expect terraform plan to show "No changes Your infrastructure matches the configuration"

Actual Behaviour

I create azurerm_storage_container resources in a different azure subscription. I can create the resource azurerm_machine_learning_datastore_datalake_gen2 for the first time through terraform by referring to these container resources. I can also see the result in Azure Machine Learning Workspace and the datastore is working properly.

Even terraform plan shows "No changes Your infrastructure matches the configuration" However, when I do terraform plan again. It will fail with the error:

Error: Unable to locate Storage Account "stvocmpprdweu002"!
with azurerm_machine_learning_datastore_datalake_gen2.dsleventdev
on main.tf line 116, in resource "azurerm_machine_learning_datastore_datalake_gen2" "dsleventdev":
resource "azurerm_machine_learning_datastore_datalake_gen2" "dsleventdev" {
Unable to locate Storage Account "stvocmpprdweu002"!

I was expecting terraform plan not to give any errors like "Error: Unable to locate Storage Account "stvocmpprdweu002""


Error: Unable to locate Storage Account "stvocmpprdweu002"!
with azurerm_machine_learning_datastore_datalake_gen2.dsleventdev
on main.tf line 122, in resource "azurerm_machine_learning_datastore_datalake_gen2" "dsleventdev":

resource "azurerm_machine_learning_datastore_datalake_gen2" "dsleventdev" {

Unable to locate Storage Account "stvocmpprdweu002"!

Steps to Reproduce

provider "azurerm" { features {} }

provider "azurerm" { features {} alias = "cmpprd" subscription_id = "xxx" # the other subscription that has the storage account }

resource "azurerm_machine_learning_datastore_datalake_gen2" "dsleventdev" { name = "levent" description = "dsleventdev" workspace_id = module.mlwsleventdev.id storage_container_id = data.azurerm_storage_containers.raw_layer_containers.containers[index(data.azurerm_storage_containers.raw_layer_containers.containers[*].name, var.levent_storage_account_container_name)].resource_manager_id tags = module.foundation.tags }

Important Factoids

The datastore and the azurerm_storage_containers are in a different subscriptoin

References

No response

ltutar commented 2 months ago

If I update the terraform implementation, terraform plan will fail also and not even show the changes.

This resource breaks the whole terraform after the initial successful resource creation.

module.mlwsleventdev.azurerm_role_assignment.ml_workspace_access_to_sa_pe_queue: Refreshing state...
module.mlwsleventdev.azurerm_role_assignment.ml_workspace_access_to_sa_pe_table: Refreshing state...
module.mlwsleventdev.azurerm_role_assignment.ml_workspace_access_to_sa_pe_file: Refreshing state...
module.mlwsleventdev.azurerm_role_assignment.ml_workspace_access_to_sa_pe_blob: Refreshing state...
module.mlwsleventdev.azurerm_role_assignment.ml_workspace_access_to_sa_sbdr: Refreshing state...
module.mlwsleventdev.azurerm_machine_learning_compute_cluster.compute_cluster: Refreshing state...
module.mlwsleventdev.azurerm_role_assignment.compute_cluster_access_to_ml_workspace_c: Refreshing state...

Planning failed. Terraform encountered an error while generating this plan.

╷
│ Error: Unable to locate Storage Account "stvocmpprdweu002"!
│
│   with azurerm_machine_learning_datastore_datalake_gen2.dsleventdev,
│   on main.tf line 122, in resource "azurerm_machine_learning_datastore_datalake_gen2" "dsleventdev":
│  122: resource "azurerm_machine_learning_datastore_datalake_gen2" "dsleventdev" {
│
│ Unable to locate Storage Account "stvocmpprdweu002"!
╵
Operation failed: failed running terraform plan (exit 1)

harshavmb commented 2 months ago

This is a bug. During create & update calls, storageAccount checks are not performed like it is performed in other resources like azurerm_storage_share, azurerm_storage_table.

Only during read operations, storageAccount are made here causing resource to fail post creation. If similar calls are made in create, update & ~delete~ calls this should be caught early.

ltutar commented 2 months ago

Thank you @harshavmb for looking into this issue. I really appreciate that. Let me know if I can do anything from my side like testing or something to have the solution on a short term. Using this resource is now breaking the whole terraform plan. I commented the code now and removed it from the terraform state. The resource is created with terraform but not maintained with terraform for the time being.

Note: I forgot to mention abouve. The creation of the resource with terraform is also giving an error but the resource is created successfully in Azure and can be used without any problems.

Also the implementation is working fine if I stay within the same subscription instead of referring to a data storage in an another subscription. Just trying to explain the situation.

ltutar commented 2 months ago

@harshavmb May I also ask a question related to your pull request? I read "*Note** storage_container_id should be in the same subscription as azurerm_machine_learning_datastore_datalake_gen2 resource." at https://github.com/hashicorp/terraform-provider-azurerm/pull/27256/files#diff-22da9b8d569a3e4441677deac2abbab589ac42c39f375d9e6ac4d2481e7b5d7aR86

Does this mean that I am not able to use the terraform resource azurerm_machine_learning_datastore_datalake_gen2 to refer to a data storage in an another subscription like I am trying to do now?

harshavmb commented 2 months ago

Hi @ltutar ,

Yeah, this change I pushed would fail right before create call so that you don't end up having a resource created with a storage account from another subscription.

If Azure supports provisioning of azurerm_machine_learning_datastore_datalake_gen2 from a storage account in another subscription, I would like to know the feedback of azurerm maintainers. Some additional parsing has to be done to find the subscriptionId from storage_container_id & build storageClient.

ltutar commented 2 months ago

Hi @harshavmb Sorry for spamming you. How can I get the feedback azurerm maintainers. Do I need to create a feature request? I would like to how this functionality i.e. referring to a data store in a different subscription in terraform.

hashicorp / terraform-provider-azurerm