azurerm_machine_learning_datastore_blobstorage throws error when using service_data_auth_identity meta argument

VickyWinner commented 1 year ago

Is there an existing issue for this?

[X] I have searched the existing issues

Community Note

Please vote on this issue by adding a :thumbsup: reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform Version

Terraform v1.0.4

AzureRM Provider Version

provider registry.terraform.io/hashicorp/azurerm v3.52.0

Affected Resource(s)/Data Source(s)

azurerm_machine_learning_datastore_blobstorage

Terraform Configuration Files

resource "azurerm_machine_learning_datastore_blobstorage" "example" {
  name                 = "test"
  workspace_id         = module.azureml.id
  storage_container_id = data.azurerm_storage_container.example.id
  service_data_auth_identity = "WorkspaceSystemAssignedIdentity"
}

Debug Output/Panic Output

Error: Missing required argument

  with azurerm_machine_learning_datastore_blobstorage.example,
  on register_storage_blob.tf line 1, in resource "azurerm_machine_learning_datastore_blobstorage" "bookitnow":
   1: resource "azurerm_machine_learning_datastore_blobstorage" "example" {

"shared_access_signature": one of `account_key,shared_access_signature` must
be specified

Error: Invalid combination of arguments

  with azurerm_machine_learning_datastore_blobstorage.example,
  on register_storage_blob.tf line 1, in resource "azurerm_machine_learning_datastore_blobstorage" "example":
   1: resource "azurerm_machine_learning_datastore_blobstorage" "example" {

"account_key": one of `account_key,shared_access_signature` must be specified

Expected Behaviour

blobstore registered.

Actual Behaviour

terraform plan erroring out

Steps to Reproduce

use the block fromabove

Important Factoids

No response

References

https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/machine_learning_datastore_blobstorage

sinbai commented 1 year ago

@VickyWinner thanks for opening this issue. In fact, one of account_key,shared_access_signature must be specified, this is by design. I have submitted PR to update the terraform doc to clarify it. Could you please provide one of account_key, shared_access_signature in configuration to resolve above error?

VickyWinner commented 1 year ago

@sinbai, we don't want to rely on account_key or sas key at all. I am assuming that if I use service_data_auth_identity = "WorkspaceSystemAssignedIdentity" I don't need to pass account_key or sas key which gets rotated frequently. I want to use MSI similar to what is available in portal. Where is this available in TF?

tombuildsstuff commented 1 year ago

@sinbai I've reopened this issue, would you mind taking another look at this one, as @VickyWinner has mentioned, it should be possible to use a System Assigned Identity for this purpose?

mlomat commented 1 year ago

@tombuildsstuff any update here? It's very annoying issue, especially that resource for DFS is working normally :(

chiman-wend commented 4 months ago

@VickyWinner have you found a solution for your bug besides using account key or sas? I have stumbled upon the same bug.

SystemAssignedIdentity should be possible for Datastore registration or creation. Same goes for Datastore_Fileshares https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/machine_learning_datastore_fileshare

Can you take another look on this please @tombuildsstuff and @sinbai?

pavanmuni321 commented 3 weeks ago

@tombuildsstuff , @VickyWinner , @sinbai

There is more to this --- Unlike user-created datastores, the default datastores created by the Azure ML workspace system use storage account key authentication. Disabling shared key access for the storage account will result in notebooks becoming inaccessible to users irrespective if manage identities are given access to the storage account, as these notebooks are stored in the workspace's default datastores. I dont see this handled by terraform or is there other ways to enforce the use of managed identity for these default datastores instead of relying on account key/SAS for authentication?

hashicorp / terraform-provider-azurerm