Support for systemDatastoresAuthMode in azurerm_machine_learning_workspace

wojciech-jakubowski commented 1 year ago

Is there an existing issue for this?

[X] I have searched the existing issues

Community Note

Please vote on this issue by adding a :thumbsup: reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment

Description

As of ARM API version 2022-12-01-preview AzureML has support for connecting to workspace storage account based on workspace's managed identity instead of account key.

On Azure portal UI they call this setting Storage account access type. Under the hood its controlled by ARM json property called: systemDatastoresAuthMode (https://learn.microsoft.com/en-us/azure/templates/microsoft.machinelearningservices/2022-12-01-preview/workspaces?pivots=deployment-language-terraform#workspaceproperties-2).

It would be nice to have support for this in AzureRM provider in azurerm_machine_learning_workspace, so it would be possible to disable SA key usage (which is not very secure).

New or Affected Resource(s)/Data Source(s)

azurerm_machine_learning_workspace

Potential Terraform Configuration

resource "azurerm_machine_learning_workspace" "example" {
  name                    = "example-workspace"
  location                = azurerm_resource_group.example.location
  resource_group_name     = azurerm_resource_group.example.name
  application_insights_id = azurerm_application_insights.example.id
  key_vault_id            = azurerm_key_vault.example.id
  storage_account_id      = azurerm_storage_account.example.id
  system_datastores_auth_mode = "identity"  #or "account_key"

  identity {
    type = "SystemAssigned"
  }
}

References

No response

marvinbuss commented 3 months ago

Even with this new property, you would not be able to disable key access on the connected storage account. Azure ML relies on Azure Files and uses the storage account key for mounting the file share when using compute instances. This property just ensures that the default datastores (e.g. workspaceartifactstore, workspaceblobstore) use identity-based auth.

wojciech-jakubowski commented 3 months ago

Yes, that is true. The networking design of AzureML is unfortunate in some places. I'd really like to get rid of storage account keys where I can.

marvinbuss commented 3 months ago

I fully understand the need to disable key based auth on all storage accounts. This is not an Azure ML related issue but more a limit of Azure Files which Azure ML depends on for some scenarios.

hashicorp / terraform-provider-azurerm