databrickslabs / overwatch

Capture deep metrics on one or all assets within a Databricks workspace
Other
221 stars 60 forks source link

Invalid configuration value detected for fs.azure.account.key #1257

Open tbaert-prest opened 2 days ago

tbaert-prest commented 2 days ago

Hello,

I am trying to install overwatch on a Databricks deployed in Azure. But after many attempts, it is not ok. I am using Unity Catalog. I follow these documentations :

The storage prefix is abfss://overwatch@***.dfs.core.windows.net/ow.

I have this error message : Capture d’écran 2024-07-03 085108

I tested to add this configuration on the cluster : https://hadoop.apache.org/docs/stable/hadoop-azure/abfs.html#Azure_Managed_Identity

But without success. I am using a deployment as a JAR.

My external location and credentials are ok. I am able to read and write on the storage account with my own notebook with the same cluster. And overwatch is able to write these files : image

I don't want to use a Service Principal like suggest in this documentation : https://databrickslabs.github.io/overwatch/deployoverwatch/configureoverwatch/securityconsiderations/#azure-storage-auth-config

Thanks