microsoft / Purview-ADB-Lineage-Solution-Accelerator

A connector to ingest Azure Databricks lineage into Microsoft Purview
MIT License
90 stars 55 forks source link

There's no longer support to use init scripts from dbfs in Databricks #234

Open jogacolhue opened 3 months ago

jogacolhue commented 3 months ago

Describe the bug When I try to run the demo, it fails during Databricks configuration. I've read the script and it seems to run the init script from dbfs, that is no longer supported in new environments.

To Reproduce Steps to reproduce the behavior:

  1. Run the demo in a new resource group.

Expected behavior The demo should finis without problems.

Logs 2024-03-27 15:48:39 [INFO] begin databricks demo cluster creation {"error_code":"FEATURE_DISABLED","message":"Init scripts cannot be loaded from DBFS. Review the init script documentation for more details on configuring init scripts.","details":[{"@type":"type.googleapis.com/google.rpc.ErrorInfo","reason":"CM_API_ERROR_SOURCE_CALLER_ERROR","domain":""}]}

Screenshots image

Desktop (please complete the following information):

Additional context Databricks documentation