search

microsoft / Purview-ADB-Lineage-Solution-Accelerator

A connector to ingest Azure Databricks lineage into Microsoft Purview
MIT License
92 stars 56 forks source link

Unable to fetch lineage when running databricks notebook from azure data factory #201

Open GouravJ123 opened 1 year ago

GouravJ123 commented 1 year ago

Hello,

while running the databricks notebook directly, i am able to get the lineage in azure purview but when i run the same notebook using azure data factory and select databricks notebook as activity then its throwing error in purviewout azure function. The error is : AdbClient-isTokenExpired: Token cache hit and "AdbClient-GetSingleAdbJobAsync: error, message: Response status code does not indicate success: 404 (Not Found)." Can you please help me in resolving this issue

gianlucadardia commented 9 months ago

I am facing the same issue. Any thoughts?

wjohnson commented 9 months ago

@GouravJ123 and @gianlucadardia by any chance are you using multiple databricks workspaces? There may be an issue if you have a Function that handles multiple databricks workspaces that could be causing this error.

jogacolhue commented 5 months ago

Notebooks called by Data Factory are considered Databricks Jobs, so you can try to follow the last part of the stand-alone guide