search

databrickslabs / overwatch

Capture deep metrics on one or all assets within a Databricks workspace
Other
230 stars 64 forks source link

[EPIC] -- Enable Support for DLT - Pipelines in Workflows #479

Open GeekSheikh opened 2 years ago

GeekSheikh commented 2 years ago

cluster_ids are not identified for DLT Workflows yet. Clearly there is a cluster but it's not linked in the workflow.

There's another service deltaPipelines but it also doesn't seem to provide the cluster_id relationship. May need DBR enhancement.

2022-08-11 12_49_28-job_run_silver_mtj_build - Databricks - Brave 2022-08-11 12_53_18-Databricks - Brave

GeekSheikh commented 2 years ago

Dependent on #545 outcome.