MarquezProject / marquez

Collect, aggregate, and visualize a data ecosystem's metadata
https://marquezproject.ai
Apache License 2.0
1.78k stars 320 forks source link

[FIX] Column lineage query with null namespace and name #2881

Closed sophiely closed 2 months ago

sophiely commented 2 months ago

Problem

Closes: #2875

In some case the input dataset used seems not to be in dataset_views table so when joining the two tables it results in null for namespace and name and make the list dataset endpoint fail. dataset_views only contains the latest dataset version so if the column lineage is created with an old version, this old version will ne be in the dataset_view and then results in an empty namespace and name.

Solution

We can get the namespace and name in the dataset_versions table to make sure to have all the version of a dataset.

Checklist

netlify[bot] commented 2 months ago

Deploy Preview for peppy-sprite-186812 canceled.

Name Link
Latest commit 9f60d046a74073c958db105201dd0327692acdca
Latest deploy log https://app.netlify.com/sites/peppy-sprite-186812/deploys/66ccab7d1e01560008d2a5f0
codecov[bot] commented 2 months ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 83.28%. Comparing base (915aff3) to head (9f60d04). Report is 1 commits behind head on main.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #2881 +/- ## ========================================= Coverage 83.28% 83.28% Complexity 1477 1477 ========================================= Files 259 259 Lines 6785 6785 Branches 313 313 ========================================= Hits 5651 5651 Misses 977 977 Partials 157 157 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.