search

MarquezProject / marquez

Collect, aggregate, and visualize a data ecosystem's metadata
https://marquezproject.ai
Apache License 2.0
1.78k stars 320 forks source link

[Question] - Can lineage data in marquez be exported in open lineage format #2970

Closed TejasRGitHub closed 1 week ago

TejasRGitHub commented 1 week ago

Hi Team,

Is there a way in the UI or with the API with which I can export all the lineage data from Marquez ?

phixMe commented 1 week ago

The data is all captured within the lineage_events table... You could theoretically wipe the data in every other table (although there are a bunch of foreign keys that prevent unilateral drops) and replay events captured in the lineage_events table.

Also, in the User Interface, there is an events page that renders the entirety of the payload if you wanted to use that... There is a corresponding api that is queried by this page that takes date bounds.