Use `api.models.Metadata` - Githubissues

MarquezProject / marquez

Collect, aggregate, and visualize a data ecosystem's metadata

https://marquezproject.ai

Apache License 2.0

1.78k stars 320 forks source link

Use `api.models.Metadata` #2950

Open wslulciuc opened 4 weeks ago

wslulciuc commented 4 weeks ago

In https://github.com/MarquezProject/marquez/pull/2853, we introduced api.models.Metadata, a utility class for OpenLineage.RunEvent, OpenLineage.JobEvent, and OpenLineage.DatasetEvent. Let's now use the class on the event write path POST /api/v1/lineage to allow for:

Event validation (returning 400 on invalid events)
Seamlessly access OpenLineage metadata: runID, job, datasets, facets, etc