MarquezProject / marquez

Collect, aggregate, and visualize a data ecosystem's metadata
https://marquezproject.ai
Apache License 2.0
1.78k stars 320 forks source link

[Question] Using API Key with Spark Open Lineage integration #2971

Open TejasRGitHub opened 1 week ago

TejasRGitHub commented 1 week ago

Hi Team,

We have multiple spark jobs, and we want to generate lineage using Spark integration for Open-lineage (https://openlineage.io/docs/integrations/spark/).

From this documentation, an API Key can be configured in the Spark Jobs ( https://openlineage.io/docs/integrations/spark/configuration/transport ).

From this GH issue , an API key support was added to the Java and Python Clients ( https://github.com/MarquezProject/marquez/issues/817 ).

Can you please confirm, if the API key support is present in Marquez and how can this authentication support be configured for Marquez API