[Feature] Paimon Table And Data Lineage Management For Flink - Githubissues

apache / paimon

Apache Paimon is a lake format that enables building a Realtime Lakehouse Architecture with Flink and Spark for both streaming and batch operations.

https://paimon.apache.org/

Apache License 2.0

2.12k stars 841 forks source link

[Feature] Paimon Table And Data Lineage Management For Flink #1105

Open FangYongs opened 1 year ago

FangYongs commented 1 year ago

Search before asking

[X] I searched in the issues and found nothing similar.

Motivation

Manage table and data lineages for Flink ETL jobs in Paimon, users can validate, debug and revise data and computation for Flink streaming and batch ETL. We(@liming30, @schnappi17 and I) will give a detailed PIP for this feature later. The PIP is https://cwiki.apache.org/confluence/display/PAIMON/PIP-5%3A+Paimon+Table+And+Data+Lineage+For+Flink

Solution

No response

Anything else?

No response

Are you willing to submit a PR?

[ ] I'm willing to submit a PR!

JingsongLi commented 1 year ago

Love this 👍🏻

legendtkl commented 1 year ago

+1

czy006 commented 4 months ago

I would like to know if this feature is currently available in 0.7 @FangYongs