[Feature][cdc] Extract schema change from record

apache / paimon

Apache Paimon is a lake format that enables building a Realtime Lakehouse Architecture with Flink and Spark for both streaming and batch operations.

https://paimon.apache.org/

Apache License 2.0

2.28k stars 912 forks source link

[Feature][cdc] Extract schema change from record #1392

Open JingsongLi opened 1 year ago

JingsongLi commented 1 year ago

Search before asking

[X] I searched in the issues and found nothing similar.

Motivation

Currently, in cdc synchronization, there may be a situation where the schema comes later but the data comes first, which may lead to a dead wait on the write side.

If we extract the schema change from the data, it is in the realm of invincibility and can never go wrong.

Solution

No response

Anything else?

No response

Are you willing to submit a PR?

[ ] I'm willing to submit a PR!

Alibaba-HZY commented 1 year ago

This is a very good feature,I have interest to work for this issue. Please assign this issue to me.