[Feature] Optimize serialization of TableSchema

apache / paimon

Apache Paimon is a lake format that enables building a Realtime Lakehouse Architecture with Flink and Spark for both streaming and batch operations.

https://paimon.apache.org/

Apache License 2.0

2.35k stars 928 forks source link

[Feature] Optimize serialization of TableSchema #679

Closed JingsongLi closed 1 year ago

JingsongLi commented 1 year ago

Search before asking

[X] I searched in the issues and found nothing similar.

Motivation

Now the serialization of TableSchema is more violent, once it is determined that there are watermark or calculated columns, is to turn all schema into options stuffed into. We should only serialize watermark and calculated columns.

Solution

No response

Anything else?

No response

Are you willing to submit a PR?

[ ] I'm willing to submit a PR!

hililiwei commented 1 year ago

I'm interested in this. May I have the ticket?

yuzelin commented 1 year ago

@hililiwei Sorry for late response. This issue is solved by https://github.com/apache/incubator-paimon/pull/1041