apache / paimon

Apache Paimon is a lake format that enables building a Realtime Lakehouse Architecture with Flink and Spark for both streaming and batch operations.
https://paimon.apache.org/
Apache License 2.0
2.1k stars 834 forks source link

[cdc] Fix richCdcMultiplexRecord cant be deserialized by kryoSerializer. #3631

Closed LinMingQiang closed 3 days ago

LinMingQiang commented 4 days ago

Purpose

Linked issue: close #3630

Tests

API and Format

Documentation

LinMingQiang commented 4 days ago

Kryo serializer use com.esotericsoftware.kryo.serializers.CollectionSerializer to deserialize Collection type class, CollectionSerializer#read will create a unmodifiable Collection instance depends on fields Type (unmodifiableList) As the picture shows, when kryo add element info the Collection will throw UnsupportedOperationException because Collection is a unmodifiableList.

image
LinMingQiang commented 4 days ago

RichCdcMultiplexRecord.fields is instantiated in RowType construction method.

Construct RichCdcMultiplexRecord:

image

Extract field.

image

Build field.

image
yuzelin commented 4 days ago

Cool~ Waiting the CI.

LinMingQiang commented 4 days ago

@JingsongLi help to review this issue, thanks.

LinMingQiang commented 4 days ago

👌

JingsongLi commented 3 days ago

+1