[cdc] Fix keyAndPartitions extractor result conflict if partitions or keys value is empty.

apache / paimon

Apache Paimon is a lake format that enables building a Realtime Lakehouse Architecture with Flink and Spark for both streaming and batch operations.

https://paimon.apache.org/

Apache License 2.0

2.12k stars 839 forks source link

[cdc] Fix keyAndPartitions extractor result conflict if partitions or keys value is empty. #3574

Closed LinMingQiang closed 1 week ago

LinMingQiang commented 1 week ago

Purpose

Linked issue: close #3570

Why causes conflict : CdcRecordKeyAndBucketExtractor use CdcRecordUtils#projectAsInsert to project partition and keys genericRow , the projectAsInsert will ignore the empty value and eventually write null into GenericRow, but RowDataKeyAndBucketExtractor would be not ignore empty value and write empty value into GenericRow.

Tests

API and Format

Documentation

LinMingQiang commented 1 week ago

@JingsongLi Help to review this PR, thanks.

JingsongLi commented 1 week ago