search

apache / paimon

Apache Paimon is a lake format that enables building a Realtime Lakehouse Architecture with Flink and Spark for both streaming and batch operations.
https://paimon.apache.org/
Apache License 2.0
2.16k stars 855 forks source link

[flink] Fix lookup IndexOutOfBounds for sequence defined with projection #3527

Closed JingsongLi closed 1 month ago

JingsongLi commented 1 month ago

Purpose

Caused by: java.lang.ArrayIndexOutOfBoundsException: 3
    at org.apache.paimon.utils.ProjectedRow.isNullAt(ProjectedRow.java:80)
    at RecordComparator$52.compare(Unknown Source)
    at org.apache.paimon.utils.UserDefinedSeqComparator.compare(UserDefinedSeqComparator.java:49)
    at org.apache.paimon.utils.UserDefinedSeqComparator.compare(UserDefinedSeqComparator.java:32)
    at org.apache.paimon.flink.lookup.SecondaryIndexLookupTable.refresh(SecondaryIndexLookupTable.java:88)
    at org.apache.paimon.flink.lookup.FullCacheLookupTable.refresh(FullCacheLookupTable.java:154)
    at org.apache.paimon.flink.lookup.FileStoreLookupFunction.refresh(FileStoreLookupFunction.java:299)
    at org.apache.paimon.flink.lookup.FileStoreLookupFunction.checkRefresh(FileStoreLookupFunction.java:288)
    at org.apache.paimon.flink.lookup.FileStoreLookupFunction.lookup(FileStoreLookupFunction.java:206)
    ... 87 more

FullCacheLookupTable should pad sequence fields for projection.

Tests

LookupTableTest.testPkTableWithSequenceFieldProjection

API and Format

Documentation