search

apache / paimon

Apache Paimon is a lake format that enables building a Realtime Lakehouse Architecture with Flink and Spark for both streaming and batch operations.
https://paimon.apache.org/
Apache License 2.0
2.48k stars 970 forks source link

[core] Support snapshot system pushdown with CompoundPredicate and fix FileNotFound error when snapshot id not exist #4135

Closed xuzifu666 closed 2 months ago

xuzifu666 commented 3 months ago

Purpose

Linked issue: close #xxx The pr had 2 point to support:

  1. currently query snapshot system with predicate only support leaf predicate for total push down to SnapshotManager, the pr handle CompoundPredicate to support multiple predicate and all push down to SnapshotManager when select * from T$snapshots where snapshot_id > 1 and snapshot < 5;

before fix:

image

after fix:

image
  1. Snapshot system table not handle with not exist snapshot id query, it would error out with FileNotFould error. this pr handle it.
image

Tests

API and Format

Documentation

Tests

API and Format

Documentation