search

apache / iceberg-rust

Apache Iceberg
https://rust.iceberg.apache.org/
Apache License 2.0
673 stars 159 forks source link

feat: abstract the MetricsEvaluator #674

Open sundy-li opened 1 month ago

sundy-li commented 1 month ago

Every query engine has its expression framework to prune files.

The current Predicate in iceberg-rust is very simple, it's not powerful yet.

For example, it does not support Cast Expression and other complex expression. such as: filter = abs( cast(id2 as int) ) > id2

Though this filter is rarely used, query engine can make its own effort to calculate the domain based on the stats (Such as ConstantFolder framework in databend)

liurenjie1024 commented 2 weeks ago

Good point, I believe this could be achieved by allowing injecting metrics evaluator into table scan planning to help pruning more unnecessary data files.