apache / pinot

Apache Pinot - A realtime distributed OLAP datastore
https://pinot.apache.org/
Apache License 2.0
5.45k stars 1.28k forks source link

Improve AND filter operator #9839

Open Jackie-Jiang opened 1 year ago

Jackie-Jiang commented 1 year ago

Related issues: #9402, #8837

There are some improvements we want to do for AND filter operator:

ashishkf commented 1 year ago

Can you please see if docs can be iterated per the "order by" clause if the order by column also happens to be in the sorted column? This is useful if say, segments are sorted by the timestamp column but the query has "order by timestamp desc". In that case, in addition to limit pushdown applyAnd could passed a flag - ascOrDesc - to use either the forward or reverse bitmap iterator.

Jackie-Jiang commented 1 year ago

@ashishkf Good point! There are actually some discussion about providing reversed doc iterator in #8979. We should consider adding a method to return the reversed iterator, and evaluate applyAnd lazily.