UDF support in filter clause

[kishoreg]

Support for expressions in the lhs only. func(... ) = '10'. func(...) = func(...) is not yet supported but quite easy to add that.

select count(*) from baseballStats where numberOfGames = 100
"aggregationResults": [
    {
        "function": "count_star",
        "value": "256"
    }]

select count(*) from baseballStats where DIV(numberOfGames,10) = 10
"aggregationResults": [
    {
        "function": "count_star",
        "value": "256"
    }

Things to do as follow up

• Support for expressions on MultiValue columns
• Support for comparisons between two expressions.

[codecov-io]

Codecov Report

Merging #28 into develop will decrease coverage by 0.23%. The diff coverage is 66.66%.

@@              Coverage Diff              @@
##             develop      #28      +/-   ##
=============================================
- Coverage      65.14%   64.91%   -0.24%     
  Complexity         4        4              
=============================================
  Files           1049     1051       +2     
  Lines          54420    55511    +1091     
  Branches        7761     8036     +275     
=============================================
+ Hits           35453    36033     +580     
- Misses         16403    16879     +476     
- Partials        2564     2599      +35

Impacted Files	Coverage Δ	Complexity Δ
...r/dociditerators/RangelessBitmapDocIdIterator.java	72.22% <ø> (ø)	0 <0> (ø)	:arrow_down:
...che/pinot/core/operator/blocks/TransformBlock.java	71.42% <100%> (+2.19%)	0 <0> (ø)	:arrow_down:
...l/parsers/pql2/ast/ComparisonPredicateAstNode.java	43.9% <100%> (-2%)	0 <0> (ø)
...core/startree/operator/StarTreeFilterOperator.java	96.22% <100%> (+0.02%)	0 <0> (ø)	:arrow_down:
...r/filter/predicate/PredicateEvaluatorProvider.java	65.21% <100%> (-2.79%)	0 <0> (ø)
...pache/pinot/common/utils/request/RequestUtils.java	89.74% <100%> (-6.84%)	0 <0> (ø)
...ava/org/apache/pinot/core/plan/FilterPlanNode.java	90.9% <100%> (+1.16%)	0 <0> (ø)	:arrow_down:
...he/pinot/common/utils/request/FilterQueryTree.java	100% <100%> (ø)	0 <0> (ø)	:arrow_down:
...e/operator/dociditerators/SVScanDocIdIterator.java	67.85% <50%> (ø)	0 <0> (ø)	:arrow_down:
...inot/core/operator/filter/FilterOperatorUtils.java	87.87% <50%> (+0.37%)	0 <0> (ø)	:arrow_down:
... and 60 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 30fe389...d80994c. Read the comment docs.

[mayankshriv]

BTW, curious if/how use of indices could work in presence of filters. Or will this always be a scan based solution, and hence would have performance penalty (or rely in presence of other filters to reduce number of docs to scan)?

[xiangfu0]

ExpressionSVBlockDocIdIterator is scan based always, just it could work with indices with isMatch method been implemented.

[kishoreg]

BTW, curious if/how use of indices could work in presence of filters. Or will this always be a scan based solution, and hence would have performance penalty (or rely in presence of other filters to reduce number of docs to scan)?

For now, its always scan-based + expression and will be evaluated after all other predicates (index based, sorted index based, scan-based).

We will have to enhance TransformFunction interface to support using indices. But for now, we can explicitly add additional predicates to the query

[kishoreg]

Not sure why the build is failing. The failed test runs successfully on my laptop and moreover, the changes in this PR in unrelated to the test case.