apache / pinot

Apache Pinot - A realtime distributed OLAP datastore
https://pinot.apache.org/
Apache License 2.0
5.2k stars 1.21k forks source link

[feature] Support funnelMaxStep function #13176

Closed xiangfu0 closed 1 week ago

xiangfu0 commented 2 weeks ago

Support funnelMaxStep function to calculate max funnel steps for a giving sliding window. The syntax is:

FunnelMaxStep(timestampExpression, windowSize, ARRAY[stepExpression, ..], [mode, [mode, ... ]])

Sample query:

SELECT 
    userId, 
    FunnelMaxStep(
        timestampCol, 
        '1000', 
        ARRAY[ 
            url = '/product/search',
            url = '/cart/add',
            url = '/checkout/start',
            url = '/checkout/confirmation
        ],
        'strict_order' ) AS maxSteps  
FROM myTable 
GROUP BY userId 
ORDER BY userId

Sample response:

{
    "resultTable": {
        "dataSchema": {
            "columnNames": [
                "userId",
                "maxSteps"
            ],
            "columnDataTypes": [
                "STRING",
                "LONG"
            ]
        },
        "rows": [
            [
                "user0",
                4
            ],
            [
                "user1",
                3
            ],
            [
                "user2",
                3
            ],
            [
                "user3",
                1
            ]
        ]
    }
}
codecov-commenter commented 2 weeks ago

Codecov Report

Attention: Patch coverage is 28.74494% with 176 lines in your changes are missing coverage. Please review.

Project coverage is 62.15%. Comparing base (59551e4) to head (1177ed4). Report is 485 commits behind head on master.

Files Patch % Lines
...ction/funnel/FunnelMaxStepAggregationFunction.java 0.00% 150 Missing :warning:
...y/aggregation/function/funnel/FunnelStepEvent.java 53.65% 14 Missing and 5 partials :warning:
...org/apache/pinot/core/common/ObjectSerDeUtils.java 88.67% 2 Missing and 4 partials :warning:
...gregation/function/AggregationFunctionFactory.java 0.00% 1 Missing :warning:
Additional details and impacted files ```diff @@ Coverage Diff @@ ## master #13176 +/- ## ============================================ + Coverage 61.75% 62.15% +0.39% + Complexity 207 198 -9 ============================================ Files 2436 2533 +97 Lines 133233 138787 +5554 Branches 20636 21496 +860 ============================================ + Hits 82274 86258 +3984 - Misses 44911 46079 +1168 - Partials 6048 6450 +402 ``` | [Flag](https://app.codecov.io/gh/apache/pinot/pull/13176/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | | |---|---|---| | [custom-integration1](https://app.codecov.io/gh/apache/pinot/pull/13176/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `<0.01% <0.00%> (-0.01%)` | :arrow_down: | | [integration](https://app.codecov.io/gh/apache/pinot/pull/13176/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `<0.01% <0.00%> (-0.01%)` | :arrow_down: | | [integration1](https://app.codecov.io/gh/apache/pinot/pull/13176/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `<0.01% <0.00%> (-0.01%)` | :arrow_down: | | [integration2](https://app.codecov.io/gh/apache/pinot/pull/13176/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `0.00% <0.00%> (ø)` | | | [java-11](https://app.codecov.io/gh/apache/pinot/pull/13176/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `62.12% <28.74%> (+0.41%)` | :arrow_up: | | [java-21](https://app.codecov.io/gh/apache/pinot/pull/13176/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `62.03% <28.74%> (+0.41%)` | :arrow_up: | | [skip-bytebuffers-false](https://app.codecov.io/gh/apache/pinot/pull/13176/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `62.14% <28.74%> (+0.39%)` | :arrow_up: | | [skip-bytebuffers-true](https://app.codecov.io/gh/apache/pinot/pull/13176/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `62.01% <28.74%> (+34.28%)` | :arrow_up: | | [temurin](https://app.codecov.io/gh/apache/pinot/pull/13176/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `62.15% <28.74%> (+0.39%)` | :arrow_up: | | [unittests](https://app.codecov.io/gh/apache/pinot/pull/13176/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `62.14% <28.74%> (+0.39%)` | :arrow_up: | | [unittests1](https://app.codecov.io/gh/apache/pinot/pull/13176/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `46.62% <28.74%> (-0.27%)` | :arrow_down: | | [unittests2](https://app.codecov.io/gh/apache/pinot/pull/13176/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `27.86% <0.00%> (+0.13%)` | :arrow_up: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

xiangfu0 commented 1 week ago

I cannot find the ser/de support for PriorityQueue<FunnelStepEvent>

Added Serde for PriorityQueue<FunnelStepEvent>