apache / pinot

Apache Pinot - A realtime distributed OLAP datastore
https://pinot.apache.org/
Apache License 2.0
5.53k stars 1.29k forks source link

Realtime segment size threshold metrics #14485

Closed mcvsubbu closed 6 days ago

mcvsubbu commented 1 week ago

Added a couple of metrics for observing the actual segment size and the predicted number of rows we indicate to the servers. The metrics should help us improve the prediction algorithm for new use cases.

codecov-commenter commented 1 week ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 63.77%. Comparing base (59551e4) to head (9af62cb). Report is 1359 commits behind head on master.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## master #14485 +/- ## ============================================ + Coverage 61.75% 63.77% +2.02% - Complexity 207 1566 +1359 ============================================ Files 2436 2667 +231 Lines 133233 146449 +13216 Branches 20636 22467 +1831 ============================================ + Hits 82274 93397 +11123 - Misses 44911 46136 +1225 - Partials 6048 6916 +868 ``` | [Flag](https://app.codecov.io/gh/apache/pinot/pull/14485/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | | |---|---|---| | [custom-integration1](https://app.codecov.io/gh/apache/pinot/pull/14485/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `100.00% <ø> (+99.99%)` | :arrow_up: | | [integration](https://app.codecov.io/gh/apache/pinot/pull/14485/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `100.00% <ø> (+99.99%)` | :arrow_up: | | [integration1](https://app.codecov.io/gh/apache/pinot/pull/14485/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `100.00% <ø> (+99.99%)` | :arrow_up: | | [integration2](https://app.codecov.io/gh/apache/pinot/pull/14485/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `0.00% <ø> (ø)` | | | [java-11](https://app.codecov.io/gh/apache/pinot/pull/14485/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `63.72% <100.00%> (+2.01%)` | :arrow_up: | | [java-21](https://app.codecov.io/gh/apache/pinot/pull/14485/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `63.67% <100.00%> (+2.04%)` | :arrow_up: | | [skip-bytebuffers-false](https://app.codecov.io/gh/apache/pinot/pull/14485/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `63.76% <100.00%> (+2.02%)` | :arrow_up: | | [skip-bytebuffers-true](https://app.codecov.io/gh/apache/pinot/pull/14485/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `63.62% <100.00%> (+35.89%)` | :arrow_up: | | [temurin](https://app.codecov.io/gh/apache/pinot/pull/14485/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `63.77% <100.00%> (+2.02%)` | :arrow_up: | | [unittests](https://app.codecov.io/gh/apache/pinot/pull/14485/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `63.77% <100.00%> (+2.02%)` | :arrow_up: | | [unittests1](https://app.codecov.io/gh/apache/pinot/pull/14485/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `55.57% <100.00%> (+8.68%)` | :arrow_up: | | [unittests2](https://app.codecov.io/gh/apache/pinot/pull/14485/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `34.06% <100.00%> (+6.33%)` | :arrow_up: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


🚨 Try these New Features: