search

mongodb / mongo-perf

performance tools for mongodb
351 stars 140 forks source link

PERF-3365 Extend index scan microbenchmarks #105

Closed leozzx closed 2 years ago

leozzx commented 2 years ago

evergreen perf patch (base) evergreen perf patch (with https://github.com/10gen/mongo/pull/7252)

variant test thread_level measurement value value_base percent_base_diff
SBE Queries.RangeQuery_MultipleIndexes_LowSelectivityMatch_LL 4 ops_per_sec 14.57 13.72 6.18%
SBE Queries.RangeQuery_MultipleIndexes_LowSelectivityMatch_LL 1 ops_per_sec 3.82 3.66 4.29%
Classic Queries.RangeQuery_MultipleIndexes_LowSelectivityMatch_LL 4 ops_per_sec 16.84 14.97 12.47%
Classic Queries.RangeQuery_MultipleIndexes_LowSelectivityMatch_LL 1 ops_per_sec 4.41 3.96 11.32%
             
SBE Queries.RangeQuery_MultipleIndexes_LowSelectivityMatch_LS 4 ops_per_sec 14.58 13.92 4.71%
SBE Queries.RangeQuery_MultipleIndexes_LowSelectivityMatch_LS 1 ops_per_sec 3.82 3.66 4.20%
Classic Queries.RangeQuery_MultipleIndexes_LowSelectivityMatch_LS 4 ops_per_sec 16.77 14.87 12.76%
Classic Queries.RangeQuery_MultipleIndexes_LowSelectivityMatch_LS 1 ops_per_sec 4.44 3.95 12.27%
             
SBE Queries.RangeQuery_SingleIndex_GenericPlan_LS 4 ops_per_sec 11.65 10.89 6.97%
SBE Queries.RangeQuery_SingleIndex_GenericPlan_LS 1 ops_per_sec 3.14 2.97 5.41%
Classic Queries.RangeQuery_SingleIndex_GenericPlan_LS 4 ops_per_sec 13.29 11.98 10.97%
Classic Queries.RangeQuery_SingleIndex_GenericPlan_LS 1 ops_per_sec 3.61 3.30 9.54%
             
SBE Queries.RangeQuery_SingleIndex_GenericPlan_ThreeFields_LS 4 ops_per_sec 10.41 7.70 35.25%
SBE Queries.RangeQuery_SingleIndex_GenericPlan_ThreeFields_LS 1 ops_per_sec 2.74 2.08 32.02%
Classic Queries.RangeQuery_SingleIndex_GenericPlan_ThreeFields_LS 4 ops_per_sec 11.26 10.73 4.86%
Classic Queries.RangeQuery_SingleIndex_GenericPlan_ThreeFields_LS 1 ops_per_sec 2.93 2.71 8.32%
             
SBE Queries.RangeQuery_SingleIndex_GenericPlan_TwoFields_LS 4 ops_per_sec 9.73 8.15 19.39%
SBE Queries.RangeQuery_SingleIndex_GenericPlan_TwoFields_LS 1 ops_per_sec 2.59 2.17 19.40%
Classic Queries.RangeQuery_SingleIndex_GenericPlan_TwoFields_LS 4 ops_per_sec 10.66 9.87 7.98%
Classic Queries.RangeQuery_SingleIndex_GenericPlan_TwoFields_LS 1 ops_per_sec 2.86 2.64 8.30%
             
SBE Queries.RangeQuery_SingleIndex_GenericPlan_TwoFields_Range_LS 4 ops_per_sec 16.76 9.79 71.14%
SBE Queries.RangeQuery_SingleIndex_GenericPlan_TwoFields_Range_LS 1 ops_per_sec 4.30 2.64 62.89%
Classic Queries.RangeQuery_SingleIndex_GenericPlan_TwoFields_Range_LS 4 ops_per_sec 17.23 15.76 9.34%
Classic Queries.RangeQuery_SingleIndex_GenericPlan_TwoFields_Range_LS 1 ops_per_sec 4.36 4.03 8.08%
             
SBE Queries.RangeQuery_SingleIndex_MultiFields_SingleIntervals_LS 4 ops_per_sec 21.34 22.49 -5.14%
SBE Queries.RangeQuery_SingleIndex_MultiFields_SingleIntervals_LS 1 ops_per_sec 5.67 5.94 -4.52%
Classic Queries.RangeQuery_SingleIndex_MultiFields_SingleIntervals_LS 4 ops_per_sec 20.39 18.40 10.84%
Classic Queries.RangeQuery_SingleIndex_MultiFields_SingleIntervals_LS 1 ops_per_sec 5.47 4.91 11.35%
             
SBE Queries.RangeQuery_SingleIndex_SimpleRange_LS 4 ops_per_sec 13.82 14.18 -2.55%
SBE Queries.RangeQuery_SingleIndex_SimpleRange_LS 1 ops_per_sec 3.70 3.84 -3.58%
Classic Queries.RangeQuery_SingleIndex_SimpleRange_LS 4 ops_per_sec 14.06 12.84 9.54%
Classic Queries.RangeQuery_SingleIndex_SimpleRange_LS 1 ops_per_sec 3.83 3.39 12.81%
             
SBE Queries.RangeQuery_SingleIndex_SimpleRange_LowSelectivityMatch_LS 4 ops_per_sec 1,761.92 1,774.67 -0.72%
SBE Queries.RangeQuery_SingleIndex_SimpleRange_LowSelectivityMatch_LS 1 ops_per_sec 444.59 447.13 -0.57%
Classic Queries.RangeQuery_SingleIndex_SimpleRange_LowSelectivityMatch_LS 4 ops_per_sec 1,793.84 1,567.02 14.47%
Classic Queries.RangeQuery_SingleIndex_SimpleRange_LowSelectivityMatch_LS 1 ops_per_sec 458.40 396.62 15.58%
leozzx commented 2 years ago

Do I understand correctly that this won't generate "Aggregation" or "IndentityView" variations of this benchmark. It didn't look like it would, which I think is a good thing, but I wanted to confirm.

@dstorch yes, there isn't "Aggregation" or "IndentityView" generated.

leozzx commented 2 years ago

Should I also add these tests into PerformanceReports/tags.yml? @dstorch

mtandrei commented 2 years ago

I would say, it's fine to wait for the tests to burn in for a bit before tagging them. If you would like to add them now, that's fine too, but there's no rush.