Open gsmiller opened 1 month ago
Here's a snippet of what this looks like when run:
TaskQPS baseline StdDevQPS my_modified_version StdDev Pct diff p-value
Fuzzy2 58.96 (7.6%) 55.02 (8.4%) -6.7% ( -21% - 10%) 0.009
Wildcard 433.37 (4.4%) 414.00 (15.3%) -4.5% ( -23% - 15%) 0.210
range 1544.90 (9.1%) 1487.45 (9.5%) -3.7% ( -20% - 16%) 0.206
BrowseDateSSDVFacets 2.21 (16.3%) 2.17 (17.2%) -2.1% ( -30% - 37%) 0.698
HighPhrase 112.00 (4.5%) 109.87 (4.0%) -1.9% ( -9% - 6%) 0.159
MedTermDayTaxoFacets 25.46 (8.1%) 25.04 (8.6%) -1.7% ( -17% - 16%) 0.524
Thanks @gsmiller! Maybe we should also separately add to the nightly tasks so nightly benchy can catch regressions? We can do that separately (and I agree we should, also separately, fix the task string range
, maybe RangeFacets
?).
Why is the range
QPS so insanely fast?
I discovered that we already have support for numeric range faceting tasks but don't seem to be using them in any of our task files. Let's add a few tasks to the wikimedium* benchmarks?
I'm not sure if there's a better way to do this. It's kind of confusing that this just shows up as "range" in the benchmark task output. It might be nice to make this a little easier to understand?