mikemccand / luceneutil

Various utility scripts for running Lucene performance tests
Apache License 2.0
202 stars 113 forks source link

add a few numeric range faceting tasks to wikimedium* #291

Open gsmiller opened 1 month ago

gsmiller commented 1 month ago

I discovered that we already have support for numeric range faceting tasks but don't seem to be using them in any of our task files. Let's add a few tasks to the wikimedium* benchmarks?

I'm not sure if there's a better way to do this. It's kind of confusing that this just shows up as "range" in the benchmark task output. It might be nice to make this a little easier to understand?

gsmiller commented 1 month ago

Here's a snippet of what this looks like when run:

                           TaskQPS baseline      StdDevQPS my_modified_version      StdDev                Pct diff p-value
                          Fuzzy2       58.96      (7.6%)       55.02      (8.4%)   -6.7% ( -21% -   10%) 0.009
                        Wildcard      433.37      (4.4%)      414.00     (15.3%)   -4.5% ( -23% -   15%) 0.210
                           range     1544.90      (9.1%)     1487.45      (9.5%)   -3.7% ( -20% -   16%) 0.206
            BrowseDateSSDVFacets        2.21     (16.3%)        2.17     (17.2%)   -2.1% ( -30% -   37%) 0.698
                      HighPhrase      112.00      (4.5%)      109.87      (4.0%)   -1.9% (  -9% -    6%) 0.159
            MedTermDayTaxoFacets       25.46      (8.1%)       25.04      (8.6%)   -1.7% ( -17% -   16%) 0.524
mikemccand commented 3 weeks ago

Thanks @gsmiller! Maybe we should also separately add to the nightly tasks so nightly benchy can catch regressions? We can do that separately (and I agree we should, also separately, fix the task string range, maybe RangeFacets?).

Why is the range QPS so insanely fast?