search

Tony-X / search-benchmark-game

https://github.com/quickwit-oss/search-benchmark-game
https://tantivy-search.github.io/bench/
MIT License
10 stars 6 forks source link

Benchmark should produce a simple plain text summary #42

Open mikemccand opened 1 year ago

mikemccand commented 1 year ago

It's nice that we get a results.json with all the juicy details of each benchmark, but, for sharing purposes (e.g. the Graviton results I just attached to #36) it'd be nicer/easier to copy/paste a text summary so that we could see at a glance what the results look like.

We could just make a new tool that parses the JSON and renders to flat text, in addition to the make bench that renders to an interactive Web UI.

mikemccand commented 1 year ago

I'll try to make a first cut at this.

mikemccand commented 1 year ago

OK I pushed a first cut. It produces many lines of output looking like this:

TOP_10_COUNT HighTerm
  145.3% from        206.1 ms  84.0 ms count=7834704 (WARNING: tantivy_count=7834746)
  143.5% were         88.1 ms  36.2 ms count=3319889 (WARNING: tantivy_count=3319906)
  143.5% has         110.8 ms  45.5 ms count=4180955 (WARNING: tantivy_count=4180986)
  143.4% which       118.0 ms  48.5 ms count=4469484 (WARNING: tantivy_count=4469528)
  143.2% have        101.0 ms  41.5 ms count=3821616 (WARNING: tantivy_count=3821633)
  142.9% also         97.7 ms  40.2 ms count=3702272 (WARNING: tantivy_count=3702292)
  142.4% his         106.9 ms  44.1 ms count=4051071 (WARNING: tantivy_count=4051077)
  142.4% first        80.8 ms  33.3 ms count=3044659 (WARNING: tantivy_count=3044671)
  141.6% been         76.6 ms  31.7 ms count=2892317 (WARNING: tantivy_count=2892324)
  141.6% had          72.8 ms  30.1 ms count=2733137 (WARNING: tantivy_count=2733149)
  141.5% one          82.6 ms  34.2 ms count=3122527 (WARNING: tantivy_count=3122538)
  141.3% he           86.8 ms  36.0 ms count=3278298 (WARNING: tantivy_count=3278314)
  140.9% who          74.6 ms  31.0 ms count=2811995 (WARNING: tantivy_count=2812021)
  140.4% its          67.9 ms  28.3 ms count=2548474 (WARNING: tantivy_count=2548493)
  140.3% other        67.9 ms  28.2 ms count=2557746 (WARNING: tantivy_count=2557757)
  139.9% name         62.7 ms  26.1 ms count=2348632 (WARNING: tantivy_count=2348644)

First column is what % slower Lucene is, then Lucene's time, then Tantivy's, then the hit count, and also Tantivy's different count if different.

mikemccand commented 1 year ago

Also, separately, I'm quite happy I got this issue number (42).