search

crate / crate-benchmarks

A collection of CrateDB benchmarks.
8 stars 4 forks source link

Rework outer-join benchmarks to stabilize them #121

Closed mfussenegger closed 6 years ago

mfussenegger commented 6 years ago

The outer-join benchmarks runtime was highly influenced by the "insert order" that isn't deterministic. This was the case because the result-set is quite large (5e7), but the HTTP soft limit was applied (1e4) - this means that if the input-order of the data being fed into the nested-loop operation happened to result in early matches the runtime was a lot faster than if the opposite was the case.

Due to that we often saw large gaps in the runtime. (E.g. from usual 40ms to 150ms; in some cases even larger)

This commit introduces a more sensible data set for joins. (Although rather small; which is why the old set is still kept, but the queries are adapted)

mfussenegger commented 6 years ago

Pushed a fixup with a greatly reduced data set - the result is still quite large (50k rows)