search

trinodb / benchto

Framework for running macro benchmarks in a clustered environment
Apache License 2.0
28 stars 30 forks source link

Add query-runs attribute to repeat all queries in sequence #55

Closed nineinchnick closed 1 year ago

nineinchnick commented 1 year ago

Repeating the same query multiple times can yield inaccurate benchmark results because the JVM can optimize for it. This is not reflecting real-life usage. By allowing to repeat all queries in a sequence, benchmark results should better reflect the actual performance.

nineinchnick commented 1 year ago

@sopel39 PTAL

radek-kondziolka commented 1 year ago

Could add in the description a motivation to do it?

nineinchnick commented 1 year ago

I just realized I added the extra loop too deep, I was confused by a benchmark having getQueries() method where we usually have a single query there and multiple benchmarks.

nineinchnick commented 1 year ago

Replaced with #56