voltrondata-labs / benchmarks

Language-independent Continuous Benchmarking (CB) for Apache Arrow
MIT License
10 stars 11 forks source link

benchmarks.json: tweak dataset-serialize invocation #127

Closed jgehrcke closed 1 year ago

jgehrcke commented 1 year ago

Changes:

I hope that this passes CI (i.e., I hope that the automatic check does not require --iterations=3 --drop-caches=true.

jgehrcke commented 1 year ago
E         -     "command": "dataset-serialize ALL --iterations=6 --all=true",
E         ?                                                    ^
E         +     "command": "dataset-serialize ALL --iterations=3 --all=true --drop-caches=true",
E         ?                                                    ^           +++++++++++++++++++
E               "flags": {
E                 "language": "Pyth

I find this rather inflexible, I think depending on the benchmark specifics we should be able to tweak things. What do we think?

I suspect that I can actually tweak these two parameters, elsewhere in code -- looking.

jgehrcke commented 1 year ago

Seems like iterations can be set as class attribute, and it seems like --drop-caches=true is hard-coded: https://github.com/voltrondata-labs/benchmarks/blob/16330feaeaa9d498f0fa08794045270798e118c1/benchmarks/_benchmark.py#L338

coveralls commented 1 year ago

Pull Request Test Coverage Report for Build 3739321263


Totals Coverage Status
Change from base Build 3706037481: 0.007%
Covered Lines: 1488
Relevant Lines: 1706

💛 - Coveralls