voltrondata-labs / arrowbench

R package for benchmarking
Other
13 stars 9 forks source link

Capture and save setup time? #44

Open jonkeane opened 2 years ago

jonkeane commented 2 years ago

We recently ran into an issue where we started using altrep in the setup which meant that our benchmark paid for the cost of array->R when it was running R->array. We should measure and keep this setup time around if possible since having that would have made obvious what was going on (e.g. the setup was suddenly faster, but the benchmark itself was slower)