search

h2oai / db-benchmark

reproducible benchmark of database-like ops
https://h2oai.github.io/db-benchmark
Mozilla Public License 2.0
327 stars 89 forks source link

measure memory usage #9

Open jangorecki opened 6 years ago

jangorecki commented 6 years ago

One of tools is currently failing on 1e9 due to lack of memory, 126GB is not enough to aggregate 50GB csv. This requires use bigger machine to run 1e9 grouping benchmark. Still memory usage should be investigated and probably added to report.

jangorecki commented 5 years ago

R's solution to call system("ps -o RSS") happens to be sub-optimal in terms of impact on benchmarks timings. It has been turned off as of now. We need to find reliable way to measure memory.