search

h2oai / db-benchmark

reproducible benchmark of database-like ops
https://h2oai.github.io/db-benchmark
Mozilla Public License 2.0
322 stars 85 forks source link

clickhouse segfault causes benchmark launcher script to fail #132

Closed jangorecki closed 4 years ago

jangorecki commented 4 years ago

When segmentation fault happens in Clickhouse server, #133, it is not releasing memory well enough. This is causing parent process, benchmark scripts launcher, to fail with the following error.

Error in system(sprintf("wc -l %s | awk '{print $1}'", x), intern = TRUE) : 
  cannot popen 'wc -l out/run_clickhouse_groupby_G2_1e9_1e2_0_1.err | awk '{print $1}'', probable reason 'Cannot allocate memory'
Calls: launch -> log_run -> as.data.table -> wcl -> system
Execution halted
Warning message:
system call failed: Cannot allocate memory

One possibility to address this issue is to reduce amount of memory set for CH to use. It is now currently using 120GB.

jangorecki commented 4 years ago

As a result of f8ad138 clickhouse is on average 7% slower than before. Because it actually solved also #112 and #133 I believe it is a proper configuration for running CH server.