search

let4be / crusty

Broad Web Crawler
GNU General Public License v3.0
83 stars 3 forks source link

Investigate possible deadlock/hanging of clickhouse writer #29

Closed let4be closed 3 years ago

let4be commented 3 years ago

Sometimes when testing on aws c5.metal I see that writers "hang" which leads to pile of unprocessed messages in buffers(particularly metrics_task as the heaviest clickhouse hitter)

let4be commented 3 years ago

Most likely related to sysctl settings as it appears only during high load(60k+ concurrent tcp connections) Today I almost pinned it with(net.ipv4.tcp_mem = 383865 1535460 2303190) but failed to reproduce, needs to more debugging.