search

armon / bloomd

C network daemon for bloom filters
http://armon.github.io/bloomd
Other
1.24k stars 112 forks source link

Time outs [+question] #53

Closed renatojf closed 1 year ago

renatojf commented 6 years ago

Hi there. Recently we started seeing a lot of timeouts when performing queries against a BIG set (over 200 Million filters). Timeouts for us, means that the query to bloomd is taking longer than 250ms (we even increased it to 500ms and still times out).

Is there any best practices regarding the size of the filters? Is it better to split them into multiple small ones and perform multiple queries? Are there any optimizations we can do regarding the query to it?

armon commented 1 year ago

@renatojf Apologies for the very long delay. My best guess is you exhausted system memory and the OS was paging out to disk. That is the only reasonable guess given the data here. The actual in-memory access times should never be that long, but if random pages were on disk and you were triggering a lot of disk reads, that could explain the timeouts.