search

chronosphereio / calyptia-benchmark-framework

Fluent Bit, Fluentd + other projects benchmark framework
0 stars 0 forks source link

Improve Fluent Bit tail buffer size setup #4

Open edsiper opened 2 years ago

edsiper commented 2 years ago

I found that Stanza and Vector configs are not making the agents perform I/O, it's almost zero, is it disabled ?

stanza_io

edsiper commented 2 years ago

Just running Stanza

no_io

I/O is there but pretty much says the agent is not configured to do buffering. how to enable that ?

edsiper commented 2 years ago

ah, found it. It's all about the buffer sizes used per Agent, in Fluent Bit if we set insane buffer sizes I/O drops to almost zero.

mem_buffers

but there is a side effect on high memory usage. We should try to accommodate all the agents to similar buffer sizes when reading files