search

logstash-plugins / logstash-output-http

Apache License 2.0
35 stars 81 forks source link

Better documentation/defaults for high throughput scenarios #117

Open robbavey opened 3 years ago

robbavey commented 3 years ago

When using this plugin in certain scenarios, such as HTTP based logstash->logstash communication, or when sending to endpoints with large amounts of latency, using defaults can have a significantly detrimental impact on performance.

We should improve the documentation to include recommendations (http_compression=>true, format=>json_batch, increased batch sizes) to enable improved performance for these scenarios.

Other potential improvements include setting format=>json_batch as default in a future major version, or logging a warning that single http events are being sent via this output when users specify a large batch size in their logstash configuration.

Catwoolfii commented 1 year ago

@robbavey, hi! How is the size of json_batch determined?