search

reidmorrison / semantic_logger

Semantic Logger is a feature rich logging framework, and replacement for existing Ruby & Rails loggers.
https://logger.rocketjob.io/
Apache License 2.0
851 stars 117 forks source link

HTTP appender batch: true #278

Open intermittentnrg opened 1 month ago

intermittentnrg commented 1 month ago

The http appender could support batch to send multiple log entries to filebeat http endpoint.

The body must be either an object or an array of objects. Any other data types will result in an HTTP 400 (Bad Request) response. For arrays, one document is created for each object in the array.

Seems simple enough? Should probably be off by default and enabled with batch: true.

Would be nice to have.

reidmorrison commented 1 week ago

Should not be too difficult to implement, the steps for someone that wants to submit a PR would be as follows:

  1. Add batch method to HTTP Appender that like log takes an array of messages. For an example, see https://github.com/reidmorrison/semantic_logger/blob/f7d54bf21b6900f334155882ffd6ab287ffa58f5/lib/semantic_logger/appender/elasticsearch.rb#L167
  2. Add batch method to JSON Formatter to output the array of log entries as a single JSON entry. Similar to the call method, except it takes an array of log entries. For an example, see: https://github.com/reidmorrison/semantic_logger/blob/f7d54bf21b6900f334155882ffd6ab287ffa58f5/lib/semantic_logger/formatters/signalfx.rb#L111

As mentioned above, all that should be needed next is to set batch: true when creating the appender instance, which should automatically create an appender thread and send the log messages in batches.