kdgregory / log4j-aws-appenders

Appenders for Log4J 1.2.x, Log4J 2.x, and Logback that write to AWS destinations.
Apache License 2.0
67 stars 18 forks source link

Performance degrade on peak loads, heap increase #193

Closed RajeshCoast closed 1 year ago

RajeshCoast commented 1 year ago

Hi Kdgregory, We see the heap size increase using this appender. Although we are using it with Async still it cause increase response time.

Can you please let us know if we are missing any setting to reduce the heap size caused by appender.

Regards

kdgregory commented 1 year ago

@RajeshCoast

The discardThreshold configuration parameter controls how many messages will be stored in the appender's internal buffer. The default value is 10,000, which should not result in excessive heap usage unless your messages are extremely large.

I would instead look at configuration of whatever async appender that you've put in front of this, because it has its own buffer. Be aware also that there's no need for an async appender with this library: it has its own buffer for unsent log messages, so the async appender will be double-buffering.

If you are generating a large number of logging messages, you should also decrease the batchDelay parameter, to send batches more frequently. The default is 2000 (2 seconds); I would reduce to 500. Beware, however, that if you decrease it too much you'll likely hit throttling limits.

Finally, if you're generating log messages too frequently for the appender to keep up, you should be aware of the discardAction parameter. By default, the oldest messages will be discarded, but you can also configure it to discard the newest.