Graylog2 / gelfclient

GELF client library for Java based on Netty 4
https://www.graylog.org/
Apache License 2.0
38 stars 19 forks source link

Limit the number of allow inflight network writeAndFlush operations #24

Closed kroepke closed 8 years ago

kroepke commented 8 years ago

Without limiting netty would happily create more and more internal buffers, using large amounts of memory. That memory pressure could lead to OOM situations or very high GC times when faced with either fast producers or slow consumers (or worst case both).

The number of maximum inflight sends is configurable and by default limited to 512 operations.

fixes #22

coveralls commented 8 years ago

Coverage Status

Changes Unknown when pulling eb1c6718e1b9cf45552610e9237fe415fda66ef1 on fix-oom-issue22 into \ on master**.

joschi commented 8 years ago

@kroepke What will happen if there are more operations queueing than maxInflightSends allows?

kroepke commented 8 years ago

@joschi It will eventually block the sender code if it used the .send() method. If the sender code doesn't care it should use trySend(). The GELF sender thread is single-threaded anyway, the limit mostly exists to protect against writing to the Channel without bounds, as it is almost always writable.

kroepke commented 8 years ago

Writing 2 million messages:

After change: screenshot

Before change: screenshot

kroepke commented 8 years ago

The significant impact, other than memory usage, is the GC times: After: screenshot

Before: screenshot

bernd commented 8 years ago

LGTM :+1:

coveralls commented 8 years ago

Coverage Status

Changes Unknown when pulling 048d9e95aecaf99a0934952935c1333a604ff400 on fix-oom-issue22 into \ on master**.