dwbutler / logstash-logger

Ruby logger that writes logstash events
MIT License
456 stars 118 forks source link

Asynchronous logging doesn't work properly for TCP and UDP #119

Open warnickr opened 7 years ago

warnickr commented 7 years ago

When synchronous logging is disabled (sync: false), nothing gets logged to TCP and UDP endpoints. I think this is because the socket is never flushed after being written to. If I modify LogStashLogger::Device::Connectable#flush as follows then it works.

def flush(*args)
  if args.empty?
    buffer_flush
  else
    messages, group = *args
    write_batch(messages, group)
  end

  @io.flush
end
warnickr commented 7 years ago

I created a pull request that addresses the issue.

dwbutler commented 7 years ago

There is a detailed explanation of this behavior in the troubleshooting section of the readme here. If you require sockets to be flushed immediately, it's up to you to set the sync: true setting. Most applications should keep the default setting of sync: false and allow Ruby / the OS to decide when to flush its internal buffers.

warnickr commented 7 years ago

That makes sense. In my experience, the unpredictable and often extended delay renders sync:false unusable. Wouldn't it be useful to allow asynchronous logging that batches log writes while still maintaining some level or predictability?

dwbutler commented 7 years ago

sync: false is a good default for applications that have a high log volume, because it improves performance. I can understand why you may want more predictable logging behavior in a low volume application.

There are several possibilities.