search

driskell / log-courier

The Log Courier Suite is a set of lightweight tools created to ship and process log files speedily and securely, with low resource usage, to Elasticsearch or Logstash instances.
Other
419 stars 107 forks source link

log-courier vs beats and ssl_verify_mode #349

Closed tomryanx closed 7 years ago

tomryanx commented 8 years ago

In preparation for upgrading to 5.0, I took a look at whether the 'killer feature' of log-courier is now available in beats - namely, strict validation of both server and client certs. It appears that it is https://www.elastic.co/guide/en/logstash/current/plugins-inputs-beats.html#plugins-inputs-beats-ssl_verify_mode - in particular force_peer mode.

I've been nothing but pleased with log-courier to date (thanks!), but I'm now wondering whether it wouldn't make sense to switch back to the 'standard' option of beats... are there other reasons to stick with log-courier?

driskell commented 8 years ago

It's entirely down to your own requirements. In all honestly beats has grown so well in the last 6-12 months most of the reasons for using Log Courier are now mute.

For me anyway, it was mainly for performance, reliability and resource efficiency. Performance and reliability wise, with the latests beats plugin pure-java event loop that's likely to be greatly improved and probably resolved. Resource efficiency is related to that but is specifically about preventing memory exhaustion which was so easy with previous iterations, but may be much better with the pure-java one as it's using pipelining.

I do still plan to update things slowly as a personal project to keep me occupied but those updates will come less often than previously. So I shouldn't worry at this stage that Log Courier will die off. Just less new features. fact-courier is something I built to collect Munin statistics for shipping and I'm still thinking about the possibility to putting it into a beat, either alongside or instead of a log-courier thing, as it's really useful to easily grab statistics from scripts and when it comes to Munin there are literally thousands of sources.

driskell commented 7 years ago

I just pushed 1.9.1 of the plugin to ease people's migrations to 5.x. I'm struggling to maintain on my own with my limited time though so I'd say it's best to look at migrating. Though if people want to stay with log-courier I'm happy to take pull requests and add maintainers. Thanks!