Support watching files with inotify

[GoogleCodeExporter]

logstash should utilize inotify to monitor files on Linux to improve 
efficiency. 

Original issue reported on code.google.com by luke.mac...@gmail.com on 11 Feb 2011 at 9:42

[GoogleCodeExporter]

eventmachine-tail uses inotify currently to watch for file changes.

However, it doesn't use inotify's directory watching for watching globs. Doing 
this will be tricky to do correctly (easy race condition between mkdir + create 
new log file and the actual mkdir notification, etc). Still, it's worth doing.

Additionally, there's an open bug to watch files on non-local filesystems 
(samba or nfs mounts) which don't support inotify anyway, so there's gonna be a 
mix of inotify+polling here anyway.

Original comment by jls.semi...@gmail.com on 11 Feb 2011 at 9:54

• Added labels: Milestone-Release1.1

[GoogleCodeExporter]

Also, it's worth mentioning (on the efficiency note) taht a stock ruby 1.8.x 
will wake up every 10ms and make a boatload of syscalls as part of ruby's 
default thread management.

REE makes this much better; so if you care about wasted effort by logstash, 
definitely use REE instead of MRI ruby (REE == ruby enterprise edition, 
http://www.rubyenterpriseedition.com/)

Original comment by jls.semi...@gmail.com on 14 Feb 2011 at 9:27

[GoogleCodeExporter]

I pushed a new ruby lib 'filewatch' that will help us watch files, directories 
(and globs), etc, on linux, freebsd, and other systems.

Work in progress; only supports inotify for now.

Original comment by jls.semi...@gmail.com on 19 Mar 2011 at 7:37