guard / listen

The Listen gem listens to file modifications and notifies you about the changes.
https://rubygems.org/gems/listen
MIT License
1.92k stars 246 forks source link

Usage of inotify leaks file descriptors #353

Closed jfinkhaeuser closed 8 years ago

jfinkhaeuser commented 8 years ago

Specifically, the @worker's close method must be called to avoid file descriptor leaks. This never happens in lib/listen/adapter/linux.rb.

For processes that create listen instances as they need them (and then destroy them afterwards), that means a steady stream of leaking file descriptors, until the machine runs out of them.

e2 commented 8 years ago

For processes that create listen instances as they need them (and then destroy them afterwards)

You mean using Listen.stop ?

Currently, this is a no-op on the backend: https://github.com/guard/listen/blob/master/lib/listen/backend.rb#L30

So it probably just needs to be implemented.

No one has ever requested this before, though. Are you running and stopping Listen instances multiple times?

jfinkhaeuser commented 8 years ago

Yes, to both questions :)

Long-running process, gets work packages, during work package execution listens for certain FS changes, and the listen pattern is always different.

I'll grant you that for any other type of application you likely won't notice this :)

FWIW, you can reproduce the issue with rb-notify quite easily (on Linux; ymmv on other systems):

#!/usr/bin/env ruby

require 'rb-inotify'

# Current process' PID
pid = Process.pid

# Number of open files for this process.
puts "lsof [start]: #{`lsof -p #{pid} 2>/dev/null | wc -l`}"

# Do some notification stuff.
notifier = INotify::Notifier.new
notifier.watch("./test.rb", :modify) {puts "test.rb was modified!"}

puts "lsof [watching]: #{`lsof -p #{pid} 2>/dev/null | wc -l`}"

notifier.process
puts "lsof [after process]: #{`lsof -p #{pid} 2>/dev/null | wc -l`}"

# Now stop the notifier
notifier.stop
puts "lsof [stopped]: #{`lsof -p #{pid} 2>/dev/null | wc -l`}"

# Closing
notifier.close
puts "lsof [closed]: #{`lsof -p #{pid} 2>/dev/null | wc -l`}"

The script watches for itself (test.rb), so either name your file accordingly or do some other stuff.

Since rb-inotify's INotify::Notifier#close doesn't get called by Listen... you leak (at least) one FD.

e2 commented 8 years ago

Cool, because that could be turned into an integration test.

OSX is also tested on Travis - AFAIR rb-fsevent also uses a descriptor.

I'll take a stab once I find some time (no promises).