Closed floehopper closed 5 years ago
Hmm. The acceptance specs don't seem to be passing on the JRuby builds on Travis CI. 😞
For some reason, I can only get them to pass if I set the OS to OSX.
Hmm. I just read this at the end of the documentation:
Most file system implementations have native support for file change notification. The Watch Service API takes advantage of this support where available. However, when a file system does not support this mechanism, the Watch Service will poll the file system, waiting for events.
Maybe that's what I'm seeing on Mac OSX.
Wow, cool that this got a quick impl so...quickly!
The lag times could be influenced by slow JRuby startup?
Also per-platform support...yes I think one thing we'll want to look into is whether there's a way to query whether it's going to use filesystem events or polling. In the cases where the JDK is using polling, we can explore other options like using FFI to call into MacOS inotify APIs.
I will have a look at this tonight or tomorrow. Thanks @floehopper!
Ok so it looks like only BSD/MacOS do not have support for using a native filesystem eventing system. Windows uses ReadDirectoryAttributes, Linux uses inotify, Solaris uses something similar.
I think the simplest way to detect this would be to get the WatchService instance and then check if it's the polling one. Unfortunately there's no public API for this, but the following kinda-gross hack should work:
ws = ... get watch service
if ws.name =~ /Polling/
# fall back on existing polling impl or try using FFI or whatever
I found a MacOS FSE library for JDK that uses JNA, a library similar to the JNR we use to implement FFI. This code could probably be translated into Ruby FFI pretty easily:
@headius:
The lag times could be influenced by slow JRuby startup?
I'm pretty sure this wasn't the issue I was seeing on my Mac OSX development machine. Running the following script and executing touch foo.txt
from another terminal window generated the following output, suggesting that there's a 5-10 sec lag.
require 'java'
java_import 'java.nio.file.FileSystems'
java_import 'java.nio.file.Paths'
java_import 'java.nio.file.StandardWatchEventKinds'
require 'pathname'
require 'fileutils'
EVENT_KINDS = [
StandardWatchEventKinds::ENTRY_CREATE,
StandardWatchEventKinds::ENTRY_MODIFY,
StandardWatchEventKinds::ENTRY_DELETE
]
directory = Pathname.pwd
watcher = FileSystems.getDefault.newWatchService
path = Paths.get(directory.to_s)
keys = {}
key = path.register(watcher, *EVENT_KINDS)
keys[key] = path
puts 'Monitoring started...'
loop do
key = watcher.take
dir = keys[key]
unless dir.nil?
key.pollEvents.each do |event|
kind = event.kind
next if kind == StandardWatchEventKinds::OVERFLOW
name = event.context
child = dir.resolve(name)
lag = Time.now - File.mtime(child.to_s)
p [child.to_s, lag]
end
end
valid = key.reset
unless valid
keys.delete(key)
break if keys.empty?
end
end
Monitoring started...
["~/Code/floehopper/listen/foo.txt", 5.1431249999999995]
["~/Code/floehopper/listen/foo.txt", 5.853784]
["~/Code/floehopper/listen/foo.txt", 3.5495989999999997]
["~/Code/floehopper/listen/foo.txt", 6.942901]
["~/Code/floehopper/listen/foo.txt", 7.41488]
["~/Code/floehopper/listen/foo.txt", 7.5622180000000006]
["~/Code/floehopper/listen/foo.txt", 8.245715]
["~/Code/floehopper/listen/foo.txt", 6.022797]
["~/Code/floehopper/listen/foo.txt", 7.021637]
Ok so it looks like only BSD/MacOS do not have support for using a native filesystem eventing system.
My development machine is running MacOs, so I guess this explains the results above - presumably in this case it has fallen back to polling...?
This seems to be confirmed by adding the following line to the script above:
puts watcher.class.name # => "Java::SunNioFs::PollingWatchService"
I've also just noticed that the JRuby builds are failing even on master
, so I'm going to revert my changes to the Travis CI config and force-push to this branch.
Doh! The Linux adapter was being picked up in preference to the JRuby adapter. I've moved the JRuby adapter to the beginning of the OPTIMIZED_ADAPTERS
array. Continuing to debug...
I've realised I don't yet fully understand how the gem is architected and so I think I have quite a bit of work to do before this will be ready to merge, so I'm going to close the PR, but continue working on the branch.
This is a quick & hacky attempt at getting the basics of #462 working closely based on the WatchDir example mentioned in the documentation.
I found that I had to bump
LISTEN_TESTS_DEFAULT_LAG
up when running the acceptance specs on my development machine (MacOS v10.13.6):Although I didn't try to minimise the value of
LISTEN_TESTS_DEFAULT_LAG
, I was using ~5 seconds to get the acceptance specs to pass successfully. This doesn't seem very encouraging, but maybe I'm doing something wrong! I'm hoping that pushing this up will yield some better results from Travis CI.To-do (not exhaustive)
callback
passed into#_configure
as intended...?#_process_event
as intended...?Listen::Adapter::Jruby.usable?
implementation is sensible/cc @headius @bratish