search

guard / rb-fsevent

FSEvents API with signals handled (without RubyCocoa)
https://rubygems.org/gems/rb-fsevent
MIT License
245 stars 37 forks source link

Events not firing #10

Closed whatupdave closed 9 years ago

whatupdave commented 13 years ago

I'm trying to pin down exactly what's happening here. The specs don't pass, I ran it in debug mode:

creating Makefile
CFLAGS='-isysroot /Developer/SDKs/MacOSX10.6.sdk -mmacosx-version-min=10.6 -mdynamic-no-pic -std=gnu99 -Os -pipe -Wmissing-prototypes -Wreturn-type -Wmissing-braces -Wparentheses -Wswitch -Wunused-function -Wunused-label -Wunused-parameter -Wunused-variable -Wunused-value -Wuninitialized -Wunknown-pragmas -Wshadow -Wfour-char-constants -Wsign-compare -Wnewline-eof -Wconversion -Wshorten-64-to-32 -Wglobal-constructors -pedantic' /usr/bin/gcc -isysroot /Developer/SDKs/MacOSX10.6.sdk -mmacosx-version-min=10.6 -mdynamic-no-pic -std=gnu99 -dead_strip -framework CoreServices -D DEBUG=true -o '/users/dave/code/temp/rb-fsevent/bin/fsevent_watch' fsevent/fsevent_watch.c
fsevent_watch compiled
.
append_path called for: /users/dave/code/temp/rb-fsevent/spec/fixtures/custom 'path
  resolved path to: /Users/dave/code/temp/rb-fsevent/spec/fixtures/custom 'path

config.sinceWhen    18446744073709551615
config.latency      0.300000
config.flags        00000000
config.paths
  /Users/dave/code/temp/rb-fsevent/spec/fixtures/custom 'path

FSEventStreamRef @ 0x1001085c0:
   allocator = 0x7fff709faee0
   callback = 0x100001522
   context = {0, 0x0, 0x0, 0x0, 0x0}
   numPathsToWatch = 1
   pathsToWatch = 0x7fff709faee0
        pathsToWatch[0] = '/Users/dave/code/temp/rb-fsevent/spec/fixtures/custom 'path'
   latestEventId = -1
   latency = 300000 (microseconds)
   flags = 0x00000000
   runLoop = 0x0
   runLoopMode = 0x0

F
append_path called for: /users/dave/code/temp/rb-fsevent/spec/fixtures/custom 'path
  resolved path to: /Users/dave/code/temp/rb-fsevent/spec/fixtures/custom 'path

config.sinceWhen    18446744073709551615
config.latency      0.300000
config.flags        00000000
config.paths
  /Users/dave/code/temp/rb-fsevent/spec/fixtures/custom 'path

FSEventStreamRef @ 0x1001085c0:
   allocator = 0x7fff709faee0
   callback = 0x100001522
   context = {0, 0x0, 0x0, 0x0, 0x0}
   numPathsToWatch = 1
   pathsToWatch = 0x7fff709faee0
        pathsToWatch[0] = '/Users/dave/code/temp/rb-fsevent/spec/fixtures/custom 'path'
   latestEventId = -1
   latency = 300000 (microseconds)
   flags = 0x00000000
   runLoop = 0x0
   runLoopMode = 0x0

append_path called for: /users/dave/code/temp/rb-fsevent/spec/fixtures
  resolved path to: /Users/dave/code/temp/rb-fsevent/spec/fixtures

config.sinceWhen    18446744073709551615
config.latency      0.500000
config.flags        00000000
config.paths
  /Users/dave/code/temp/rb-fsevent/spec/fixtures

FSEventStreamRef @ 0x1001085e0:
   allocator = 0x7fff709faee0
   callback = 0x100001522
   context = {0, 0x0, 0x0, 0x0, 0x0}
   numPathsToWatch = 1
   pathsToWatch = 0x7fff709faee0
        pathsToWatch[0] = '/Users/dave/code/temp/rb-fsevent/spec/fixtures'
   latestEventId = -1
   latency = 500000 (microseconds)
   flags = 0x00000000
   runLoop = 0x0
   runLoopMode = 0x0

F
append_path called for: /users/dave/code/temp/rb-fsevent/spec/fixtures
  resolved path to: /Users/dave/code/temp/rb-fsevent/spec/fixtures

config.sinceWhen    18446744073709551615
config.latency      0.500000
config.flags        00000000
config.paths
  /Users/dave/code/temp/rb-fsevent/spec/fixtures

FSEventStreamRef @ 0x1001085e0:
   allocator = 0x7fff709faee0
   callback = 0x100001522
   context = {0, 0x0, 0x0, 0x0, 0x0}
   numPathsToWatch = 1
   pathsToWatch = 0x7fff709faee0
        pathsToWatch[0] = '/Users/dave/code/temp/rb-fsevent/spec/fixtures'
   latestEventId = -1
   latency = 500000 (microseconds)
   flags = 0x00000000
   runLoop = 0x0
   runLoopMode = 0x0

append_path called for: /users/dave/code/temp/rb-fsevent/spec/fixtures
  resolved path to: /Users/dave/code/temp/rb-fsevent/spec/fixtures

config.sinceWhen    18446744073709551615
config.latency      0.500000
config.flags        00000000
config.paths
  /Users/dave/code/temp/rb-fsevent/spec/fixtures

FSEventStreamRef @ 0x1001085e0:
   allocator = 0x7fff709faee0
   callback = 0x100001522
   context = {0, 0x0, 0x0, 0x0, 0x0}
   numPathsToWatch = 1
   pathsToWatch = 0x7fff709faee0
        pathsToWatch[0] = '/Users/dave/code/temp/rb-fsevent/spec/fixtures'
   latestEventId = -1
   latency = 500000 (microseconds)
   flags = 0x00000000
   runLoop = 0x0
   runLoopMode = 0x0

F
append_path called for: /users/dave/code/temp/rb-fsevent/spec/fixtures
  resolved path to: /Users/dave/code/temp/rb-fsevent/spec/fixtures

config.sinceWhen    18446744073709551615
config.latency      0.500000
config.flags        00000000
config.paths
  /Users/dave/code/temp/rb-fsevent/spec/fixtures

FSEventStreamRef @ 0x1001085e0:
   allocator = 0x7fff709faee0
   callback = 0x100001522
   context = {0, 0x0, 0x0, 0x0, 0x0}
   numPathsToWatch = 1
   pathsToWatch = 0x7fff709faee0
        pathsToWatch[0] = '/Users/dave/code/temp/rb-fsevent/spec/fixtures'
   latestEventId = -1
   latency = 500000 (microseconds)
   flags = 0x00000000
   runLoop = 0x0
   runLoopMode = 0x0

append_path called for: /users/dave/code/temp/rb-fsevent/spec/fixtures
  resolved path to: /Users/dave/code/temp/rb-fsevent/spec/fixtures

config.sinceWhen    18446744073709551615
config.latency      0.500000
config.flags        00000000
config.paths
  /Users/dave/code/temp/rb-fsevent/spec/fixtures

FSEventStreamRef @ 0x1001085e0:
   allocator = 0x7fff709faee0
   callback = 0x100001522
   context = {0, 0x0, 0x0, 0x0, 0x0}
   numPathsToWatch = 1
   pathsToWatch = 0x7fff709faee0
        pathsToWatch[0] = '/Users/dave/code/temp/rb-fsevent/spec/fixtures'
   latestEventId = -1
   latency = 500000 (microseconds)
   flags = 0x00000000
   runLoop = 0x0
   runLoopMode = 0x0

F
append_path called for: /users/dave/code/temp/rb-fsevent/spec/fixtures
  resolved path to: /Users/dave/code/temp/rb-fsevent/spec/fixtures

config.sinceWhen    18446744073709551615
config.latency      0.500000
config.flags        00000000
config.paths
  /Users/dave/code/temp/rb-fsevent/spec/fixtures

FSEventStreamRef @ 0x1001085e0:
   allocator = 0x7fff709faee0
   callback = 0x100001522
   context = {0, 0x0, 0x0, 0x0, 0x0}
   numPathsToWatch = 1
   pathsToWatch = 0x7fff709faee0
        pathsToWatch[0] = '/Users/dave/code/temp/rb-fsevent/spec/fixtures'
   latestEventId = -1
   latency = 500000 (microseconds)
   flags = 0x00000000
   runLoop = 0x0
   runLoopMode = 0x0

Failures:

  1) FSEvent should work with path with an apostrophe
     Failure/Error: @results.should == [custom_path.to_s + '/']
       expected: ["/users/dave/code/temp/rb-fsevent/spec/fixtures/custom 'path/"]
            got: [] (using ==)
       Diff:
       @@ -1,2 +1,2 @@
       -["/users/dave/code/temp/rb-fsevent/spec/fixtures/custom 'path/"]
       +[]
     # ./spec/rb-fsevent/fsevent_spec.rb:30:in `block (2 levels) in <top (required)>'

  2) FSEvent should catch new file
     Failure/Error: @results.should == [@fixture_path.to_s + '/']
       expected: ["/users/dave/code/temp/rb-fsevent/spec/fixtures/"]
            got: [] (using ==)
       Diff:
       @@ -1,2 +1,2 @@
       -["/users/dave/code/temp/rb-fsevent/spec/fixtures/"]
       +[]
     # ./spec/rb-fsevent/fsevent_spec.rb:40:in `block (2 levels) in <top (required)>'

  3) FSEvent should catch file update
     Failure/Error: @results.should == [@fixture_path.join("folder1/").to_s]
       expected: ["/users/dave/code/temp/rb-fsevent/spec/fixtures/folder1/"]
            got: [] (using ==)
       Diff:
       @@ -1,2 +1,2 @@
       -["/users/dave/code/temp/rb-fsevent/spec/fixtures/folder1/"]
       +[]
     # ./spec/rb-fsevent/fsevent_spec.rb:49:in `block (2 levels) in <top (required)>'

  4) FSEvent should catch files update
     Failure/Error: @results.should == [@fixture_path.join("folder1/").to_s, @fixture_path.join("folder1/folder2/").to_s]
       expected: ["/users/dave/code/temp/rb-fsevent/spec/fixtures/folder1/", "/users/dave/code/temp/rb-fsevent/spec/fixtures/folder1/folder2/"]
            got: [] (using ==)
       Diff:
       @@ -1,3 +1,2 @@
       -["/users/dave/code/temp/rb-fsevent/spec/fixtures/folder1/",
       - "/users/dave/code/temp/rb-fsevent/spec/fixtures/folder1/folder2/"]
       +[]
     # ./spec/rb-fsevent/fsevent_spec.rb:61:in `block (2 levels) in <top (required)>'

Finished in 13.4 seconds
5 examples, 4 failures

I'm running ruby-1.9.2-p180 Mac OSX 10.6.6

I'm not sure what other information would be helpful. If i build the debug binary and run it on a directory i'm not seeing any events firing.

Any ideas?

whatupdave commented 13 years ago

Ok after filing this I checked apple update and updated to Mac 10.6.7 which has fixed the problem of file system events not firing.

ttilley commented 13 years ago

CRAZY! I just read the email github sent about the bug and was worried I forgot to double-check everything in xcode3.latest or something silly. ;)

I'm glad that fixed things for you, but it's worrisome that there might be a configuration out there that just flat out doesn't work. shrug

whatupdave commented 13 years ago

I'm not 100% if it was the mac version or something else going on in the system. I rebooted a few times to see if it made a difference. I did install windows boot camp the other day so I guess my beautiful mac will never be the same....

rmoriz commented 13 years ago

having exactly the same issues. running 10.6.7 with all patches + Xcode 4.0.2

events get fired by OSX as http://www.fernlightning.com/doku.php?id=software:fseventer:start shows them

could not find a way to fix this. tried several different rubies (1.8.7, 1.9.2, 1.8.7EE) without success.

rmoriz commented 13 years ago

However v0.3.10 works (at least specs):

(in /private/tmp/rb-fsevent)
/Users/rmoriz/.rvm/rubies/ree-1.8.7-2011.03/bin/ruby -S bundle exec rspec ./spec/rb-fsevent/fsevent_spec.rb
No examples were matched by {:focus=>true}, running all
creating Makefile
fsevent_watch compiled
....

Finished in 13.27 seconds
4 examples, 0 failures
ttilley commented 13 years ago

rmoriz: there are two fsevent APIs. one is private, fairly low level, and essentially allows for a userspace application to insert itself in the middle of I/O events in the kernel... which allows not-well-behaved code to cause serious problems. This is the API used by spotlight, and apparently also fseventer.

The public API for FSEvents is based on a daemon that makes use of this much lower level API to log a much less detailed version of events, trimmed down to the directory level, under the /root_of_that_volume/.fseventsd/ directory.

Now... the most confusing detail of what you're seeing is that 0.3.10 works and 0.4 does not. This is different from anyone else I've heard from so far has been seeing: you either have issues that break use of FSEvents by any and all applications (minus spotlight, and cheaters that use undocumented APIs), or issues that break FSEvents when working within a specific volume.

I recently heard from a user who disabled spotlight, re-enabled it, rebooted twice, and POOF... magic. Things worked again. I'm still a bit frustrated I wasn't able to figure out what was going on before his machine just started working again out of nowhere, and thus don't understand the solution to what he was seeing. However, more out of superstition than logic (as spotlight uses /dev/fsevents, not fseventsd), you might want to go to preferences -> spotlight -> privacy, add your root filesystem, reboot, remove it, reboot. You have no idea how painful it is for me to even suggest such a thing.

Before hopping down that rabbit hole, however, it's worth re-trying 0.4 if you're seeing 0.3.10 work. With the changes involved it makes very little sense for you to not be seeing events. I can see there being compounding issues of all kinds that might stop them from reaching you, but not that fsevent_watch itself isn't seeing them. Please re-compile with debugging as well.

export FWDEBUG="true"
gem install rb-fsevent

...Note to self: make that a commandline option.

rmoriz commented 13 years ago

@ttilley that was me on twitter ;)

btw. 0.3.10 didn't work, only the tests were all green, sorry for the confusion. As far as I saw the 0.4 released added more tests.

The strange thing was, that other fsevent based apps worked but rb-fsevent did not. First I thought maybe the noatime mount option (b/c of my SSD) could be a problem. I remembered that I've disabled spotlight indexing on the main partition. After removing the partition form the ignore list the indexer started. I rebooted 2 times because of other things…

Then rb-fsevent (used in guard) started working perfectly out of the blue. The rb-fsevent tests got green! I've now disabled spotlight indexing again, rb-fsevent still works.

tl;dr

BTW: I've updated my Xcode to the 4.0.x release. That alone did not help but maybe it's part of the solution? (System was already 10.6.7 when I started)

ttilley commented 13 years ago

oh. well. alrighty then!

Definitely good to know that we have since improved the quality of the test suite so that it doesn't give false passes... Especially since I wasn't aware of them. A few of the modifications between 0.3.10 and 0.4 were actually done to prevent false failures. Both may have had the same cause.

lox commented 13 years ago

I just had exactly the same issue. Very strange, nothing had changed in terms of ruby, rb-fsevent or any other software on my system, my sync script just stopped getting events. I could see events with fseventer, but nothing was coming through to my script. OSX is 10.6.7.

I tried rebooting twice, no result. I excluded the root partition from indexing, rebooted and everything is working again.

lox commented 13 years ago

Unfortunately the problem re-surfaces once I remove my root partition from the Spotlight exclusion list. Frustrating.

quackingduck commented 13 years ago

Same issue here. Adding the root partition to the spotlight exclusion list and rebooting fixed the problem

lox commented 13 years ago

Actually, I was able to add just the directories in question to the exclusion list and it works fine.

bquorning commented 13 years ago

On OSX 10.6.7, I have absolutely no luck making rb-fsevent work. Tried adding my "code" folder to Spotlight’s exclusion list and rebooted the machine. Still doesn’t work.

ttilley commented 13 years ago

can you all check each volume for /.fseventsd/no_log ? Also, make sure each volume has a /.fseventsd/fseventsd-uuid file containing a UUID of the format:

ABC01D2E-F345-6ABC-D7E8-F9AB01C234D5

Since I'm not using the volume specific API, it might also just blindly be using the root filesystem settings... That's certainly possible.

Also check to make sure there's an as-root process running called fseventsd (or, more specifically, /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/CarbonCore.framework/Versions/A/Support/fseventsd).

I guess also make sure there's a /dev/fsevents ? Looks like it should be mode 0644 with major,minor of 12,6393236.

Also, grep /private/var/log/system.log for 'fseventsd'. Note that "destroying old logs" messages are harmless, and simply mean that the filesystem was modified without that specific fseventsd knowing about it (like... booting into your lion partition on the same drive and having its' spotlight index your snow leopard partition... or what have you). Example:

./system.log:Jun 17 15:09:38 Travis-Tilleys-MacBook-Pro fseventsd[20767]: event logs in /.fseventsd out of sync with volume.  destroying old logs. (1 170038 170050)

I really really wish I knew what the problem was, specifically, for you guys so that I could write in a check for it and display a helpful warning. Any information is invaluable.

bquorning commented 13 years ago

Just tried it on my home computer: it works fine. Running the “Singular path” example code from the README, I now see fsevent.run happily printing a line when a Dir is changed.

My work computer (which I reported rb-fsevent not working on yesterday) just returned when calling fsevent.run. Maybe pipe.eof? returns true right away? Anyhow, I’ll try debugging per your instructions above when I get back to my work machine on monday.

robwilliams commented 13 years ago

I came across this issue the other day and noticed it may be due to case sensitivity of directory names in the home folder.

All my projects were stored in:

/Users/Rob/Projects

I wanted to tab complete without typing the uppercase 'P' so I renamed projects to:

/Users/Rob/projects

Great so now I can tab complete.. Hold on a minute Guard has stopped working.

Renaming it back to Projects fixed it.

I tried creating a spec to catch this error but it passes without issues. :/

  # This only happens in the users home directory from what i can tell
  # so to prove it we test both
  {
    :in_home_directory     => Pathname.new(File.expand_path('~')),
    :in_fixtures_directory => Pathname.new(File.expand_path('../../fixtures/', __FILE__))
  }.each { |directory_description, path|

    it "should work with folder that has been renamed from Uppercase to lowercase #{directory_description}" do

      uppercase_path = path.join("FseventCasetest")
      lowercase_path = path.join("fseventcasetest")
      uppercase_file_path = uppercase_path.join("watch.txt")
      lowercase_file_path = lowercase_path.join("watch.txt")

      # Make sure fixtures are clean
      FileUtils.rm_rf(uppercase_path)
      FileUtils.rm_rf(lowercase_path)

      FileUtils.mkdir(uppercase_path)

      FileUtils.touch(uppercase_file_path)

      # Use OS X mv that is case sensitive and allows renaming of a directory
      # to its uppercase equivalent in the same parent directory
      `mv -v #{uppercase_path} #{lowercase_path}`

      @fsevent.watch lowercase_path.to_s do |paths|
        @results += paths
      end

      run
      FileUtils.touch lowercase_file_path
      stop

      File.delete lowercase_file_path
      # Make sure fixtures are clean
      FileUtils.rm_rf(uppercase_path)
      FileUtils.rm_rf(lowercase_path)

      @results.should == [lowercase_path.to_s + '/']

    end
  }
ttilley commented 13 years ago

The subset of events involving directory renames (while running) aren't -actually- handled properly, since the communication format between the C subprocess and ruby library doesn't allow for it. This is only an issue when renaming a directory that's explicitly watched (not a subdirectory). I was determined to fix that and a number of other things (see the "coming soon" comments in the readme), but, erm... I got distracted. ^^;

You know, I actually have a fair chunk of time today and throughout this week to devote to the work necessary to check off those TODO items. I guess it's time to get motivated and bang that out...

An aside for @thibaudgg - I intend to use tagged netstrings (probably) as a serialization format. It honestly took an attempt at serializing to JSON in C to make me realize I was being silly creating so much extra work for myself.

bquorning commented 13 years ago

Before I got around to debugging per @ttilley’s request above, I found the following: Using the irbtools gem (version 1.0.4 here, haven’t tested with others) on ruby-1.8.7-p174 (ruby 1.8.7 (2009-06-12 patchlevel 174) [universal-darwin10.0]) causes rb-fsevent (calling fsevent.run per the README) to return instantly. Using ruby-1.8.7-p334 [ x86_64 ] or ruby-1.9.2-p180 [ x86_64 ] works just fine with irbtools.

Not requiring irbtools makes rb-fsevent work just fine on ruby-1.8.7-p174 here.

So, that looks like a bug right there. (Though maybe not on this gem…)

ttilley commented 13 years ago

twitch

bquorning commented 13 years ago

My problems aren’t over yet: Now rb-fsevent does not work on my work computer anymore. Not in 1.8.7 (any patchlevel) or 1.9.2. With or without irbtools, no difference.

This is on OSX 10.6.8, 2.4GHz Intel Core 2 Duo. Let me know if you need more information.

bquorning commented 13 years ago

I just tried running the README sample code from an unsaved TextMate file. Here, Dir.pwd is "/private/var/folders/ds/[something very long]/-Tmp-/" – and rb-fsevent works!

Now, why doesn’t it work when watching my home dir? And how can I help you debug this issue?

ttilley commented 13 years ago

...wtf? so it works under /private but not outside of /private? can you try other folders that are under /private/?

bquorning commented 13 years ago

Today’s round of debugging leaves me equally baffled…

  1. Tried watching /private (works), /Users/benjamin (works!) and /Users/benjamin/code (doesn’t work).
  2. I tried making a new subfolder /Users/benjamin/kode and watching it as first “kode”, then “Kode.” Both work.
  3. Suspecting that my code folder’s name (“code”) was the problem, I renamed the folder and watched it. Yay, now it works.
  4. Renamed back to “code” to confirm that the problem were indeed the folder’s name. Argh (but yay, I guess), it still works.

Concluding: I don’t know what the problem was, but changing the folder’s name to something else, and back, seems to have solved the problem.

ttilley commented 13 years ago

That actually says quite a bit. When you rename a folder, that doesn't change the inode that data is kept in... just metadata. So, technically, there wasn't a new 'thing' to watch. It's not the folder itself that's a problem, OR that it's under the specially-handled OSX 'private' folder where exceptions are made in the realpath logic.

[09:13:08][~]$ stat -x code
  File: "code"
  Size: 68           FileType: Directory
  Mode: (0751/drwxr-x--x)         Uid: (  501/ ttilley)  Gid: (   20/   staff)
Device: 14,2   Inode: 5577795    Links: 2
Access: Fri Jul 15 09:12:43 2011
Modify: Fri Jul 15 09:12:43 2011
Change: Fri Jul 15 09:12:43 2011
[09:13:17][~]$ mv code kode
[09:13:24][~]$ stat -x kode
  File: "kode"
  Size: 68           FileType: Directory
  Mode: (0751/drwxr-x--x)         Uid: (  501/ ttilley)  Gid: (   20/   staff)
Device: 14,2   Inode: 5577795    Links: 2
Access: Fri Jul 15 09:12:43 2011
Modify: Fri Jul 15 09:12:43 2011
Change: Fri Jul 15 09:12:43 2011
[09:13:29][~]$ mv kode code
[09:13:40][~]$ stat -x code
  File: "code"
  Size: 68           FileType: Directory
  Mode: (0751/drwxr-x--x)         Uid: (  501/ ttilley)  Gid: (   20/   staff)
Device: 14,2   Inode: 5577795    Links: 2
Access: Fri Jul 15 09:12:43 2011
Modify: Fri Jul 15 09:12:43 2011
Change: Fri Jul 15 09:12:43 2011

Note that the inode (where it's kept on disk, the thing being watched) hasn't changed. It remains 5577795. Also, the access/modify/change times have stayed the same throughout. ALL that has changed is a small chunk of metadata in the inode referring to the directory.

I mean, it's not a solution, but it IS a significant amount of information.

ttilley commented 13 years ago

It's good to know we're not the only ones seeing fsevents do crazy things of a similar nature -> http://lists.apple.com/archives/filesystem-dev/2011/Jan/msg00000.html

Note that his testing scenario resulted in the file creation event in one scenario, but not another (where the file was deeper in the tree before the multiple-level-ancestor directory was renamed)... and at least for time machine, both cases resulted in an inaccurate backup.

Fun times.

volkanunsal commented 13 years ago

I'm having this exact same problem. The event are not firing and as a result Guard, which depends on rb-fsevent, does not work on my system. Hopefully a solution can be found for it.

muddymatches commented 13 years ago

Upgrading to Lion fixed this for me.

ttilley commented 13 years ago

@tenaciousflea can you perform the checks I mentioned where I re-opened this bug?

I wish I could ssh into one of your machines or something to poke around... in a shared screen session perhaps. As of yet, I have been unable to reliably reproduce this and so I can't dig into what might be causing it in depth. >_<

ttilley commented 13 years ago

Actually... I live on east coast USA. If any of you still having this problem are ok with letting me shell in and poke around, I'm often on irc.freenode.net as either Aphelion or ttilley (though not as often non-idle). Alternatively, we could set up screen sharing over ichat, which would be nice since we could talk at the same time.

I really really really want to stomp this bug flat and be done with it.

volkanunsal commented 13 years ago

@ttilley I can offer to help you using my computer. I looked up your post from when you reopened the issue, but I'm not proficient enough with shell programming to do the tests. If you give me brief snippets of code to run, I can run them and tell you what they produce.

Is there a chatroom you go to often? I'm on the East Coast as well, and I can be available over the weekend.

ttilley commented 13 years ago

I tend to idle in: #ruby-lang #ruby #macruby #rubinius #jruby

thibaudgg commented 13 years ago

You can also use #guard (irc.freenode.net).

ttilley commented 13 years ago

...I just realized it's 7am and I haven't gone to sleep yet. So I hope you're not a morning person.

I have, however, converted fsevent_watch into its own xcode project, replaced the extconf.rb fakeout build system with a rakefile that calls xcodebuild, performed some cleanup, did extensive testing to ensure that this works as expected in 10.6+10.7 using xcode 3.2.6, 4.0.2, 4.1, and 4.2b5 (4.1+ for lion), refactored some C, added a hidden command-line argument to enable file-level events in lion for giggles, and introduced a system of pluggable output formats with the eventual goal of exposing extensive metadata on each event fired.

Graph here: https://github.com/thibaudgg/rb-fsevent/network

Not bad for a sitting. I think.

volkanunsal commented 13 years ago

Awesome. I'm trying to reproduce the problem at my home computer. Just the day before I was seeing the issue both on my home and my work computer. Then I upgraded to Lion at both places -- which meant upgrading XCode too. Yesterday when I left office, I was still seeing the problem on my work computer, but at my home computer it has...evolved. At least I think it's no longer something to do with rb-fsevent. The problem has moved up to the guard-coffeescript. I am able to see the events are detected on my computer, but guard-coffeescript is not compiling javascript files. Well, at least that's something.

I'm going to have to report back on how the office computer is functioning on Monday. I already posted on guard-coffeescript issue log to let them know about this.

ttilley commented 13 years ago

crap. there goes another chance to figure out, exactly, what the problem is and write a check for it. just like everyone else who reports this bug you did something and then it magically went away.

volkanunsal commented 13 years ago

Seems updating to Lion solved it for me as well. It may have been the XCode update, not Lion by itself, though.

ttilley commented 13 years ago

...I'm going to just close this issue. I can't reproduce it and everyone who reports it also reports it going away after doing X where X varies. I'm strongly debating a pre-compiled binary approach and am doing some heavy refactoring of fsevent_watch.

andreyvit commented 13 years ago

Guys, I just wanted to chime in to say that LiveReload 2 users are experiencing this exact problem too, most with folders inside Dropbox. I'm 100% sure it's an OS bug. I generally recommend the create-new-folder-and-move-everything approach, and it's yet to fail for anyone (others reported that rebooting solved the issue).

So when you detect a failing folder /foo/bar/boz, you do

cd /foo/bar
mv boz boz.0
mkdir boz
mv boz.0/* boz/*
rmdir boz.0

Ditto for /foo/bar and /foo, if needed. Problem will be solved.

I have no idea what the cause is, but I have plans to include automatic detection of this issue into LiveReload 2.

volkanunsal commented 13 years ago

@andreyvit

In my case, I figured out the folder name was the cause of the problem. The folders had . and ( characters in their names. Moving to folders without those characters solved my problem. The example you give seems to validate my conclusion.

andreyvit commented 13 years ago

@tenaciousflea Um… which example? /foo/bar/boz was the original name in the example, and it's not even a real name. (This happened to me with perfectly regular folders. In fact, it used to happen on ~/Dropbox/Projects/livereload itself. :)

volkanunsal commented 13 years ago

@andreyvit

In that case, the mystery continues.

ttilley commented 13 years ago

This bug makes me want to stab myself in the face.

If I can poke at a machine/setup where this problem actually happens that'd be great. Every person who has offered has ended up doing something before I get there (install something, rename folders, upgrade xcode, upgrade macos) and the problem goes away before I can take a look.

gvt commented 13 years ago

I am experiencing this on OSX 10.7.1 with rb-event version 0.4.3.1. Its a real pain.

I am finding that autotest-fsevent does work (0.2.5).

ttilley commented 13 years ago

I've been mostly without reliable Internet access for the past week. This should be resolved Monday afternoon/evening (hopefully). If you see ttilley/Aphelion on irc.freenode.net that's me and I'd love to poke around and try to get to the bottom of this.

On Saturday, September 24, 2011, Gregory Tomei < reply@reply.github.com> wrote:

I am experiencing this on OSX 10.7.1. Its a real pain.

Reply to this email directly or view it on GitHub: https://github.com/thibaudgg/rb-fsevent/issues/10#issuecomment-2188022

volkanunsal commented 13 years ago

@ttilley

This started happening on my computer again after a routine firmware update. I haven't tested it thoroughly to understand its symptoms just yet, but when I moved the folder to the desktop, the issue went away. If you want to sync up today (or tomorrow) to look at my system, that would work for me.

ttilley commented 13 years ago

Alright! I'm on IRC as Aphelion as I type, and have plenty of time to poke around right now.

On Wed, Sep 28, 2011 at 12:19 PM, tenaciousflea < reply@reply.github.com>wrote:

@ttilley

This started happening on my computer again after a routine firmware update. I haven't tested it thoroughly to understand its symptoms just yet, but when I moved the folder to the desktop, the issue went away. If you want to sync up today (or tomorrow) to look at my system, that would work for me.

Reply to this email directly or view it on GitHub: https://github.com/thibaudgg/rb-fsevent/issues/10#issuecomment-2227010

ttilley commented 13 years ago

it was massively helpful to poke around on a machine where this was consistently breaking, and i'm now sure i have a clearer picture of the bug. @andreyvit - ping in case you're interested (or have more info than I do here).

In this particular instance of the bug, the folder 'Xperiments' was returning 'xperiments' via realpath as well as apple's path resolving apis. After renaming the folder to 'xperiments' and then back to 'Xperiments', those apis returned the expected with-caps name. Unfortunately I accidentally fixed the bug on the machine before getting the kind of data I wanted, and I have no idea how to directly parse the fseventsd data logs, but I'm guessing the events were being produced using the with-caps version of the name and thus watching for the without-caps version will miss them entirely.

In short, the bug is that realpath() doesn't give a correct result all the time for case-insensitive-but-preserving HFS+ volumes.

ttilley commented 13 years ago

i'd also like to note that stat -x was also returning unmodified whatever path it was given. if called on the with-caps path, that's what it gave as a name. same for without-caps.

ttilley commented 13 years ago

...we are of course back to the scenario where I don't have access to a machine where the bug is occurring, so I can only blindly try things and hope it works. if realpath() isn't returning the correct result, readdir() might not either (though i'm hoping this is unlikely, since ls shows the correct case).

anyways, the idea is to use realpath() or similar and then manually check each directory node for the correct case for that child. slightly painful, but at least it only needs to happen once... when registering a path to watch.

andreyvit commented 13 years ago

@ttilley Great findings. Does actually explain why recreating folders fixes this problem. And btw I have info that 10.7 users also still experience the problem, so it's not fixed in OS X. Will try to play with this new info too.