search

SpartanJ / efsw

efsw is a C++ cross-platform file system watcher and notifier.
Other
666 stars 103 forks source link

Shifted events in Mac OS v10.7.5 from Finder #47

Closed SpartanJ closed 11 years ago

SpartanJ commented 11 years ago

Original report by Batte HUCHAI (Bitbucket: bhuchai, ).

Hello,

I have some troubles with EFSW on Mac OS v10.7.5. Events seem to be sent with a shift ...

For example :

I have 2 files in my watched folder : BAR.txt and FOO.txt

1. I delete BAR.txt : no event is sent.

2. I rename FOO.txt in FOO2.TXT : the below event will be printed :

DIR (/Users/myuser/Documents/) FILE (FOO.txt) has event Moved from (BAR.txt)

3. I create TEST.txt : the below event will be printed :

DIR (/Users/myuser/Documents/) FILE (TEST.txt) has event Added

4. I rename TEST.txt in TEST2.txt : the below event will be printed :

DIR (/Users/myuser/Documents/) FILE (FOO2.txt) has event Moved from (TEST.txt)

So, the only correct step is the number 3. All others seem to be shifted compared to normal way.

It's important to know that if I do the same 3 steps directly by command lines (Terminal), no from Finder, it's working. Is problem coming from Finder ?

Do you have any idea ?

Thanks by advance,

B.

SpartanJ commented 11 years ago

Original comment by Martín Lucas Golini (Bitbucket: SpartanJ, GitHub: SpartanJ).

Hi Batte, This is clearly an efsw bug. The problem is that i don't have access to any OS X 10.7 to test it, sadly my Mountain Lion VM isn't working right now. The problem comes from what i clarify in the README.md file, this is what it says: "FSEvents for OS X Lion and beyond in some cases will generate more actions that in reality ocurred, since fine-grained implementation of FSEvents doesn't give the order of the actions retrieved, in some cases i need to guess/aproximate the order of them." May be you can help me a little if you send me a complete log of the events with efsw compiled as debug ( it gives some extra information for debugging ). Finder generates weird events on all OS X versions, but i don't know exactly why, they are correct, but sometimes you don't receive exactly what you can expect. As a temporary fix you can compile efsw with the flag "EFSW_FSEVENTS_NOT_SUPPORTED", it will use the Kqueue backend instead the FSEvents backend, or you can change this function FileWatcherFSEvents::isGranular() to return always false, and it will use FSEvents without granularity. And if you are brave and wan't to take a look at the problem, the problem starts in WatcherFSEvents.cpp at line 137.

Thank you for reporting it, i'll try to take a look at this ASAP.

SpartanJ commented 11 years ago

Original comment by Batte HUCHAI (Bitbucket: bhuchai, ).

Hello,

Thank you for answering me.

For your debug logs, I compiled EFSW in debug mode (with "config=debug" instead of "config=release") but it seems to print the same thing (for the same events sequence as described in my initial post) :

[Good] With Terminal :

DIR (/Users/buchet_b/Documents/Watched/) FILE (BAR.txt) has event Delete
DIR (/Users/buchet_b/Documents/Watched/) FILE (FOO2.txt) has event Moved from (FOO.txt)
DIR (/Users/buchet_b/Documents/Watched/) FILE (TEST.txt) has event Added
DIR (/Users/buchet_b/Documents/Watched/) FILE (TEST2.txt) has event Moved from (TEST.txt)

[Bad] With Finder : (n.b. the 3rd step of my initial post is a "touch TEST.txt" done with Terminal)

DIR (/Users/buchet_b/Documents/Watched/) FILE (.DS_Store) has event Added
DIR (/Users/buchet_b/Documents/Watched/) FILE (.DS_Store) has event Modified
DIR (/Users/buchet_b/Documents/Watched/) FILE (FOO.txt) has event Moved from (BAR.txt)
DIR (/Users/buchet_b/Documents/Watched/) FILE (TEST.txt) has event Added
DIR (/Users/buchet_b/Documents/Watched/) FILE (FOO2.txt) has event Moved from (TEST.txt)

Note that I use ./lib/libefsw-static-debug.a linked to a Qt C++ project with your main.cpp example (on watched folder "/Users/buchet_b/Documents/Watched/"). Why a Qt project ? Because I would like to use EFSW for a final project which is made with Qt framework (I've actually no choice but it shouldn't be a problem).

Does it normally print others lines ?

B.

SpartanJ commented 11 years ago

Original comment by Martín Lucas Golini (Bitbucket: SpartanJ, GitHub: SpartanJ).

Sorry, i made a mistake, compiling as debug isn't enough, it is needed to compile with the define EFSW_VERBOSE, if you are using the premake4 file, generate the Makefiles with: premake4 gmake verbose, and then recompile. Also, read this: https://bitbucket.org/SpartanJ/efsw/issue/5/event-sent-twice to mix Qt and efsw without problems. Please in the next report, make the examples following the exact same actions with the same files. Thanks for your patience!

SpartanJ commented 11 years ago

Original comment by Batte HUCHAI (Bitbucket: bhuchai, ).

Sorry by advance if I made a mistake but it still seems to print the same.

I did :

iMac:efsw_project buc_b$ ./premake4 gmake verbose

Building configurations...
Running action 'gmake'...
Generating make/macosx/Makefile...
Generating make/macosx/efsw-static-lib.make...
Generating make/macosx/efsw-test.make...
Generating make/macosx/efsw-shared-lib.make...
Done.

iMac:efsw_project buc_b$ cd make/macosx/ && make config=debug

==== Building efsw-static-lib (debug) ====
FileWatcherFSEvents.cpp
Linking efsw-static-lib
==== Building efsw-test (debug) ====
Linking efsw-test
==== Building efsw-shared-lib (debug) ====
FileWatcherFSEvents.cpp
Linking efsw-shared-lib

iMac:macosx buc_b$ ls -la ../../lib/

total 8024
drwxr-xr-x   5 buc_b  staff      170 26 aoû 23:17 .
drwxr-xr-x@ 16 buc_b  staff      544  7 jui 00:50 ..
-rwxr-xr-x   1 buc_b  staff   843976 26 aoû 23:17 libefsw-debug.dylib
-rw-r--r--   1 buc_b  staff  3041272 26 aoû 23:17 libefsw-static-debug.a
-rwxr-xr-x   1 buc_b  staff   215760  6 jul 01:18 libefsw.dylib
SpartanJ commented 11 years ago

Original comment by Martín Lucas Golini (Bitbucket: SpartanJ, GitHub: SpartanJ).

It looks like you forgot to: make clean before building again ( it only rebuilded one file, but the file that sends this verbose notifications is not the file rebuilded, which is WatcherFSEvents.cpp ).

SpartanJ commented 11 years ago

Original comment by Batte HUCHAI (Bitbucket: bhuchai, ).

Indeed, sorry about this "novice mistake" ^.^

So, for the exactly 4 steps of my initial post, I get :

Using backend: FSEvents
File descriptor limit 10240
Event in: /Users/buc_b/Documents/Watched/BAR.txt - flags: 67584
New lastRenamed: BAR.txt
Event in: /Users/buc_b/Documents/Watched/.DS_Store - flags: 69632
DIR (/Users/buc_b/Documents/Watched/) FILE (.DS_Store) has event Modified
Event in: /Users/buc_b/Documents/Watched/FOO.txt - flags: 67584
DIR (/Users/buc_b/Documents/Watched/) FILE (FOO.txt) has event Moved from (BAR.txt)
Event in: /Users/buc_b/Documents/Watched/FOO2.txt - flags: 67584
New lastRenamed: FOO2.txt
Event in: /Users/buc_b/Documents/Watched/TEST.txt - flags: 65792
DIR (/Users/buc_b/Documents/Watched/) FILE (TEST.txt) has event Added
Event in: /Users/buc_b/Documents/Watched/TEST.txt - flags: 67840
DIR (/Users/buc_b/Documents/Watched/) FILE (FOO2.txt) has event Moved from (TEST.txt)
Le programme s'est terminé subitement.
/Users/buc_b/Documents/efsw_test/efsw_test-build-desktop-Desktop_Qt_4_8_1_for_GCC__Qt_SDK__Debug/efsw_test.app/Contents/MacOS/efsw_test a quitté avec le code 0

Do not hesitate if you need more information/help.

SpartanJ commented 11 years ago

Original comment by Martín Lucas Golini (Bitbucket: SpartanJ, GitHub: SpartanJ).

Well, i recovered my OS X Lion VM and made an attempt to fix this problems, it should be working. It's probable that you'll still receive more events than expected but it's not an efsw problem, is a combination of the horrible FSEvents events reports and the aggressive file modification that Finder usually does. Please test it and give me some report about this commit.

SpartanJ commented 11 years ago

Original comment by Batte HUCHAI (Bitbucket: bhuchai, ).

OK, I'm gonna test as soon as possible.

Just a question : your advice which consists to return always FALSE in isGranular() function works well but is there some bad effects if we use it (i.e. loss of functionalities, etc.) ?

SpartanJ commented 11 years ago

Original comment by Martín Lucas Golini (Bitbucket: SpartanJ, GitHub: SpartanJ).

Hi Batte, you don't lost any functionality. The difference relies in the fact that i need to keep a copy of the structure of every folder, and that translates in some memory usage. But the memory usage depends on how big is the directory tree that you're watching. Calculate something like 400 KB every 1000 files, it's it something to take in consideration, but depending on the usage, this could be irrelevant or not.

SpartanJ commented 11 years ago

Original comment by Batte HUCHAI (Bitbucket: bhuchai, ).

Hello,

I've just tried your commit and good news : it's working !

Indeed, for the same events sequence as described in my initial post, I get :

DIR (/Users/buc_b/Documents/Watched/) FILE (BAR.txt) has event Delete
DIR (/Users/buc_b/Documents/Watched/) FILE (.DS_Store) has event Modified
DIR (/Users/buc_b/Documents/Watched/) FILE (FOO2.txt) has event Moved from (FOO.txt)
DIR (/Users/buc_b/Documents/Watched/) FILE (TEST.txt) has event Added
DIR (/Users/buc_b/Documents/Watched/) FILE (TEST2.txt) has event Moved from (TEST.txt)
SpartanJ commented 11 years ago

Original comment by Martín Lucas Golini (Bitbucket: SpartanJ, GitHub: SpartanJ).

Excellent!

I'll close this bug as fixed, but let me know if you ever find something wrong.

Thanks for your help!