search

project8 / hornet

Hornet is a nearline data processor for the Project 8 experiment
Other
0 stars 0 forks source link

Watcher doesn't like directories being removed #10

Closed nsoblath closed 9 years ago

nsoblath commented 9 years ago

Oddities:

2015/06/04 15:20:42 [watcher dir event] "/data/hot/dirtest2": 0x40000100 == IN_CREATE|IN_ISDIR
2015/06/04 15:20:42 [watcher] added subdirectory to file watch [/data/hot/dirtest2]
2015/06/04 15:20:42 [watcher] added subdirectory to dir watch [/data/hot/dirtest2]
2015/06/04 15:20:51 [watcher dir event] "/data/hot/dirtest2": 0x8000 == IN_IGNORED
2015/06/04 15:20:51 [watcher dir event] "/data/hot/dirtest2": 0x40000200 == IN_DELETE|IN_ISDIR
2015/06/04 15:20:51 [watcher] removing watch on /data/hot/dirtest2...
2015/06/04 15:20:51 [watcher] can't remove file watch on /data/hot/dirtest2 [inotify_rm_watch: invalid argument]
2015/06/04 15:20:51 [watcher] can't remove dir watch on /data/hot/dirtest2 [inotify_rm_watch: invalid argument]
2015/06/04 15:20:51 [watcher file event] "/data/hot/dirtest2": 0x8000 == IN_IGNORED
2015/06/04 15:20:51 [scheduler] sending <dirtest2> to the classifier
2015/06/04 15:20:51 [classifier] file </data/hot/dirtest2> does not exist
2015/06/04 15:20:51 [scheduler] error received from the classifier:
    [classifier] file </data/hot/dirtest2> does not exist

This is from testing on ignatius

nsoblath commented 9 years ago

In testing on teselecta I see the following:

2015/06/08 12:04:47 [watcher dir event] "/home/nsoblath/go/src/github.com/project8/hornet/build-dftesting/run0": 0x8000 == IN_IGNORED
2015/06/08 12:04:47 [watcher dir event] "/home/nsoblath/go/src/github.com/project8/hornet/build-dftesting/run0": 0x40000200 == IN_DELETE|IN_ISDIR
2015/06/08 12:04:47 [watcher] removing watch on /home/nsoblath/go/src/github.com/project8/hornet/build-dftesting/run0...
2015/06/08 12:04:47 [watcher] can't remove file watch on /home/nsoblath/go/src/github.com/project8/hornet/build-dftesting/run0 [inotify_rm_watch: invalid argument]
2015/06/08 12:04:47 [watcher] can't remove dir watch on /home/nsoblath/go/src/github.com/project8/hornet/build-dftesting/run0 [inotify_rm_watch: invalid argument]
2015/06/08 12:04:47 [watcher file event] "/home/nsoblath/go/src/github.com/project8/hornet/build-dftesting/run0": 0x8000 == IN_IGNORED
2015/06/08 12:04:47 [scheduler] sending <run0> to the classifier
2015/06/08 12:04:47 [classifier] file </home/nsoblath/go/src/github.com/project8/hornet/build-dftesting/run0> does not exist
2015/06/08 12:04:47 [scheduler] error received from the classifier:
    [classifier] file </home/nsoblath/go/src/github.com/project8/hornet/build-dftesting/run0> does not exist
nsoblath commented 9 years ago

Reading through the inotify man page (), I found this comment regarding the bits that can be set in the return from an inotify read call:

IN_IGNORED
Watch was removed explicitly (inotify_rm_watch(2)) or automatically (file was deleted, or file system was unmounted).

This implies that the watches on directories that are deleted are automatically removed. Therefore we don't have to remove them ourselves. It further explains I was seeing IN_IGNORED events when directories were deleted. Those were the watches being removed.

Further testing shows that anytime a directory being watched is deleted, or moved, the remove-watch call is invalid: inotify_rm_watch: invalid argument. Therefore the course of action is to stop acting on IN_DELETE and IN_MOVED_FROM events.

nsoblath commented 9 years ago

Fixed in commit 0de83b5