search

shama / navelgazer

:tangerine::crystal_ball: A super fast, light weight, simple file watcher and platform layer for gaze.
MIT License
21 stars 1 forks source link

Event Stream Facade and Tree Caching #14

Open joshperry opened 8 years ago

joshperry commented 8 years ago

While I was working on getting tests to pass on Linux, I found an an immensely informative article about managing events from inotify, handling edge-cases, and even renames. An interesting method for efficiently tracking state changes is effected by creating a cache of the watched tree.

https://lwn.net/Articles/605128/

If I understand the purpose of navelgazer correctly, it is meant to abstract and unify the API for different methods of native OS file change notification, none of which seem to support recursive watches. Gaze on top of navelgazer is meant to add globbing and recursive watching of filesystem subtrees.

To that end, I think we should to change the architecture to make meeting these goals a bit easier.

I want to do as little in C++ as possible, so I'd like to add a platform-specific javascript facade on top of the native code. There is currently the FileSystemWatcher object that communicates directly with the native modules, and is responsible for smoothing out the differences in the event streams, while at the same time providing the logic for watcher setup and bookkeeping.

This new facade layer would be in charge of homogenizing the event streams coming from each of their respective native modules, and delivering them to the FileSystemWatcher for notification of the various interested parties.

In addition to the facades, I'd like to add a tree cache similar to the one used in the above article for keeping an internal representation of the filesystem subtrees that are being watched by the system. This can be leveraged by the FileSystemWatcher and the facades to optimize the number of watches needed, and the efficient removal/creation of watches by analyzing the tree as they are added or removed. The cache will also be useful for the facades to properly implement multi-message events like renames and moves.

Does this perspective seem to jive with the project's actual goals?

shama commented 8 years ago

Big +1, that sounds awesome.

shama commented 8 years ago

Check out this library: https://github.com/emcrisostomo/fswatch Seems like it could make a great platform layer or at the very least pull from.