Feature Request: support for directory watcher

[Shaderboy]

It would be awesome to have support for a directory watcher for remote changes made on public or shared private directories, something like inotify. Alternatively, some sort of built in notification system for when those directories are updated. Any plans for something like this?

[cjb]

We've been talking about it! Unfortunately inotify only hooks into the VFS for local filesystem changes -- there are no network filesystems that support inotify.

So we probably instead have to make our own watcher.

[Shaderboy]

Hmm, I'm successfully running inotify on my NFS. The client that makes the changes can get notified by watching the mounted directory, and the server obviously gets notifications as well (running an instance of inotify on each). Any other clients don't, but it should be enough to pass them along from the server to the client via a notification in the keybase app. What kind of driver is keybase running on?

[strib]

KBFS on OSX/Linux is using FUSE. I haven't tried it, but it's possible inotify would work for purely local changes. The issue is how to deliver remote notifications. We already deliver them to the remote clients in a timely fashion. So if you wanted something quick and dirty, you could easily just stat the directory every second, and you would see the changes. (If nothing changed, this would hit the cache and wouldn't cause network traffic -- changes are pushed from the servers.)

@cjb was talking about coming up with our own way to deliver the changes to a remote watcher -- inotify can't do that for us, unfortunately, so we'd have to build our own watcher system for that.

[Shaderboy]

Ah I see. Isn't there already a protocol to send notifications from the server to the client's keybase process? Forgive me if I'm just not getting how this works. I know you all are working hard and you're doing an awesome job!

[strib]

Yep, that protocol exists. The issue is the very last hop -- from the keybase process to some other process (written by you) running on the client, which desires watch notifications. I don't know of any standard protocol for this, so we'd have to invent one. We have fine-grained notification data to invalidate kernel cache data anyway (this file was created in this directory, this section of a file was overwritten, etc). We just have no standard way to deliver that information outside of keybase right now.

[malgorithms]

As for awareness of what's going on in the filesystem, it's also going to be a very common request to know about new TLF's that were previously unknown to the local client. I just figured I'd bring it up to have in the backs of our minds. To expose that info, I assume we'd need either an API call to the local keybase client or to expose a .kbfs_something file with that info? Whatever the answer is might also answer these questions about changes on known data.

[atondwal]

I'm not sure if these bindings are exposed to the go FUSE bindings, but it is possible to trigger inotify events with FUSE filesystems. Stef Bon has an example in c: https://github.com/stefbon/notifyfs

[cjb]

@atondwal Hm, are you sure that's injecting inotify events, rather than just consuming them?