rjeczalik / notify

File system event notification library on steroids.
MIT License
900 stars 128 forks source link

NonrecursiveTree does not watch recreated folders #200

Closed FabianKramm closed 3 years ago

FabianKramm commented 3 years ago

Hey there! I think I found a rather severe issue on operating systems that use the NonrecurseTree watcher like linux for recursive watches. The problem is that after you create and delete a folder several times, no subsequent events for that folder will be logged anymore.

I think that the underlying problem is that the non-recursive tree implementation only adds directories to watch and never deletes them from the internal tree structure (except if you call Stop() manually from the outside). This can be seen in the func (t *nonrecursiveTree) dispatch(c <-chan EventInfo) function: https://github.com/rjeczalik/notify/blob/e2a77dcc14cf6732bfa4c361554f27dc696d5d79/tree_nonrecursive.go#L67-L75

Here only 'Create' events are forwarded to the internal handler that adds new watchpoints. However, when a directory is actually removed and recreated, the underlying inotify watch does not work anymore and the tree implementation does not re-add it because it thinks the folder is still being watched, because there is a node for it already in the internal in-memory tree. This means that all subsequent events are lost for this folder and it is essentially unwatched until you restart the complete watcher.

I guess this could be also the reason for some other issues like #190 or https://github.com/syncthing/syncthing/issues/7198

The problem can be reproduced on linux with this snippet:

package main

import (
    "context"
    "fmt"
    "github.com/rjeczalik/notify"
    "io/ioutil"
    "log"
    "os"
    "path/filepath"
    "time"
)

func main() {
        // Cleanup test folder if it is still there
    _ = os.RemoveAll("test")
    err := os.Mkdir("test", 0777)
    if err != nil {
        log.Fatal(err)
    }
    time.Sleep(time.Second)

        // Start watching
    ctx, _ := context.WithCancel(context.Background())
    eventChan := make(chan notify.EventInfo, 1000)
    err = notify.Watch("./test/...", eventChan, notify.All)
    if err != nil {
        log.Fatal(err)
    }

    // Print events
    go func(){
        for {
            select{
            case e := <-eventChan:
                fmt.Println(e.Path() + " - " + e.Event().String())
            }
        }
    }()

    // 1. Create a folder and a file within it
        // This will create a node in the internal tree for the subfolder test/folder
        // Will create a new inotify watch for the folder
    fmt.Println("######## First ########")
    err = recreateFolder("test/folder")
    if err != nil {
        log.Fatal(err)
    }

    // 2. Create a folder and a file within it again 
        // This will set the events for the subfolder test/folder in the internal tree
        // Will create a new inotify watch for the folder because events differ
    time.Sleep(time.Second)
    fmt.Println("######## Second ########")
    err = recreateFolder("test/folder")
    if err != nil {
        log.Fatal(err)
    }

    // 3. Create a folder and a file within it yet again
        // This time no new inotify watch will be created, because the events
        // and node already exist in the internal tree and all subsequent events 
        // are lost, hence there is no event for the created file here anymore
    time.Sleep(time.Second)
    fmt.Println("######## Third ########")
    err = recreateFolder("test/folder")
    if err != nil {
        log.Fatal(err)
    }

    fmt.Println("Done")
    <-ctx.Done()
}

func recreateFolder(dir string) error {
    // Give the sync some time to process events
    _ = os.RemoveAll(dir)
    err := os.Mkdir(dir, 0777)
    if err != nil {
        return err
    }

    time.Sleep(time.Second)

    // Create a file
    fmt.Println("######## Create File ########")
    err = createFile(filepath.Join(dir, "file"))
    if err != nil {
        log.Fatal(err)
    }
    return nil
}

func createFile(file string) error {
    // Now create a file to check if we see that
    err := ioutil.WriteFile(file, []byte("abc"), 0666)
    if err != nil {
        return err
    }

    time.Sleep(time.Second)
    return nil
}