Hey there! I think I found a rather severe issue on operating systems that use the NonrecurseTree watcher like linux for recursive watches. The problem is that after you create and delete a folder several times, no subsequent events for that folder will be logged anymore.
Here only 'Create' events are forwarded to the internal handler that adds new watchpoints. However, when a directory is actually removed and recreated, the underlying inotify watch does not work anymore and the tree implementation does not re-add it because it thinks the folder is still being watched, because there is a node for it already in the internal in-memory tree. This means that all subsequent events are lost for this folder and it is essentially unwatched until you restart the complete watcher.
The problem can be reproduced on linux with this snippet:
package main
import (
"context"
"fmt"
"github.com/rjeczalik/notify"
"io/ioutil"
"log"
"os"
"path/filepath"
"time"
)
func main() {
// Cleanup test folder if it is still there
_ = os.RemoveAll("test")
err := os.Mkdir("test", 0777)
if err != nil {
log.Fatal(err)
}
time.Sleep(time.Second)
// Start watching
ctx, _ := context.WithCancel(context.Background())
eventChan := make(chan notify.EventInfo, 1000)
err = notify.Watch("./test/...", eventChan, notify.All)
if err != nil {
log.Fatal(err)
}
// Print events
go func(){
for {
select{
case e := <-eventChan:
fmt.Println(e.Path() + " - " + e.Event().String())
}
}
}()
// 1. Create a folder and a file within it
// This will create a node in the internal tree for the subfolder test/folder
// Will create a new inotify watch for the folder
fmt.Println("######## First ########")
err = recreateFolder("test/folder")
if err != nil {
log.Fatal(err)
}
// 2. Create a folder and a file within it again
// This will set the events for the subfolder test/folder in the internal tree
// Will create a new inotify watch for the folder because events differ
time.Sleep(time.Second)
fmt.Println("######## Second ########")
err = recreateFolder("test/folder")
if err != nil {
log.Fatal(err)
}
// 3. Create a folder and a file within it yet again
// This time no new inotify watch will be created, because the events
// and node already exist in the internal tree and all subsequent events
// are lost, hence there is no event for the created file here anymore
time.Sleep(time.Second)
fmt.Println("######## Third ########")
err = recreateFolder("test/folder")
if err != nil {
log.Fatal(err)
}
fmt.Println("Done")
<-ctx.Done()
}
func recreateFolder(dir string) error {
// Give the sync some time to process events
_ = os.RemoveAll(dir)
err := os.Mkdir(dir, 0777)
if err != nil {
return err
}
time.Sleep(time.Second)
// Create a file
fmt.Println("######## Create File ########")
err = createFile(filepath.Join(dir, "file"))
if err != nil {
log.Fatal(err)
}
return nil
}
func createFile(file string) error {
// Now create a file to check if we see that
err := ioutil.WriteFile(file, []byte("abc"), 0666)
if err != nil {
return err
}
time.Sleep(time.Second)
return nil
}
Hey there! I think I found a rather severe issue on operating systems that use the NonrecurseTree watcher like linux for recursive watches. The problem is that after you create and delete a folder several times, no subsequent events for that folder will be logged anymore.
I think that the underlying problem is that the non-recursive tree implementation only adds directories to watch and never deletes them from the internal tree structure (except if you call Stop() manually from the outside). This can be seen in the
func (t *nonrecursiveTree) dispatch(c <-chan EventInfo)
function: https://github.com/rjeczalik/notify/blob/e2a77dcc14cf6732bfa4c361554f27dc696d5d79/tree_nonrecursive.go#L67-L75Here only 'Create' events are forwarded to the internal handler that adds new watchpoints. However, when a directory is actually removed and recreated, the underlying inotify watch does not work anymore and the tree implementation does not re-add it because it thinks the folder is still being watched, because there is a node for it already in the internal in-memory tree. This means that all subsequent events are lost for this folder and it is essentially unwatched until you restart the complete watcher.
I guess this could be also the reason for some other issues like #190 or https://github.com/syncthing/syncthing/issues/7198
The problem can be reproduced on linux with this snippet: