Files added to a new subdirectory of the watched directory don't get an Event

[erikd]

I'm on Linux and I'm interested in getting notifications of new files in a directory tree so I'm using the following predicate:

fileAddedPredicate :: Event -> Bool
fileAddedPredicate ev =
    case ev of
        Added _ _ isDir -> not isDir
        Modified {} -> False
        Removed {} -> False
        Unknown {} -> False

Now if I set up to get notifications on a directory "foo" that starts off empty, and I add a file to a subdirectory like "foo/bar/baz", I do not get a notification for that file. I already have a second test for the test suite that currently hangs, because the test expects a file added Event that never occurs.

My test looks like:

    it "recursive sourceFileChanges" $ withSystemTempDirectory "source-file-changes" $ \root -> do
        putStrLn "asdasdasda"
        let actions =
                [ ("foo", Just "goodbye")
                , ("bar/foo", Just "cruel")
                , ("bar/baz/qux", Just "world")
                , ("bar/baz/qux", Nothing)
                , ("bar/foor", Nothing)
                , ("foo", Nothing)
                ]
            go (path, mcontents) = do
                liftIO $ threadDelay 1000000
                liftIO $ case mcontents of
                    Nothing -> removeFile (root </> path)
                    Just contents -> do
                        case splitPath (root </> path) of
                         [] -> error "Unexpected empty list"
                         [_] -> pure ()
                            xs -> createDirectoryIfMissing True (joinPath $ init xs)
                        writeFile (root </> path) contents
                mnext <- await
                case mnext of
                    Nothing -> error "Unexpected empty"
                    Just event -> liftIO $ eventPath event `shouldBe` path
            settings =
                setPredicate fileAddedPredicate
                    . setRelative True
                    . setRecursive True
                    $ mkFileChangeSettings root
        Data.Acquire.with (acquireSourceFileChanges settings) $ \src ->
            runConduit $ src .| mapM_ go actions
        return () :: IO () -- force the type

and fileAddedPredicate is listed above. This is a bug right?

I had an initial idea that whenever a new directory is created FS.watchDirChan should terminate and be called again recursively, but that would open the possibility of race conditions. That makes it seem to me like this is a bug in the fsnotify package. Thoughts?

[snoyberg]

My initial reaction is that this is in fact a problem in fsnotify, I'd recommend trying to put together a repro against that library instead and see if the bug exists there too.

On Sun, Jul 8, 2018, 9:21 AM Erik de Castro Lopo notifications@github.com wrote:

I'm on Linux and I'm interested in getting notifications of new files in a directory tree so I'm using the following predicate:

fileAddedPredicate :: Event -> Bool fileAddedPredicate ev = case ev of Added isDir -> not isDir Modified {} -> False Removed {} -> False Unknown {} -> False

Now if I set up to get notifications on a directory "foo" that starts off empty, and I add a file to a subdirectory like "foo/bar/baz", I do not get a notification for that file. I already have a second test for the test suite that currently hangs, because the test expects a file added Event that never occurs.

My test looks like:

it "recursive sourceFileChanges" $ withSystemTempDirectory "source-file-changes" $ \root -> do
    putStrLn "asdasdasda"
    let actions =
            [ ("foo", Just "goodbye")
            , ("bar/foo", Just "cruel")
            , ("bar/baz/qux", Just "world")
            , ("bar/baz/qux", Nothing)
            , ("bar/foor", Nothing)
            , ("foo", Nothing)
            ]
        go (path, mcontents) = do
            liftIO $ threadDelay 1000000
            liftIO $ case mcontents of
                Nothing -> removeFile (root </> path)
                Just contents -> do
                    case splitPath (root </> path) of
                        [_] -> pure ()
                        xs -> createDirectoryIfMissing True (joinPath $ init xs)
                    writeFile (root </> path) contents
            mnext <- await
            case mnext of
                Nothing -> error "Unexpected empty"
                Just event -> liftIO $ eventPath event `shouldBe` path
        settings =
            setPredicate fileAddedPredicate
                . setRelative True
                . setRecursive False
                $ mkFileChangeSettings root
    Data.Acquire.with (acquireSourceFileChanges settings) $ \src ->
        runConduit $ src .| mapM_ go actions
    return () :: IO () -- force the type

and fileAddedPredicate is listed above. This is a bug right?

I had an initial idea that whenever a new directory is created FS.watchDirChan should terminate and be called again recursively, but that would open the possibility of race conditions. That makes it seem to me like this is a bug in the fsnotify package. Thoughts?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/fpco/fsnotify-conduit/issues/3, or mute the thread https://github.com/notifications/unsubscribe-auth/AADBB88lESMppoHknebfr7CwXvcCpwbxks5uEaUGgaJpZM4VGnEZ .

[erikd]

As an experiment, I tried swicthing to the polling version. This helped in that one of the expected events got through, but then it hung.

That's when I remember this text from the top of the Control.Concurrent.Chan documentation:

The channels are implemented with MVars and therefore inherit all the caveats that apply to MVars (possibility of races, deadlocks etc). The stm (software transactional memory) library has a more robust implementation of channels called TChans.

I remember it, because it was me who added it.

It looks like fsnotify-conduit just uses whatever channels are provided by fsnotify so I do need to mess with that package.

[erikd]

And not surprisingly, the fsnotofy test suite does not cover this case.

[erikd]

This bug is actually an instance of this: https://github.com/haskell-fswatch/hfsnotify/issues/54

Closing this.