File changes not detected when run in a symlinked directory

[tammersaleh]

I'm likely configuring modd wrong, but I can't seem to get my prep: command to run other than during initial startup.

Here's my modd.conf:

*.foo {
    prep: echo -e "\n\nCHANGED: @mods\n\n"
}

Started modd in one terminal. Ran these shell commands in another:

$ touch file.foo
$ echo content >> file.foo
$ cat file.foo
content

...and this was the output I saw on the modd terminal during those commands:

$ ./modd --debug
15:39:24: prep: echo -e "\n\nCHANGED: \n\n"

CHANGED:

>> done (1.453ms)
15:39:34: notify.Create: "/private/tmp/test/file.foo"
15:39:34: Delta:
Added: [/private/tmp/test/file.foo]
Deleted: []
Changed: []
15:39:40: notify.Write: "/private/tmp/test/file.foo"
15:39:40: Delta:
Added: []
Deleted: []
Changed: [/private/tmp/test/file.foo]

The first CHANGED output is from the startup. Interesting that modd correctly identified the file changes, but didn't run the associated prep: command. Is the modd.conf incorrect?

[cortesi]

Hmm... I'm unable to reproduce this, even using the exact same config file. Could you please try this with a recent master of modd? There have been a number of dependency updates that may be related to this.

The proximate cause of the error here is pretty clear - the path in the Changed list should be be relative to the current directory. The absolute paths we see here don't match the pattern. If this is a corner case in the notification subsystem, I can correct it by applying a path normalisation step in the matcher.

[tammersaleh]

Your tip helped me corner this - it's because the directory in question is a symlink (/tmp is a symlink to /private/tmp). So I think that's a bug in modd, but now I know how to work around it. I'll update the title to match.

(also, for reference, I'm running the 0.4 release from the releases page)

[sampsyo]

I'm running into the same problem here—it took me a while to track down, but it is indeed because cwd has a symlinked directory somewhere in the path. Everything works normally if I cd to the "real path" to the directory. This is on macOS High Sierra, if that helps.

[cortesi]

So, I've now spent some hours digging into this, and there are a number of issues with symlinks.

• On startup, we traverse all watched trees to do an initial event notification. If we follow symlinks here, it's impossible in the general case to prevent loops.
• We receive change notifications as absolute paths, with symlinks resolved. To follow symlinks, we would need to reverse this by cataloguing symlinks in our tree and matching them against the notification path. This means that more than one symlink in our tree could match a given notification, which leads to ambiguity.
• There are a whole slew of issues to do with symlinks-within-symlinks - it's completely unclear to me that it's possible to handle this in a principled, consistent way.
• The behaviour of notification backends differ - it appears that watching an enclosing directory containing a symlink will give events within the symlink on some platforms, but not on others.

At this point, I'm leaning towards just stating clearly in the docs that we don't follow "inner" symlinks, and that if you want them to be monitored you'll have to list them explicitly in the modd.conf. If you explicitly list a symlink in your modd.conf, we will follow the symlink and monitor the end directory as if it was explicitly specified.

Thoughts?

[sampsyo]

Good points. I do agree that needing explicit configuration for "inner" symlinks does seem like a reasonable trade-off.

For "outer" symlinks, in the path to the top-level watch directory, could it be as simple as normalizing this path into a symlink-free absolute path first before matching it against the OS-provided paths in the change events, which are also absolute?

[cortesi]

Yes, that's the plan. From the user's perspective this should be totally transparent, and it avoids all of the complications.

[cortesi]

I've just committed a pretty comprehensive re-assessment of how we handle file paths, and we now have a consistent way to handle symlinks.

[sampsyo]

Wahoo; thank you!