Closed solarispika closed 2 months ago
Thank you very much for reporting the issue. This is interesting, I did not expect it to happen and the Windows documentation does not clarify about it, this was implemented assuming that the event FILE_ACTION_RENAMED_NEW_NAME
comes always right after a FILE_ACTION_RENAMED_OLD_NAME
event (we could have other events types in between though).
This is only fixable if and only if Windows reports the events in FIFO, if this is not the case this is a kernel bug and is un-fixable, so I'll assume that this is the case an implement it as a FIFO queue.
I find this article which might explain why such behavior can happen. Basically, the function is not designed for file monitoring but for file listing from the beginning.
The intended purpose of the ReadDirectoryChangesW function is to assist programs like Windows Explorer which display the contents of a directory. If something happens that results in a change to the directory listing, then it is reported by ReadDirectoryChangesW. In other words, ReadDirectoryChangesW tells you when the result of a FindFirstFile/FindNextFile loop changes. The intended usage pattern is doing a FindFirstFile/FindNextFile to collect all the directory entries, and then using the results from ReadDirectoryChangesW to update that collection incrementally.
I managed to fix the issue by ReadDirectoryChangesExW since it provides file id for comparing. However, the function needs newer OS and it only supports NTFS, this makes it less useful for this project.
Could you share the changes? It's possible to detect the OS version and the disk file system and use this particular more fine-grained version.
Sure, here it is. poc-fix-rename-issue.patch The patch contains debug logs and some experimental code, so don't be serious on it :-)
To use the patch, some macro is needed to set sdk version.
I use cmake for building, and the command line argument for configuration has -DCMAKE_CXX_FLAGS_INIT="/DWINVER=0x0A00 /D_WIN32_WINNT=0x0A00"
.
With this patch, I can't reproduce the problem.
Ohh I see, very interesting, thanks for sharing it, I'll try to provide the feature at least optionally. But as you can see, all these file system monitoring APIs are very messy and you'll encounter things like the one you encountered quite easily. efsw already tries to do more than most libraries to get it as close to "right" as possible.
I implemented something based on your fix. ReadDirectoryChangesExW
will be detected dynamically and used if available.
It looks like file systems which ReadDirectoryChangesExW doesn't support will fail to be watched on Windows versions that support the function. I think monitored filesystem should also be considered. Maybe we can try running ReadDirectoryChangesExW first and see if it fail or not.
Summary
efsw incorrectly reports move events when multiple file move operations occur simultaneously across different subdirectories within the monitored folder.
Expected behavior
efsw should correctly report the source and destination of each move event, even when multiple moves occur simultaneously.
Actual behavior
efsw sometimes reports incorrect source directories for move events, mixing up the origins of moved files.
Environment
Steps to reproduce
Create a folder structure:
Open three PowerShell windows:
Observe the output in Window 1 for incorrect move events
Monitoring script (Window 1)
Renaming script (Windows 2 and 3)
Example of incorrect output
Possible cause
Each rename operation is reported as two events,
FILE_ACTION_RENAMED_OLD_NAME
andFILE_ACTION_RENAMED_NEW_NAME
. efsw saves filename towatch->OldFileName
on handlingFILE_ACTION_RENAMED_OLD_NAME
and later uses it on handlingFILE_ACTION_RENAMED_NEW_NAME
to combine both events as aActions::Moved
event. However, multipleFILE_ACTION_RENAMED_OLD_NAME
might occur successively, before their correspondingFILE_ACTION_RENAMED_NEW_NAME
, thus one of remove operation's old name saved inwatch->OldFileName
is overwritten.https://github.com/SpartanJ/efsw/blob/0c70ed2cd069a18d534989e8ae1c25ee39cba1f1/src/efsw/FileWatcherWin32.cpp#L162-L164 https://github.com/SpartanJ/efsw/blob/0c70ed2cd069a18d534989e8ae1c25ee39cba1f1/src/efsw/FileWatcherWin32.cpp#L168-L212
Additional notes