Closed xavierd closed 2 years ago
Leaving the old name in the placeholder is by design. Updates to the backing store are outside ProjFS's control, so it has to work on the principle that local file system operations, like rename, will only affect the local file system. It leaves the old name in the placeholder so that data recalls will continue to work in a case like this (assume foo.txt starts as a non-hydrated placeholder):
mv .\foo.txt .\bar.txt
cat .\bar.txt
<expect to see contents that were originally in foo.txt>
If the provider does indeed reflect local file system operations into the backing store immediately, then it is the provider's responsibility to update the projection accordingly. In the rename example, the provider should call PrjDeleteFile
, as you did, to clear the now-stale placeholder.
In the case where a rename moves the file into a non-placeholder directory, ProjFS detects that the destination is not a placeholder directory and hydrates the data before allowing the rename. The provider would not call PrjDeleteFile
in this case.
Thanks for getting back to me so quickly! A couple of questions below:
If the provider does indeed reflect local file system operations into the backing store immediately, then it is the provider's responsibility to update the projection accordingly.
Isn't this racy by design? Since notifications are sent after the local filesystem has been updated, the rename may be visible to other process before the backing store had a chance to call PrjDeleteFile
. Or is there some locking in ProjFS preventing operations on files/directory whose notifications haven't completed?
In the case where a rename moves the file into a non-placeholder directory, ProjFS detects that the destination is not a placeholder directory and hydrates the data before allowing the rename. The provider would not call PrjDeleteFile in this case.
Interesting, I'm not seeing the destination file being hydrated prior to the rename, which makes cat
still fail:
% fsutil.exe reparsepoint query .\TARGETS
Reparse Tag Value : 0x9000001c
Tag value: Microsoft
Tag value: Directory
Reparse Data Length: 0x146
Reparse Data:
0000: 02 00 00 00 00 00 00 00 4a 4c b4 33 5d b7 5a 4c ........JL.3].ZL
0010: b2 08 ff 25 f4 f2 a6 ff 00 00 00 00 00 00 00 00 ...%............
0020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0030: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0040: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0050: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0060: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0070: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0080: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0090: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0100: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0110: 00 00 00 00 00 00 00 00 2c 00 66 00 62 00 63 00 ........,.f.b.c.
0120: 6f 00 64 00 65 00 5c 00 65 00 64 00 65 00 6e 00 o.d.e.\.e.d.e.n.
0130: 5c 00 66 00 73 00 5c 00 54 00 41 00 52 00 47 00 \.f.s.\.T.A.R.G.
0140: 45 00 54 00 53 00 E.T.S.
% mkdir bar
Directory: C:\open\fbsource2\fbcode\eden\fs
Mode LastWriteTime Length Name
---- ------------- ------ ----
d----- 7/30/2021 4:30 PM bar
% fsutil.exe reparsepoint query bar
Error: The file or directory is not a reparse point.
% mv .\TARGETS bar
% fsutil.exe reparsepoint query bar
Error: The file or directory is not a reparse point.
% fsutil.exe reparsepoint query bar/TARGETS
Reparse Tag Value : 0x9000001c
Tag value: Microsoft
Tag value: Directory
Reparse Data Length: 0x146
Reparse Data:
0000: 02 00 00 00 08 00 00 00 4a 4c b4 33 5d b7 5a 4c ........JL.3].ZL
0010: b2 08 ff 25 f4 f2 a6 ff 00 00 00 00 00 00 00 00 ...%............
0020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0030: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0040: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0050: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0060: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0070: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0080: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0090: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0100: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0110: 00 00 00 00 00 00 00 00 2c 00 66 00 62 00 63 00 ........,.f.b.c.
0120: 6f 00 64 00 65 00 5c 00 65 00 64 00 65 00 6e 00 o.d.e.\.e.d.e.n.
0130: 5c 00 66 00 73 00 5c 00 54 00 41 00 52 00 47 00 \.f.s.\.T.A.R.G.
0140: 45 00 54 00 53 00 E.T.S.
% cat .\bar\TARGETS
cat : An internal error occurred.
At line:1 char:1
+ cat .\bar\TARGETS
+ ~~~~~~~~~~~~~~~~~
+ CategoryInfo : ReadError: (C:\open\fbsourc...\fs\bar\TARGETS:String) [Get-Content], IOException
+ FullyQualifiedErrorId : GetContentReaderIOError,Microsoft.PowerShell.Commands.GetContentCommand
Isn't this racy by design? Since notifications are sent after the local filesystem has been updated, the rename may be visible to other process before the backing store had a chance to call
PrjDeleteFile
.
Yes, I suppose so. But the current version of ProjFS wasn't designed to have the backing store be tightly in sync with the disk state. We designed it in tandem with VFS For Git, so it was designed around a model where either the backing store or the view are updated intentionally, such as when the user does a 'git push' or 'git checkout', respectively. The notifications are mainly to give the provider a way to track what's happened in the virtualization root so it can reason about what to do when it is asked to update the view.
When I said that rename into a non-placeholder directory hydrates the file first, that was an oversimplification. Also I'd forgotten some things and had to go back and re-read the code :-) Sorry about that. What really happens is that if ProjFS detects that the destination isn't a descendant of a virtualization root (or is a descendant of a different one), it returns STATUS_NOT_SAME_DEVICE. This makes the rename code, such as the MoveFile API, turn the rename into a copy followed by a delete. Or more precisely, and what the file system would see: hOld = open(old_path); read(hOld); hNew = create(new_path); write(hNew); delete(old_path);
. So the result in the destination is a full file. But in your case, you've just created a new directory beneath the virtualization root. ProjFS doesn't do anything special in this case because the provider should still be able to get the data. But since you update your backing store immediately, and the GetFileData uses the (old) path in the reparse point, you're unable to find the data for the file.
Hello,
I work on EdenFS (https://github.com/facebookexperimental/eden) and one of the bug I'm currently looking at is the surprising behavior of ProjectedFS when renaming a placeholder file:
In the logs of EdenFS, I can see that PRJ_GET_FILE_DATA_CB was called with:
fbcode\\eden\\fs\\TARGETS
, ie: the path prior to the rename, which makes for some fun behavior:To fix this this, I've tried calling
PrjDeleteFile
after rename to clear the placeholder which subsequent reads would recreate by using PRJ_GET_PLACEHOLDER_INFO_CB, but this doesn't work when the file is renamed to a directory that isn't a placeholder (like one thatmkdir
just created).From what I can see, this really feels like a bug in ProjectedFS as the placeholder should be updated with its new filename. In the case where this isn't a bug, what is the proper way to handle renaming placeholders?