File projection

[kivikakk]

Fixes #6. Or it will, because it's still WIP.

[chrisd8088]

Hey @kivikakk -- I thought I should mention that I stuck the fd member into the projfs_event structure on the guess that we might be able to use it pass an open file descriptor up to the callback, which could then write to that descriptor to fill in its contents.

struct projfs_event {
        ...
    int fd;             /* file descriptor for projection */
};

This may not be viable in practice, of course -- it was a "Hail Mary pass" into the future! :-)

[kivikakk]

on the guess that we might be able to use it pass an open file descriptor up to the callback, which could then write to that descriptor to fill in its contents

Yes, thank you! I'm running with that direction, or running toward that direction anyway.

[kivikakk]

@chrisd8088 There's more work to be done, but basic file projection does work now. See following (MirrorProvider debug output interleaved with commands):

Mounting /home/kivikakk/TestRoot
Virtualization instance started successfully
Press Enter to end the instance
$
$ find TestRoot
TestRoot
TestRoot/.mirror
TestRoot/.mirror/lower
TestRoot/.mirror/config
OnEnumerateDirectory(0, '.', 45804, find)
TestRoot/src
TestRoot/src/hello
OnEnumerateDirectory(0, 'hello', 45804, find)
TestRoot/src/hello/extra
OnEnumerateDirectory(0, 'hello/extra', 45804, find)
TestRoot/src/xyz
$ cat TestRoot/src/xyz
OnGetFileStream(0, 'xyz', 128/91:0, 128/7:0, 45864, cat, 0x7FF41D13E9C0)
it is a good day

for mirroring

i guess
$ cat TestRoot/.mirror/lower/xyz
it is a good day

for mirroring

i guess
$

I need to do a lot more testing, and hopefully your projection tests might be able to find some bugs -- some strange things have come up while working on this -- but an initial review pass would be much appreciated to make sure I haven't missed anything obvious.

[kivikakk]

Summary so far of ops that need to actually project (hydrate) files:

• link — we can't let a user create a hard link to a placeholder file, otherwise they could access the placeholder through the link without triggering hydration
• create — I don't see a codepath (starting in the Linux VFS) that leads to create being called where the file does exist, but there's no guarantee made per se that it'll only be called when the target doesn't exist. (See fuse_create_open in fs/fuse/dir.c)
  • The concern is that we might get called without O_EXCL or O_TRUNC, meaning existing files can be opened without truncation — this necessitates hydration. Will it ever happen? I have nfi.
  • We ignore ENOENT encountered during hydration because there may be no corresponding placeholder in the lowerdir (i.e. we are creating a new file).
• open — this is obviously necessary.
  • Per above, ignore ENOENT in hydration (because we may be creating a new file).
• rename — we need to hydrate the file before moving it, otherwise we'll move a placeholder (and the provider won't be able to hydrate the moved placeholder, because the path won't match anything it knows about).
• truncate — we need to hydrate before truncation, as we could truncate to any length.

[chrisd8088]

This is great, thank you @kivikakk! I will test it out over the weekend and take a look through all the enhancements!

[kivikakk]