Notification of when the mount is ready

[mmjconolly]

Hi, Is there a recommended way to know if the mount is ready once squashfuse starts?

In an application, I'm fork()ing out to squashfuse (with the -f option) then trying to use calls like statfs/getmntent in a polling loop to check if the mount point is ready to use, so that I don't race ahead and find an empty mountpoint.

Should I skip forking out to squashfuse and just call functions like sqfs_ll_mount directly?

Thanks Matthew

[mmjconolly]

Here is an example of how I try to detect the mount

#define FUSE_SUPER_MAGIC 0x65735546

static bool is_squashfuse_mount(const std::string& dir) {
        // Check if the filesystem backing 'dir' is of type 'fuse'.
        struct statfs fs;
        if (0 != statfs(dir.c_str(), &fs)) {
                std::cerr << "statfs:" << strerror(errno) << std::endl;
                return false;
        }
        if (fs.f_type != FUSE_SUPER_MAGIC)
                return false;

        // Check if 'dir' is really a mountpoint
        FILE *fd = setmntent("/proc/mounts", "re");
        if (!fd) {
                std::cerr << "setmntent:" << strerror(errno) << std::endl;
                return false;
        }
        struct mntent me;
        char buf[1024];
        while (getmntent_r(fd, &me, (char*)&buf, sizeof(buf)) != NULL) {
                // Check that squashfuse asked for the mount entry to be created,
                // I've seen the names 'squashfuse' or 'lt-squashfuse' so far
                if (me.mnt_dir == dir && strstr(me.mnt_fsname, "squashfuse") != NULL) {
                        endmntent(fd);
                        return true;
                }
        }
        endmntent(fd);
        return false;
}

[chipturner]

For xarexec (our main use for squasfuse_ll), we use polling to find a file expected inside the filesystem.

https://github.com/facebookincubator/xar/blob/b061e8b434da11d392c4b5c1d1bab72d16f9773b/xar/XarExecFuse.cpp#L462-L471

and

https://github.com/facebookincubator/xar/blob/b061e8b434da11d392c4b5c1d1bab72d16f9773b/xar/XarExecFuse.cpp#L139-L157

and finally

https://github.com/facebookincubator/xar/blob/b061e8b434da11d392c4b5c1d1bab72d16f9773b/xar/XarLinux.cpp#L55-L58

This works well for us in practice. Directly using/linking squashfuse might work but we've found the above to be robust (plus the mount persists after your process exits which, for us, combined with idle timeout, is ideal).

[vasi]

Yeah, the FUSE API is a bit grotty, I wouldn't rely on calling functions in the FUSE layer and having them do what you want. (Calling lower-level squashfuse functions is fine.)

If you were going to invoke squashfuse and receive some sort of notification, what side-channel would you want?

[mmjconolly]

A message on a pipe could work

• parent makes a pipe
• parent forks, gives the writeable pipe fd number on the command line
• child puts a known message onto the pipe (when mount is ready)
• parent reads read-side of the pipe to find known message (to be told mount is ready)

To play with it, I've written out some toy code (with no error handling):

$ gcc -o fake-squashfuse fake-squashfuse.c
$ gcc -o parent parent.c
$ export PATH=$PATH:$PWD
$ ./parent
child write: : Success
parent read: : Success
parent: mount message recieved! 

parent.c:

#include <stdlib.h>
#include <stdio.h>
#include <sys/types.h>
#include <unistd.h>
#include <string.h>

int main() {
        int fildes[2];
        pipe(fildes);

        pid_t pid = fork();
        if (pid == 0) {
                char file_to_close[5];
                snprintf(file_to_close, 5, "%d", fildes[1]);
                execlp("fake-squashfuse", "squashfuse", "--pipe-notify", file_to_close, "-f", NULL);
                perror("exec: ");
                exit(1);
        }

        char buf[10];
        read(fildes[0], buf, 10);
        perror("parent read: ");

        if (strncmp("mounted", buf, strlen("mounted")) == 0) {
                printf("parent: mount message recieved!\n");
                // do stuff with squashfuse mount!
                exit(0);
        }

        return 1;
}

fake-squashfuse.c:

#include <stdlib.h>
#include <stdio.h>
#include <sys/types.h>
#include <unistd.h>
#include <string.h>

int main(int argc, char* argv[]) {
        if (argc < 3)
                // usage: fake-squashfuse --pipe-notify pipe_to_close
                exit(1);

        int fd = atoi(argv[2]);

        // do_the_mounting();

        // Notify parent process that the mount is ready
        char* msg = "mounted";
        write(fd, msg, strlen(msg));
        perror("child write: ");

        close(fd);
        return 0;
}

I think a shell scripting user could use it too if we could provide a named pipe from mkfifo instead of just fd numbers

Does this approach seem ok?

[vasi]

Cute, I don't mind that! Can I see the WIP?

[hallyn]

This would sure be nice...

[DrDaveD]

Apptainer could use this too. It is also polling currently.

[ariel-miculas]

puzzlefs has this implemented It uses a named pipe for notification when mounting in the foreground and an anonymous pipe when mounting in the background, so the process which mounts puzzlefs only returns after the mountpoint is available. Since the xarexec already waits for the squashfuse_ll process to finish (and probably other users as well), I could implement the same logic for squashfuse as well, i.e. exit the process only after the mountpoint is ready. Of course, I could also implement a pipe for both the foreground and background cases, whichever makes most sense. @DrDaveD how does Apptainer launch and wait for the squasfuse process?

[DrDaveD]

Apptainer polls every 25 milliseconds looking for the mountpoint to show up in /proc/self/mountinfo. Actually it uses the same code for other fuse programs as well so if a special solution was made for squashfuse_ll I'm not sure it would be worth changing Apptainer to use it for just one.

[hallyn]

If the solution were available for squashfuse_ll, at least our user which mounts overlays of squashfuse (atomfs, and stacker, through atomfs) would use it.