/mnt/cache is not a mergerfs mount (but it is)

[hacktek]

/usr/bin/mergerfs -o category.create=lus,async_read=false,cache.files=partial -o dropcacheonclose=true,use_ino,minfreespace=0 -o xattr=nosys,statfs_ignore=ro,allow_other,umask=002,noatime /mnt/storage=RW:/mnt/storage1=RW /mnt/cache

/mnt/storage:/mnt/storage1 1967047816 181080404 1735917620 10% /mnt/cache

Manually calling it with python3 also doesn't work:

/usr/bin/python3 /usr/local/bin/mergerfs.balance /mnt/cache /mnt/cache is not a mergerfs mount

[trapexit]

Does the file /mnt/cache/.mergerfs exist? That's how it determines it.

[hacktek]

It doesn't. Question is, why doesn't it?

This was a simple mount I prepared some time ago with just /mnt/storage and I added /mnt/storage1 yesterday and meant to balance them. Can I just create that file manually?

EDIT:

actually it does exist, I typo'd before

ls -la /mnt/cache/.mergerfs -rwxrwxr-x 1 root root 0 May 1 23:28 /mnt/cache/.mergerfs

[trapexit]

And you've run balance as root?

[hacktek]

Yeah.

I just reworked the mount to add /mnt/storage0 and /mnt/storage1 and point that to /mnt/storage instead of /mnt/cache and that worked :/

Weird.

[trapexit]

I don't quite follow. The code simply recursively looks for .mergerfs because that is a pseudo file which holds the information it needs to work. You could simply strace the running of the program to see it searching.

[hacktek]

Yeah well it didn't work until I changed my mount settings for some reason.

Still, it did something very weird. Instead of balancing like it should have it actually moved everything from the most filled drive to the least filled drive until the most filled drive mount point had nothing, it never actually stopped. When there was nothing else to transfer it just output this and stopped:

Could not find file to transfer: exiting...
Branches within 2.0% range: 

du shows:

hacktek@xxx:~$ sudo du -sh /mnt/storage0
80M     /mnt/storage0
hacktek@xxx:~$ sudo du -sh /mnt/storage1
212G    /mnt/storage1

I ran it again right after and got the same thing:

$ sudo mergerfs.balance /mnt/storage
/mnt/storage is not a mergerfs mount

So no idea what the deal is. Current mount settings:

/usr/bin/mergerfs -o category.create=lus,async_read=false,cache.files=partial -o dropcacheonclose=true,use_ino,minfreespace=0 -o xattr=nosys,statfs_ignore=ro,allow_other,umask=002,noatime /mnt/storage0=RW:/mnt/storage1=RW /mnt/storage

EDIT: yeah any time I copy something into /mnt/storage0 and run balance it just copies it to /mnt/storage1.

/mnt/storage1 is on an iscsi mounted drive, maybe that has something to do with it?

[trapexit]

Still, it did something very weird. Instead of balancing like it should have it actually moved everything from the most filled drive to the least filled drive until the most filled drive mount point had nothing, it never actually stopped.

I don't understand. Balancing is moving from the most filled drive to the least filled drive. Till the percentage filled between the two is within whatever percentage set. What did it print after that?

[hacktek]

Nothing was printed after "Branches within 2.0% range:". I just re-did the paths and made the iscsi lun the same size as the local partition and that worked so who knows what the deal was.

[hacktek]

I'm reopening as I tried to balance again today and it failed. Here's the strace, seems to be failing getting attributes from .mergerfs?:

lstat("/mnt", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0 lstat("/mnt/storage", {st_mode=S_IFDIR|0775, st_size=4096, ...}) = 0 stat("/mnt/storage/.mergerfs", {st_mode=S_IFREG|0775, st_size=0, ...}) = 0 lgetxattr("/mnt/storage/.mergerfs", "user.mergerfs.version", 0x7ff0cc62c7f0, 64) = -1 EOPNOTSUPP (Operation not supported) write(1, "/mnt/storage is not a mergerfs m"..., 37/mnt/storage is not a mergerfs mount ) = 37 rt_sigaction(SIGINT, {sa_handler=SIG_DFL, sa_mask=[], sa_flags=SA_RESTORER, sa_restorer=0x7ff0cdd5ff20}, {sa_handler=0x62ffc0, sa_mask=[], sa_flags=SA_RESTORER, sa_restorer=0x7ff0cdd5ff20}, 8) = 0 sigaltstack(NULL, {ss_sp=0xe47670, ss_flags=0, ss_size=8192}) = 0 sigaltstack({ss_sp=NULL, ss_flags=SS_DISABLE, ss_size=0}, NULL) = 0 exit_group(1) = ? +++ exited with 1 +++

Some weirdness is happening here as I can ls the file but not cat it?

hacktek@xxx:~$ cat /mnt/storage/.mergerfs cat: /mnt/storage/.mergerfs: No such file or directory hacktek@xxx:~$ ls -la /mnt/storage/.mergerfs -rwxrwxr-x 1 root root 0 May 2 18:10 /mnt/storage/.mergerfs

[trapexit]

xattr=nosys

Don't do that. You've disable the ability to talk to mergerfs.

[trapexit]

https://github.com/trapexit/mergerfs#xattr

As it says in the docs... the runtime controls are via xattr calls. You've turned them off. All the tools rely on those to work. I missed that in the original post.

[trapexit]

I can add to the error message in the tools this fact but the intent is to change the runtime controls eventually to not use xattr.

[trapexit]

Have you tried changing the value?