Displayed storage and free space incorrect

[EverettPorter]

First of all, THANK YOU for this software. I've been using it for a couple of years, and it's been very reliable and extremely useful!

I just installed mergerfs-2.33.1-1.el7.x86_64.rpm on our Centos7 server. I didn't change the mount options in /etc/fstab, but with this version, the df -h shows incorrect values for the total storage and available free space. This worked correctly with the previous installed version (2.32xx).

This is the merged disk: PC_Merge 361T 356T 5.5T 99% /mnt/PC_Merge

And this one of the members (note free space) /dev/sdd 279T 123T 157T 44% /mnt/WD18Tx19-2021-9_PC

and the fstab line /mnt/WD18Tx19-2021-9_PC/PolyCast2:/mnt/HGST14TBx18-2/PolyCast4/PolyCast4:/mnt/WD18TBx18-1/PolyCast1-2:/mnt/HGST14TBx18/PolyCast3:/mnt/WD18Tx16-2021-9/PolyCast5:/mnt/WD18Tx16-2021-9/PolyCast6:/mnt/lsi_ext5.1/PolyCast7:/mnt/SG12TB/PC8 /mnt/PC_Merge fuse.mergerfs defaults,allow_other,ignorepponrename=true,use_ino,hard_remove,minfreespace=500G,category.create=mfs,category.search=all,fsname=PC_Merge,cache.files=auto-full,cache.writeback=true,dropcacheonclose=true,splice_write,threads=4 0 0

To Reproduce mount filesystem!

Steps to reproduce the behavior. List all steps to reproduce. All settings.

Please simplify the reproduction as much as possible.

• Unless it is dependenat on multiple branches, use a single branch
• Use standard tooling if possible (touch,truncate,rm,rmdir,ln,etc.)

Expected behavior

correct totals of used and free space

System information:

• OS, kernel version: uname -a Linux newserver.poly 3.10.0-957.el7.x86_64 #1 SMP Thu Nov 8 23:39:32 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
• mergerfs version: mergerfs -V mergerfs version: 2.33.1
• mergerfs settings
• List of drives, filesystems, & sizes:
  • df -h
  • devtmpfs 16G 0 16G 0% /dev tmpfs 16G 0 16G 0% /dev/shm tmpfs 16G 11M 16G 1% /run tmpfs 16G 0 16G 0% /sys/fs/cgroup /dev/sda5 25G 18G 7.3G 71% / /dev/sda1 1014M 243M 772M 24% /boot /dev/sda2 25G 701M 25G 3% /home /dev/sdb 246T 216T 31T 88% /mnt/WD18TBx18-1 /dev/sdh 230T 214T 16T 94% /mnt/WD18Tx16-2021-9 /dev/sdf 191T 169T 23T 89% /mnt/HGST14TBx18-2 /dev/sdg 153T 144T 9.1T 95% /mnt/SG12TB /dev/sde1 62T 62T 517G 100% /mnt/lsi_ext5.1 /dev/sde2 62T 59T 3.6T 95% /mnt/lsi_ext5.2 mergerfsPool 227T 202T 26T 89% /mnt/sambashare/SessMed /dev/sdd 279T 123T 157T 44% /mnt/WD18Tx19-2021-9_PC /dev/sdc 191T 179T 13T 94% /mnt/HGST14TBx18 //192.168.9.1/ftp 848G 313G 536G 37% /mnt/ftp tmpfs 3.1G 0 3.1G 0% /run/user/506 PC_Merge 361T 356T 5.5T 99% /mnt/PC_Merge
  • lsblk
  • NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sda 8:0 0 223.1G 0 disk ├─sda1 8:1 0 1G 0 part /boot ├─sda2 8:2 0 25G 0 part /home ├─sda3 8:3 0 15.6G 0 part [SWAP] ├─sda4 8:4 0 1K 0 part ├─sda5 8:5 0 25G 0 part / ├─sda6 8:6 0 1G 0 part ├─sda7 8:7 0 25G 0 part ├─sda8 8:8 0 15.6G 0 part [SWAP] └─sda9 8:9 0 25G 0 part sdb 8:16 0 245.6T 0 disk /mnt/WD18TBx18-1 sdc 8:32 0 191T 0 disk /mnt/HGST14TBx18 sdd 8:48 0 278.3T 0 disk /mnt/WD18Tx19-2021-9_PC sde 8:64 0 123.7T 0 disk ├─sde1 8:65 0 61.9T 0 part /mnt/lsi_ext5.1 └─sde2 8:66 0 61.9T 0 part /mnt/lsi_ext5.2 sdf 8:80 0 191T 0 disk /mnt/HGST14TBx18-2 sdg 8:96 0 152.8T 0 disk /mnt/SG12TB sdh 8:112 0 229.2T 0 disk /mnt/WD18Tx16-2021-9
• A strace of the application having a problem:
  • strace -fvTtt -s 256 -o /tmp/app.strace.txt <cmd>
  • strace -fvTtt -s 256 -o /tmp/app.strace.txt -p <appPID>
• strace of mergerfs while app tried to do it's thing:
  • strace -fvTtt -s 256 -p <mergerfsPID> -o /tmp/mergerfs.strace.txt

Additional context

Add any other context about the problem here.

[trapexit]

Without the traces or something that is reproduceable by me I'm a little stuck as this doesn't happen on my system. The calculation code didn't change between versions.

What happens if you only have a single branch? What are those mounts attributes (filesystem, mounted read only, etc.)?

[EverettPorter]

Thanks for the quick response. Downgraded to the last 2.32 version, and displayed storage was back to normal. Everything is read-write mounted, all volumes are xfs. I'm not experienced with traces, but happy to re-install 2.33 and run a trace if it helps.

[trapexit]

Yes. I need as much information as possible. All that stuff the template lists including a minimal reproduceable version of the issue. Since statfs is the aggregation of all the branches it would be useful to compare the differences with 1 branch, 2 branches, etc. mergerfs has options to change how statfs works so it's important to know all the machine's details related to that feature as well.

[EverettPorter]

Apologies for the delay in responding. It's a production system, and I've reinstalled 2.32 to get around the problem. I've just tried to reproduce on another system using Centos8, and don't yet see the same problem. I'll find some downtime soon to try and test again on the system above.

[trapexit]

There isn't any reason to have to do that. You can simply install the other version after booting, or build it yourself and use the executabl, and then just create a new mount on the same machine.