search

waltligon / orangefs

Official repository for PVFS/OrangeFS
Other
62 stars 7 forks source link

df (disk free) system utility reports wrong size on ubuntu 18.04 #83

Closed steven-varga closed 2 years ago

steven-varga commented 4 years ago

I am to report a misbehaviour on ubuntu 18.04 system provided df disk free utility, that it reports the wrong size of mounted pvfs. 

aggregate statistics:
---------------------------------------

    fs_id: 19730429
    total number of servers (meta and I/O): 1
    handles available (meta and I/O):       4611686018427387843
    handles total (meta and I/O):           4611686018427387904
    bytes available:                        3361294614528
    bytes total:                            3361890787328

But from shell df -h where /homedir is layered on top of a 3.1TB drive prints out 3.1GB as if the block size was wrong. 

/dev/md0                   3.3T  569M  3.1T   1% /mnt/ephemeral
tcp://master:3344/homedir  3.1G  568K  3.1G   1% /home

I tried this with v2.9.7 v2.9.8 and the master combined with two kernel version: v5.6.12 and v5.8.14 built in orangefs module. I copied massive data on the orangefs filesystem and md5 checksum them each time. Out of 10 checksum tests, no test failed, an unlikely event if something was wrong with the data stored on the server.

interestingly pydf utility reports the correct information:

root@master:~# pydf
Filesystem                 Size  Used Avail  Use%                                 Mounted on                 
tcp://master:3344/homedir 3131G  568M 3130G   0.0 [.............................] /home                      
/dev/md0                  3299G  569M 3130G   0.0 [.............................] /mnt/ephemeral

Has anyone noticed similar behaviour?

tomgreen66 commented 3 years ago

Hit similar problem. Seems to be related to changes in the kernel where page size of 4kb is now used for file system buffersize e.g frbuf. This ignores the 4mb buffer used for actual data transfer. Seems to impact the statfs. Maybe logic needed in the kernel module to handle different size depending on type of object. E.g torvalds/linux@9f8fd53cd05596f6792f769c9fd5fd2b0d624507

Have used the fuse method instead to get around the problem.

waltligon commented 3 years ago

Thanks for letting us know about this. We have tracked down the problem to an uninitialized variable in the kernel module. Luckily it was an easy fix. I don't think this change got into the release pull that just happened, but we will get it in ASAP and backport to earlier kernels. Again, thanks for letting us know.

boydwilson commented 2 years ago

This issue has been resolved and backported to the relevant kernel versions as of July 2021 (forgot to close the issue earlier, apologies for that).