gluster / glusterfs

Gluster Filesystem : Build your distributed storage in minutes
https://www.gluster.org
GNU General Public License v2.0
4.51k stars 1.07k forks source link

fallocate with flag FALLOC_FL_ZERO_RANGE failed #4313

Closed powerfooI closed 2 months ago

powerfooI commented 2 months ago

Description of problem:

One of our application runs in kubernetes, mounting a glusterfs volume which was set up with glustefs-kubernetes. The application uses syscall fallocate with flag FALLOC_FL_ZERO_RANGE to set zero a file storage, but the application failed here.

The exact command to reproduce the issue: We wrote a simple test file try_zero_range.cpp as follows,

#include <stdio.h>
#include <stdlib.h>
#include <fcntl.h>
#include <sys/syscall.h>
#include <unistd.h>
#include <errno.h>

#define FILE_SIZE 1 << 30

int main(int argc, char **argv)
{
    int fd;
    off_t offset = 0;
    size_t length = FILE_SIZE;

    char* file_name = argv[1];

    fd = open(file_name, O_RDWR | O_CREAT, 0644);
    if (fd == -1) {
        perror("open");
        exit(EXIT_FAILURE);
    }
    if (-1 == ::fallocate(fd, FALLOC_FL_ZERO_RANGE, offset, length)) {
        perror("fallocate");
        printf("fallocated failed, errno=%d\n", errno);
        exit(EXIT_FAILURE);
    }

    printf("File %s has been allocated with %ld bytes.\n", file_name, length);
    close(fd);
    return 0;
}

Then we compile and execute the program,

g++ try_zero_range.cpp
./a.out /path/to/glusterfs-mount/testfile

The full output of the command that failed:

fallocate: Operation not supported
fallocated failed, errno=95

Expected results:

File /path/to/glusterfs-mount/testfile has been allocated with 1073741824 bytes.
**Mandatory info:** **- The output of the `gluster volume info` command:** **- The output of the `gluster volume status` command:** image **- The output of the `gluster volume heal` command:** **- Provide logs present on following locations of client and server nodes - /var/log/glusterfs/** Sorry, the log cannot be accessed because we don't have permission. **- Is there any crash ? Provide the backtrace and coredump** The volume does not crash. **Additional info:**

- The operating system / glusterfs version:

uname -a: Linux node2 3.10.0-1160.el7.x86_64 #1 SMP Mon Oct 19 16:18:59 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux os version: CentOS Linux release 7.9.2009 (Core) glusterfs version: 7.1

Note: Please hide any confidential data which you don't want to share in public like IP address, file name, hostname or any other configuration

mohit84 commented 2 months ago

Are you able to run the program for non-gluster file? The flag "FALLOC_FL_ZERO_RANGE" does not support by all the file system, If filesystem does not support it it gives an error. If you will try to run the program without using flag it should work fine.

powerfooI commented 2 months ago

Yes, the program could work on non-glusterfs file. It wrote to /tmp/testfile which was not on gluster successfully.

mohit84 commented 2 months ago

Are you able to run a program without using the flag (FALLOC_FL_ZERO_RANGE) for glusterfs?

powerfooI commented 2 months ago

Yes, fallocate could work without FALLOC_FL_ZERO _RANGE flag. It seems that this flag is not supported well.

mohit84 commented 2 months ago

It seems the flag was implemented in fuse(2.31) so you would be able to use only the flag after the version(2.31). https://lore.kernel.org/all/20210615103357.GP26415@redhat.com/T/

powerfooI commented 2 months ago

Thank you so much! This information helps a lot. I could not find fuse code in version 2.31, while actually found the support for FALLOC_FL_ZERO_RANGE flag appears only after linux kernel v5.14 and the support was not patched backwards any older version kernel. My kernel is of version 3.10 which is too old.

mohit84 commented 2 months ago

Thanks for confirming the same.