fallocate issue

[changyp6]

exfat linux doesn't support fallocate When running command fallocate -l 1G test1G command failed with output

fallocate -l 1G test1g
fallocate: fallocate failed: Operation not supported

[arter97]

Unfortunately, this is the intended behavior.

Unlike NTFS, exFAT doesn't support sparse files, which renders fallocate broken or useless(zero'ing out all ranges, which takes a lot of time).

EDIT: Update below

[changyp6]

@arter97 I'm not familar with the defination of fallocate. My understanding is that fallocate just pre-allocate a continous block of storage for a large file. If on exFAT, fallocate just allocates a large file block without zero'ing out all ranges, is this behaviour still called fallocate ?

[2259361174]

Unfortunately, this is the intended behavior.

Unlike NTFS, exFAT doesn't support sparse files, which renders fallocate broken or useless(zero'ing out all ranges, which takes a lot of time).

We just want to use fallocate to occupy free space with minimal overhead, to reduce exFAT fragmentation. while writing several video files constantly to the SD card at same time. If we don't use fallocate the free space for every file, then all of the video files will be stored to interleaved clusters, and huge video file will be in many pieces. then if one of the video file is deleted, it will create many small free spaces and high fragmentation.

One of the ideas is to use fallocate to occupy (eg. 256MB) size for each of the video file, but without real data written to SD card when we call fallocate, then we assume every file will be stored to contiguous 256MB space, and there will be no fragmentation at all. even if we finish writing the file and found it is only 180MB, we can ftruncate the file to 180MB and the left (256-180=76 MB) is still a very big free space and not fragmentation.

We can use ftruncate() or lseek() to create a 256MB file but it will write a lot of data to SD card but not just simplify "mark the space" in exFAT table without writing data.

VFAT in Linux can support fallocate() so we hope exfat can support it also.

We don't need to create any Holes on exfat. just want contiguous file on SD card.

any ideas ?

thanks for your help.

[arter97]

Unfortunately, this is the intended behavior. Unlike NTFS, exFAT doesn't support sparse files, which renders fallocate broken or useless(zero'ing out all ranges, which takes a lot of time).

We just want to use fallocate to occupy free space with minimal overhead, to reduce exFAT fragmentation. while writing several video files constantly to the SD card at same time. If we don't use fallocate the free space for every file, then all of the video files will be stored to interleaved clusters, and huge video file will be in many pieces. then if one of the video file is deleted, it will create many small free spaces and high fragmentation.

One of the ideas is to use fallocate to occupy (eg. 256MB) size for each of the video file, but without real data written to SD card when we call fallocate, then we assume every file will be stored to contiguous 256MB space, and there will be no fragmentation at all. even if we finish writing the file and found it is only 180MB, we can ftruncate the file to 180MB and the left (256-180=76 MB) is still a very big free space and not fragmentation.

We can use ftruncate() or lseek() to create a 256MB file but it will write a lot of data to SD card but not just simplify "mark the space" in exFAT table without writing data.

VFAT in Linux can support fallocate() so we hope exfat can support it also.

We don't need to create any Holes on exfat. just want contiguous file on SD card.

any ideas ?

thanks for your help.

https://github.com/torvalds/linux/commit/b13bb33eacb7 https://github.com/torvalds/linux/commit/28016128d37a

Looks like exFAT needs similar implementation. I suggest writing an email detailing the needs to fallocate to: Namjae Jeon namjae.jeon@samsung.com, Cc: linux-fsdevel@vger.kernel.org

Given that exFAT driver is also made by Namjae, this could be done.

[Thor-x86]

This command is working for me

fallocate -xl 1G test1g

For some reason -x flag which means "allocate in POSIX way" is working on non-Linux filesystem including NTFS-3G

[tsingakbar]

This command is working for me

fallocate -xl 1G test1g

For some reason -x flag which means "allocate in POSIX way" is working on non-Linux filesystem including NTFS-3G

"the posix way" will fallback to something like plain zero man 2 write for un-supported filesystem, which is slow.