sarah-walker-pcem / pcem

PCem
http://pcem-emulator.co.uk
GNU General Public License v2.0
1.55k stars 216 forks source link

Is there any reason fallocate(2) fallocate(3) isn't used to initiate a raw file on linux? #44

Open ghost opened 3 years ago

ghost commented 3 years ago

Describe the bug

fallocate(2) syscall/libc wrapper let's you allocate space immediately, by default(on ext4, didn't test other file systems) the file is zero filled/blocks are marked as not written to, and so you only have to use fallocate to claim the space on a disk, and write to the MBR header. Instead of taking a minute to initiate a 10-20GiB raw file on a samsung 860(even with smaller values, writting zeros is blasphemy), it will take couple ms even on a hdd.

To Reproduce

Just create a raw file image, takes longer than it really could.

Expected behavior

CLICK, here's your raw disk image. And if necessary it will write those couple bytes/kilobytes or necessary bits if any.

Screenshots

image

Host machine

Additional context

It is what kvm/qemu/virt-manager uses to initiate vm disk images, this is what steam on linux and windows(the equivalent syscall on windows is way slower) does to initiate files.

ghost commented 3 years ago

fallocate(fd_for_disk_image, FALLOC_FL_ZERO_RANGE, 0, length_in_bytes); on success returns zero, on -1 errno is set;

shermp commented 3 years ago

Portability is probably the biggest reason, a quick look indicates fallocate is linux specific and limited to certain filesystems as well.

Another concern could be security. Zeroing out the image file ensures an absolutely clean image, although I don't know how much of a concern this actually is.

The current image creation in PCem is essentially fopen(), fwrite(), fclose(), which basically works everywhere.

ghost commented 3 years ago

"limited to certain filesystems", thing is those are the most popular linux filesystems on desktop an ext4 is the defacto major one used there.

Even if you can then just fallback to default slow method.

Fallocate allows a flag which marks file data as one containning just zeros.

And even then I believe the major filesystem used will be ext4 on the linux desktop.

You can also do a crude method: If fallocate call fails, then you can fallback to default method. Fallocate isn't going away anytime soon. User can also generate a file with fallocate command instead.

And so you can do it like so:

#ifdef linux
if(fallocate(fd, FALLOC_FL_ZERO_RANGE, 0, size) == 0){
return;
}
#endif
fwrite loop

It is very unlikely you will encounter someone who uses resizerfs, nilfs, zfs(?) maybe but due to licensing I don't think any distros other than Oracle (Enterprise?) Linux ships with zfs by default.

ext4, xfs, btrfs which zero filled fallocate supports will encompass 99% of linux desktop users

ghost commented 3 years ago

Windows has it is own different implementation. BSDs have posix_fallocate() but there isn't an explicit mode for zeroing stuff out. os X/XI/XII doesn't seem to have a file preallocation syscall suprisingly.

shermp commented 3 years ago

Yeah, definitely a "have fun with #IFDEF's" situation...

(Raw) image creation is in wx-config.c if anyone is interested in implementing this.

ghost commented 3 years ago

46 needs some work/mild clean up @shermp