Horribly fragmented files

[colorimeter]

Files are created terribly fragmented on btrfs, maybe because of forced -noprealloc, but it makes gocryptfs unusably slow.

Update: it happens on ext4 too.

160 extents:

$ dd if=/dev/zero of=plain/zero bs=1M count=10
10+0 records in
10+0 records out
10485760 bytes (10 MB, 10 MiB) copied, 0.158727 s, 66.1 MB/s
$ filefrag crypt/1MPpzFXZXawqttKZtOFO9A 
crypt/1MPpzFXZXawqttKZtOFO9A: 160 extents found

doing the same over the same btrfs filesystem with encfs: 1 extent:

$ dd if=/dev/zero of=plain/zero bs=1M count=10
10+0 records in
10+0 records out
10485760 bytes (10 MB, 10 MiB) copied, 0.17905 s, 58.6 MB/s
$ filefrag crypt/e8DCLUQYahZecPzLAvQxg529
e8DCLUQYahZecPzLAvQxg529: 1 extent found

And the BTRFS fs itself without encryption handles the task well:

$ dd if=/dev/zero of=zero bs=1M count=10
10+0 records in
10+0 records out
10485760 bytes (10 MB, 10 MiB) copied, 0.0162497 s, 645 MB/s
$ filefrag zero
zero: 1 extent found

[alex3d]

@colorimeter you can fix excessive fragmentation by passing -noprealloc command line switch. Without -noprealloc, I've got 30-80 fragments on ext4 (file created by dd if=/dev/zero of=plain/zero bs=1M count=10). With -noprealloc, only 1 fragment.

[colorimeter]

@alex3d: interesting, but -noprealloc is always forced for BTRFS, so there is no way to improve the allocation there.

[alex3d]

@colorimeter: No, actually -noprealloc is not forced for BTRFS, despite misleading console message. Try to pass this option manually.

[colorimeter]

@alex3d : wow, you are right! What a misleading message! Thank you.

[rfjakob]

Fixed via https://github.com/rfjakob/gocryptfs/pull/830/commits/841784124281024c503eb863f02d09a255d446ae . Now btrfs actually causes -noprealloc to be enabled, like the message says.