dwarfs 0.7.3 doesn't use section index properly

[tpwrules]

Filesystem initialization times for a ~45GiB .dwarfs image on spinning rust (created with 0.7.3), doing echo 3 | sudo tee /proc/sys/vm/drop_caches before each trial:

version 0.7.2: 207.8ms, 240.5ms, 250.3ms, and reports "read 8018 blocks and 103798996 bytes of metadata"

version 0.7.3: 33.87s, 13.05s, 24.502s, "read 8018 blocks and 103798996 bytes of metadata"

It seems a regression was introduced which causes each section header to be read off the disk anyway during mount, negating significantly the advantages of the section index. The variance in the 0.7.3 times is probably ZFS caching in action; it's hard to fully empty those.

I would really appreciate a 0.7.x series fix as this is quite debilitating for mounting large images, and 45GiB is rather small to me.

[mhx]

Thanks for your report. I can reproduce this and I think I know what the problem is. I'll look into this as soon as possible (very likely in the next two days) and will definitely release a fix.

[mhx]

$ hyperfine -p 'dd of=test.dwarfs oflag=nocache conv=notrunc,fdatasync count=0' -w 1 -L dwarfs ./dwarfs-0.7.2-Linux-x86_64/sbin/dwarfs,./dwarfs-0.7.3-Linux-x86_64/sbin/dwarfs,./dwarfs '{dwarfs} test.dwarfs mnt; umount mnt'
Benchmark 1: ./dwarfs-0.7.2-Linux-x86_64/sbin/dwarfs test.dwarfs mnt; umount mnt
  Time (mean ± σ):      14.1 ms ±   1.8 ms    [User: 8.0 ms, System: 2.0 ms]
  Range (min … max):    11.8 ms …  21.3 ms    121 runs

Benchmark 2: ./dwarfs-0.7.3-Linux-x86_64/sbin/dwarfs test.dwarfs mnt; umount mnt
  Time (mean ± σ):      3.709 s ±  0.032 s    [User: 0.019 s, System: 0.442 s]
  Range (min … max):    3.668 s …  3.791 s    10 runs

Benchmark 3: ./dwarfs test.dwarfs mnt; umount mnt
  Time (mean ± σ):      13.4 ms ±   2.5 ms    [User: 4.0 ms, System: 4.8 ms]
  Range (min … max):    10.5 ms …  35.1 ms    210 runs

Summary
  ./dwarfs test.dwarfs mnt; umount mnt ran
    1.05 ± 0.24 times faster than ./dwarfs-0.7.2-Linux-x86_64/sbin/dwarfs test.dwarfs mnt; umount mnt
  276.82 ± 51.82 times faster than ./dwarfs-0.7.3-Linux-x86_64/sbin/dwarfs test.dwarfs mnt; umount mnt

[mhx]

Fixed in v0.7.4.