Closed babam86 closed 6 years ago
Creating a tar archive from an archivemount mountpoint will (obviously) access the contents of each file in the mountpoint. The way that archivemount works, this leads to a random access operation into the mounted archive, extracting each file one by one in the order that your tar call wants to have it. Yes, that is slow. The only way to make it quicker though is to actually unpack the archive during mounting. But obviously, if that is the behaviour you want, you don't need a fuse filesystem at all. Just replace mount with "tar xzf archive.tgz" and unmount with "tar -czf" followed by "rm -rf".
Extraction / decompression by Archivemount should be much faster than compressing by tar.
Decompression is always much faster than compression.
Sure it is, but never as fast as accessing an uncompressed file on a native, kernel based filesystem, which is what you compared it to. But you know what? This is open source software! Make a fork and fix it if you think you can do better!
I can't do better than you.
My concern is why the read speed on Archivemount mountpoint is very slow.
Let's test again,
archivemount archive.tar.gz mountpoint
time tar czf test.tar.gz mountpoint
Compare with:
time gzip -dc archive.tar.gz | gzip -c > test.tar.gz
I'm using archivemount on uncompressed .tar archive. I'm extracting one single 6 gb file and the read speed is 500 kb/s on 2.6 GHz cpu and 100% cpu usage. I compiled from this commit - master branch (Mon Apr 20 13:31:35 2020 +0200) 78c306538065de9b14f48cfc2024f50f843d3b29
and it's getting worse:
root@storage mnt# pv Dropbox.rar >/dev/null 312MiB 0:10:49 [ 252kiB/s] [> ] 4% ETA 3:27:16
It's sequential access, single file, uncompressed archive. Here is how fast this is using tar:
root@storage tape# time tar xf test.tar Dropbox.rar -O |pv >/dev/null
6.16GiB 0:00:19 [ 330MiB/s] [ <=> ]
real 0m19.059s
user 0m2.127s
sys 0m21.988s
(well typically it takes more time - about a minute, but this is after tar file got into zfs cache)
Again using archivemount (after big part of the archive is in ZFS cache):
root@storage mnt# pv Dropbox.rar >/dev/null
246MiB 0:06:08 [ 318kiB/s] [> ] 3% ETA 2:30:42
And getting slower over time.
OS: CentOS 7.9 Libarchive version: libarchive-3.5.1 Fuse version: 2.9.2-11.el7
compile comand:
# lib archive is installed in /usr/local/libarchive
export LDFLAGS=-L/usr/local/libarchive/lib
export CPPFLAGS=-I/usr/local/libarchive/include
./configure --prefix=/usr/local/archivemount
make --quiet -j 4
make install
archivemount archive.tar.gz mountpoint time tar czvf test.tar.gz mountpoint
Compare with:
tar xvf archive.tar.gz -C dir time tar czvf test.tar.gz dir
Creating archives from Archivemount mountpoint is very slow.
Thanks.