Modern file systems now have compression schemes that surpasses userland compression schemes in convenience and sometimes even on efficiency, rendering the current use of gzip inconvenient when such filesystems are used.
As a use case, BTRFS with zstd:15 gave me excellent results on compression, surpassing what gzip could do, with faster read access. At very least, I don't need to recompress the WARCball after a recording session.
Ideally it should be possible to allow the user to choose if they are want to use the warcio gzip support, or prefer to rely on the file system for such - preventing the user to uncompress the thing manually to take advantage of the file system compression.
Modern file systems now have compression schemes that surpasses userland compression schemes in convenience and sometimes even on efficiency, rendering the current use of
gzipinconvenient when such filesystems are used.As a use case,
BTRFSwithzstd:15gave me excellent results on compression, surpassing what gzip could do, with faster read access. At very least, I don't need to recompress theWARCballafter a recording session.Ideally it should be possible to allow the user to choose if they are want to use the
warciogzip support, or prefer to rely on the file system for such - preventing the user to uncompress the thing manually to take advantage of the file system compression.