bzip3 bigger dictionary, worser compression ratio

[lukypko]

lrzip-next Version

my version https://github.com/lukypko/lrzip-next/tree/files1

lrzip-next command line

space

What happened?

When I use a bzip3 compression, I would expect when block size is higher, then compressed file will be smaller, but opposite is true (which is wrong).

It is maybe not a bug report, I'm just try to think what happen actually. I'm trying to compress Windows 7 64bit c:/windows folder (its size is about 11GB)

Is it because dictionary is stored in each independently compressed block and larger dictionary does not bring a better compression ratio, it just holds more and more place?

lrzip-next-dev - is "my version which can process multiple files" from https://github.com/lukypko/lrzip-next/tree/files1

I can add more information if needed, I do not have enough free space to compress it and use just input tar file, but I guess it is not related to my changes.

lrzip-next-dev -vfUB --bzip3bs 2 -T  -R 9 -L 9 -vv -o windows.bz3.bzip3bs2.lrzip ~/sda1/Windows/
Total time: 00:54:42.47

-rw-rw-r-- 1 luky luky 1941994612 Feb  4 22:04 windows.bz3.bzip3bs2.lrzip
-rw-rw-r-- 1 luky luky    6749072 Feb  4 22:04 windows.bz3.bzip3bs2.lrzip.files

lrzip-next-dev -vfUB --bzip3bs 3 -T  -R 9 -L 9 -vv -o windows.bz3.bzip3bs3.lrzip ~/sda1/Windows/
Total time: 00:57:38.06

-rw-rw-r-- 1 luky luky 1942259724 Feb  4 19:58 windows.bz3.bzip3bs3.lrzip
-rw-rw-r-- 1 luky luky    6749072 Feb  4 19:58 windows.bz3.bzip3bs3.lrzip.files

lrzip-next-dev -vfUB --bzip3bs 5 -T  -R 9 -L 9 -vv -o windows.bz3.bzip3bs5.lrzip ~/sda1/Windows/
Total time: 01:00:04.82

-rw-rw-r-- 1 luky luky 1955217712 Feb  4 21:01 windows.bz3.bzip3bs5.lrzip
-rw-rw-r-- 1 luky luky    6749072 Feb  4 21:01 windows.bz3.bzip3bs5.lrzip.files

LZMA (just for a reference)

lrzip-next-dev -vfU -R 9 -L 9 -vv -o windows.lzma.lrzip ~/sda1/Windows/
Total time: 01:22:31.43

-rw-rw-r-- 1 luky luky 1691287496 Feb  4 18:48 windows.lzma.lrzip
-rw-rw-r-- 1 luky luky    6749072 Feb  4 18:48 windows.lzma.lrzip.files

What was expected behavior?

better compression with larger dictionary

Steps to reproduce

lazy

Relevant log output

No response

Please provide system details

x

Additional Context

No response

[pete4abw]

• bzip3 is best with text only files. If files are of binary type, you may not get the best results.
• bzip3bs is independent of level. Using -L# and --bzip3bs is meaningless unless you set -R#.
• If you use L9 bzip3bs would be 8.
• lzma-22.01 branch will reduce bzip3 block size as necessary, whats-next will not.
• I would test bzip3 alone and see what it does. If results are similar, report upstream.
• check with lrzip codebase, although bzip3 won't be available.

Marking as not a bug because you have forked and modified base code.

[pete4abw]

Done for now