Some questions about compression parameters

[crashGG]

The feature in cso format v2 is very practical,(when the length of a compressed block is >= block_size, the block must not be compressed.), which directly avoids It saves some unnecessary overhead of system resources on some data that is difficult to compress. Can introduce more parameters to set a threshold for this function, such as 95%? That is, the size of the compressed data block is greater than 95%, which will not compress the data block, but only store it. This can greatly speed up the decompression, as well as the cpu overhead.

[unknownbrackets]

There is such an option already:

   --orig-cost=N    Allow uncompressed to increase block size by N% at most

For example, if you used --block=2048 --orig-cost=1 then any compression that resulted in 2029 or more bytes would be considered "worse" than the original uncompressed data, and would not be compressed.

Note that CSOv2 was designed just as an experiment, but this option also works for ZSO and CSOv1. With that said, if you are using CSOv2, you may also consider this setting:

   --lz4-cost=N     Allow lz4 to increase block size by N% at most (cso2 only)

Which is essentially the same rules, but for picking lz4 over zlib. For example, using --block=2048 --lz4-cost=1 would pick an lz4 block at 1001 bytes even if zlib produced one that was 982 bytes. It's basically how much compression you're willing to sacrifice for lz4.

-[Unknown]

[crashGG]

Thank you for your answer, I am not a native English speaker, so I am still a little unclear in some logical terms. If I want to use libdeflate to compress, use 2048 for the data block, when the compressed data block is larger than 1952, it will not be compressed, only stored, should such parameters be used? --use-libdeflate --block=2048 --orig-cost=96 And these parameters also apply to CSOv1 ?

[unknownbrackets]

The --orig-cost option is a percentage, not bytes. If you want 1952, that is about 95% of 2048, so you would use --orig-cost=4.69. This is because 4.69 / 100 * 2048 = ~96.05, therefore 96 bytes at most will be wasted.

If you only want to use libdeflate, you would have to specify --only-libdeflate --block=2048 --orig-cost=4.69. If you want to allow it alongside the defaults, --use-libdeflate is correct.

Yes, all of these parameters work for CSOv1 as well.

-[Unknown]

[crashGG]

Thank you for your answer,further questions:If I want to use libdeflate as “performance” to compress, use lz4brute as “speed” , use 2048 for the data block, when the compressed data block size is larger than 95%, it will not be compressed, only stored,and when the compressed data block size of libdeflate and lz4brute differs within 2%, use lz4brute, and use libdeflate when it exceeds 2%. should such parameters be used?

--use-libdeflate --use-lz4brute --block=2048 --orig-cost=5 --lz4-cost=2

And because this mixed encoding method is only applicable to csov2, it is necessary to add the parameter --format=cso2 ?

[unknownbrackets]

Correct:

--format=cso2 --use-libdeflate --use-lz4brute --block=2048 --orig-cost=5 --lz4-cost=2

If you don't care about compression time, you can also add --use-zopfli. It will be very slow but zlib may get a bit smaller. Decompression time will not change, only compression time.

-[Unknown]

[crashGG]

Encounter an issue, when using --format=cso2 parameter to compress the cso v2, the newest ppsspp report :iso corrupt,and the game fmv broken 。 default cso1 is ok.

[unknownbrackets]

PPSSPP doesn't support CSOv2 or ZSO. As I mentioned - they're experimental. Also, any device powerful enough to run PPSSPP is not going to benefit much from lz4 or leaving blocks uncompressed - CSO is very fast, and PPSSPP (and its CSO support) is well optimized.

Some PSP CFW had performance issues with CSO files, but it was a bug in older CFW versions and has been fixed. PPSSPP doesn't have that bug.

Generally, if you're on a device with 1000 Mhz or higher, your best bet is to use cso1 and maximize compression. If you're using a device slower than 1000 Mhz, it may be better to leave some blocks uncompressed... but PPSSPP won't run very well anyway, so it won't matter.

-[Unknown]

[crashGG]

Another question: If set a large block size , will it increase the latency of reading? For example, when the block size is set to 16M, if the requested data is at the end of the block, will it wait to decompress the entire block to read this data?

[unknownbrackets]

Yes, the entire block must be decompressed to read from the end of the block. I don't really recommend block sizes larger than 32K or 64K at the most, as usually the compression benefits are small and it indeed affects the latency.

-[Unknown]

[crashGG]

Thanks for the recommend value, I just gave an extreme example for convenience. Through repeated testing, the high affordable value of most images are between 4k-16k. After more than that, the benefits is not great.

[unknownbrackets]

Right, well, DEFLATE doesn't really allow large backreferences so the benefits tend to drop off after that, unless you get some good Huffman symbols.

-[Unknown]