unknownbrackets
commented
2 years ago There's probably not a significant reason to use Lizard in CSO files. Really, I see the LZ4 experiment in CSOv2 as a failure.
Originally, the motivation for adding LZ4 to ZSO was a performance problem in PSP CFW. Some games played using CSOs were slow. Even after LZ4 was implemented, this problem wasn't fixed - those same games were still slow.
It turned out the performance issue was caused by a bug: the CSO/ZSO code was turning a read of 64KB from the UMD into 32 reads of 2KB each. The overhead per each read was the performance problem, not the tiny cost of decompression. When this problem was fixed, the performance issues for CSO (and ZSO) were solved. LZ4 itself didn't move the needle.
The next reason to use LZ4 is that it can sometimes compress better, and is still cheap to decompress (unlike i.e. LZMA.) However, even this is also counterproductive: on PSP CFW, kernel is very memory constrained. Including the LZ4 code burns memory budget that would honestly be better utilized supporting 4 KB block size CSOs, which would compress better and decompress about as quickly.
On desktops, where you have plenty of RAM and decompression performance, LZ4 doesn't lose you much RAM - but you're still better off with larger block sizes and zlib, or a better compression method.
Lizard doesn't really have different tradeoffs than LZ4, it's just a bit closer to zlib.
A more interesting direction is zstd: still a fairly simple implementation, but faster AND better compression ratios than zlib. However, I've done some tests and the compression ratio gain is not that huge. In theory having some pre-shared dictionary or common dictionary for the CSO might help more, though.
In my tests, I found that implementations of compressed ISO reading (.gz on PCSX2, CHD, etc.) can be pretty slow, and sometimes limp along using bulky and complex caches to paper over their performance problems. But CSO, as well as its implementation in most emulators, is fast. The difference between CSO zlib and CSO LZ4 (or Lizard) would be comparatively small.
Long story short: I don't think there's any advantage to Lizard for maxcso specifically.
-[Unknown]
malvarenga123
Hi @unknownbrackets, just recently I found out about https://github.com/mcmilk/7-Zip-zstd which supports a few more compression methods over standard 7-zip. It seems that Lizard is supposed to be the successor of LZ4 and LZ5 with really fast decompression speeds. Would there be any advantage implementing it in maxcso?