unknownbrackets / maxcso

Fast cso compressor
ISC License
390 stars 23 forks source link

Create a repo with a database of ISOs that are have issues after compressing them #49

Closed KOLANICH closed 3 years ago

KOLANICH commented 3 years ago

Some isos are known to have issues after compressing them, such as

It may make sense to detect if an iso is affected by the issue and take countermeasures, such as non-compressing sectors affecting certain files and/or ranges of bytes within them at all or compressing them using a more performant algo.

unknownbrackets commented 3 years ago

The issues you're thinking of are bugs in some CFW that have been fixed already in the latest ProCFW (and possibly others), as far as I know. This is covered in the README. Especially on emulators, but also on CFW with the fix, you don't really need countermeasures.

The CFW bug was this: games would read large chunks of data (such as 64KB) from the disc at once. This had a particular timing (call it X + 32 Y for 64KB) when reading as an ISO or when reading from a UMD. However, the CSO code would break the read up into slices, which would have timing more like 32 X + 32 * Y.

In those timings, X represents the upfront cost of any memory stick read operation, and Y represents the cost of transferring about 2KB of data.

X was not a small amount of time. So in some games, reading this way caused the total time taken (the result of the formula) to be MUCH longer than it would've ever been uncompressed. This was fixed - all the compressed data is now read in one single operation now. People had theorized that it was due to time spent decompressing on the CPU, but they were simply wrong. It was the per IO operation overhead.

Such a table would also be confusing, because within an emulator like PPSSPP or PCSX2, the usage of a CSO is transparent. Except any bugs in the emulator, game behavior and timing doesn't even differ at all between ISO and CSO, and emulated cycles are unaffected.

Lastly, CSO is a format and as noted above - is even used by PS2 emulators. If a table of games affected by this bug is needed, it seems like that table should be associated with the CFWs that still have the bugs, rather than within maxcso, since its mission statement is simply to create good CSO files that exactly represent the original data at best compression.

-[Unknown]

KOLANICH commented 3 years ago

Thank you for the extended answer!