has207 / psxtract-2

Psxtract by Hykem
GNU General Public License v3.0
14 stars 2 forks source link

Vib-Ribbon (SCES-02873) - Checksum doesn't match Redump #4

Open Rot-gut opened 1 year ago

Rot-gut commented 1 year ago

Firstly, I greatly appreciate the effort to improve this tool to recognize multi-track bins, as I thought it was abandoned.

While it's understood that the multi-track bins will have mismatched checksums due to re-conversion of audio, I'm still getting a main data track mismatch from Redump for Vib-Ribbon.

Redump Data track bin MD5: e116bccbc0828e45d7b3f19234cd3670 http://redump.org/disc/3474/

My DATA TRACK.BIN result MD5: 2875C53260C6DBD598A97C89423ED512

C:\RipAgain\SCES02873>"C:\psxtract-2\psxtract.exe" "C:\RipAgain\SCES02873\EBOOT.PBP" "C:\RipAgain\SCES02873\DOCUMENT.DAT" "C:\RipAgain\SCES02873\KEYS.BIN" Decrypting DOCUMENT.DAT... PGD: Invalid 0x80 MAC hash! DOCUMENT.DAT successfully decrypted! Saving as DOCUMENT_DEC.DAT...

Using PGD key: redacted

Unpacking PBP C:\RipAgain\SCES02873\EBOOT.PBP... [0] 944 bytes | PARAM.SFO [1] 11541 bytes | ICON0.PNG [2] 0 bytes | ICON1.PMF [3] 38089 bytes | PIC0.PNG [4] 2651 bytes | PIC1.PNG [5] 0 bytes | SND0.AT3 [6] 12271 bytes | DATA.PSP [7] 25073598 bytes | DATA.PSAR Successfully unpacked C:\RipAgain\SCES02873\EBOOT.PBP!

Single disc game detected!

Found STARTDAT offset: 0x017e6350 Saving STARTDAT as STARTDAT.BIN...

Decrypting ISO header... ISO header successfully decrypted! Saving as ISO_HEADER_0.BIN...

ISO disc: SCES_02873 ISO title: Vib-Ribbon

Found special data offset: 0x017e8bee Decrypting special data... Special data successfully decrypted! Saving as SPECIAL_DATA.BIN...

Found unknown data offset: 0x017e6240 Decrypting unknown data... Unknown data successfully decrypted! Saving as UNKNOWN_DATA.BIN...

Offset 1m:40s:0f at 0428 Offset 4m:19s:0f at 0432 seeking to 100000 + 536dc0 (636dc0) Extracting audio track 2 (11925 sectors, 2630784 bytes) Offset 4m:19s:0f at 0432 Offset 6m:53s:0f at 043c seeking to 100000 + 7b90c0 (8b90c0) Extracting audio track 3 (11550 sectors, 2548224 bytes) Offset 6m:53s:0f at 043c Offset 9m:25s:0f at 0446 seeking to 100000 + a27140 (b27140) Extracting audio track 4 (11400 sectors, 2515200 bytes) Offset 9m:25s:0f at 0446 Offset 12m:0s:0f at 0450 seeking to 100000 + c8d0c0 (d8d0c0) Extracting audio track 5 (11625 sectors, 2564736 bytes) Offset 12m:0s:0f at 0450 Offset 14m:47s:0f at 045a seeking to 100000 + eff1c0 (fff1c0) Extracting audio track 6 (12525 sectors, 2763264 bytes) Offset 14m:47s:0f at 045a Offset 17m:20s:0f at 0464 seeking to 100000 + 11a1a40 (12a1a40) Extracting audio track 7 (11475 sectors, 2531712 bytes) Offset 17m:20s:0f at 0464 Offset 20m:22s:66f at 0414 seeking to 100000 + 140ba40 (150ba40) Extracting audio track 8 (13716 sectors, 2992512 bytes) 7 audio tracks extracted to ATRAC3 Attempting to convert from ATRAC3 to WAV, this may take awhile...

C:\psxtract-2\at3tool.exe -d "D01 TRACK02.AT3" "D01 TRACK02.WAV" Decoding 132 kbps (ATRAC3) Decoded Bytes = 2630400 Bytes@6850frames(ave=384bytes)

C:\psxtract-2\at3tool.exe -d "D01 TRACK03.AT3" "D01 TRACK03.WAV" Decoding 132 kbps (ATRAC3) Decoded Bytes = 2547840 Bytes@6635frames(ave=384bytes)

C:\psxtract-2\at3tool.exe -d "D01 TRACK04.AT3" "D01 TRACK04.WAV" Decoding 132 kbps (ATRAC3) Decoded Bytes = 2514816 Bytes@6549frames(ave=384bytes)

C:\psxtract-2\at3tool.exe -d "D01 TRACK05.AT3" "D01 TRACK05.WAV" Decoding 132 kbps (ATRAC3) Decoded Bytes = 2564352 Bytes@6678frames(ave=384bytes)

C:\psxtract-2\at3tool.exe -d "D01 TRACK06.AT3" "D01 TRACK06.WAV" Decoding 132 kbps (ATRAC3) Decoded Bytes = 2762880 Bytes@7195frames(ave=384bytes)

C:\psxtract-2\at3tool.exe -d "D01 TRACK07.AT3" "D01 TRACK07.WAV" Decoding 132 kbps (ATRAC3) Decoded Bytes = 2531328 Bytes@6592frames(ave=384bytes)

C:\psxtract-2\at3tool.exe -d "D01 TRACK08.AT3" "D01 TRACK08.WAV" Decoding 132 kbps (ATRAC3) Decoded Bytes = 2992128 Bytes@7792frames(ave=384bytes)

7 audio tracks converted to WAV Need to pad 348372 bytes 7 audio tracks converted to BIN

Building the data track... ISO offset 100000 ..... Read 536dc0 bytes, wrote 10d4f00 bytes Data track successfully reconstructed! Saving as ISO.BIN...

Offset 0m:0s:0f at 041e Offset 1m:40s:0f at 0428 Converting the final image to BIN/CUE... Patching ECC/EDC data... Processing 7350 sectors Encountered unknown mode! This is probably not a proper image.

Generating CUE file... adding DATA TRACK.BIN.ISO adding D01 TRACK02.BIN adding D01 TRACK03.BIN adding D01 TRACK04.BIN adding D01 TRACK05.BIN adding D01 TRACK06.BIN adding D01 TRACK07.BIN adding D01 TRACK08.BIN Disc successfully converted to BIN/CUE format!

has207 commented 1 year ago

Looking into it...

has207 commented 1 year ago

Okay this is interesting, there's a few failures here. The entry in redump you want to be looking at is actually http://redump.org/disc/17774/, the one you were linking is the JP version. And the track file is actually TEMP/DATA_TRACK.BIN.ISO not TEMP/DATA_TRACK.BIN -- that's the initially dumped track before it's fixed, the .ISO file is the finalized track1 as it gets written to the BIN. So if we look at the md5 hashes for the correct files they do match up, BUT the resulting .BIN file is somehow borked, at least it seems to be the wrong size and Duckstation isn't able to properly md5 the tracks, let me investigate more...

has207 commented 1 year ago

Okay I think I see the issue, this game uses some very unusual pregap values, generally every other game I've seen uses 2 seconds gaps between tracks but this one just kind of goes nuts with using random pregaps, and the psxtract code has some hardcoded expectations that are causing the final .BIN file to be pretty broken. Let me see if it's going to be possible to infer these values on the fly and do the right thing, but as kind of a workaround what you can do is grab the CUE file from redump, and then take individual BIN files from the TEMP directory (remembering to use DATA_TRACK.BIN.ISO for track 1) and combine them manually. I should have some kind of fix in psxtract for this hopefully soon though

has207 commented 1 year ago

Well this keeps getting more and more interesting, seems that Sony also assumed 2 second gaps when making the EBOOT so their track listing is actually wrong and doesn't account for the actual pregaps in the audio. This makes me think this EBOOT might actually be a bit off in terms of when the audio starts playing in the game vs when it should start playing. On the other hand the 11 second pregap on Track2 on disc just seems weird as well, I don't know why there should be such a long silence on that track, though I'm not familiar with the game and don't know when it actually plays. In any case, it doesn't look like it's possible to reproduce the CUE file exactly, but it should be possible to fix the issue with data track length that causes the MD5 mismatch (and likely breaks the resulting image).

has207 commented 1 year ago

I pushed some changes that introduce some initial support for this, it's not working correctly yet though. I've also noticed previous audio handling code might have had some issues too so need to do more tweaking and testing. Unfortunately it seems that Sony might have messed up with this game as the CUE entries they have in the EBOOT do not account for the variable pregaps for the audio tracks so my guess is they will play a bit wrong, and I've also discovered there might be some similar inconsistencies in other games as well. It's not fatal as the music starting a few frames sooner or later is probably not a big deal but it makes for a challenge reconstructing the audio layout and especially verifying it across a large number of games and tracks.

has207 commented 1 year ago

Initial fix is committed, and I've compared PSOne releases with games that have unusual pregap values based on redump cue data. Here's what's likely to be the full list of currently affected titles:

It will take a bit of time to test, as I think there may be a few titles in there that will reveal additional problems. Will make a new release after checking them.

has207 commented 1 year ago

I have a fix close to working, I had to basically hard code pregap values for the affected games as there is no way to reconstruct them on the fly. Found some other issues while I was in there as well, so next release should hopefully be a big improvement across the board. Testing any changes is a real challenge with this so it's been taking some time to get it right.

mr-blackmore commented 6 months ago

I also have a list of games, which failed to verify checksum with Redump: Dezaemon Plus (SLPS-00335) Disney-Pixar Toy Story Racer (SLES-03396) Disneys Atlantis - Det Forsvundne Rige (SCES-03534) Galaxy Fight - Universal Warriors (SLPS-00138) Kaisoku Tenshi - The Rapid Angel (SLPS-01553) Mickey's Wild Adventure (SCES-00163) Oddworld - Abe's Exoddus (Disc 2) (SLES-11480) Resident Evil - Director's Cut (SLES-00969) Street Fighter Alpha - Warriors' Dreams (SLES-00199) Street Skater (SLES-01759) Tekken (SCES-00005) Tekken 2 (SLUS-00213) Twisted Metal (SCES-00061)

There's also Driver (SLES-01816), but I suspect that has more to do with Ubisoft, as they bought the series in 2006, so they actually made small changes, like replacing any mentions of GT Interactive with themselves as publisher.

has207 commented 6 months ago

@mr-blackmore thanks for the list, I'll check them out