Soul Reaver strange intermittent hang

[AndreasV81]

Soul Reaver NTSC at Xstation Firmware 1.1.0 hangs intermittently. Looks like a game or frame freeze. Strange thing though, that the music continues playing and that the issue is always resolved by opening the lid. The game continues instantly when the lid is opened, NOT closed. This happens sometimes before cutscenes but now in the drowned abbey all the time. This renders the game unplayable.

Easily reproducible: Teleport to the drowned abbey, switch to physical realm, run down the corridor, open the door and right before the door closes behind you the game freezes. Opening the ps1 lid then continues the game.

This might look like some trivial loading error but actually if you play the game in an already loaded room with the lid opened the game does not halt, it simply does not load new objects.

Tested 3 different sdcards, all formatted with the format tool recommended, all the same. Game hangs at the very same position. When i revisit other areas like "necropolis" or "nupraptors retreat" this issue does not happen at all.

Tested Xenogears and FF8, both NTSC, the intro FMVs play flawlessly, so i dont have any sdcard access problems when it gets critical, like during FMVs.

PU18 board in a 5552 pal console with ps1digital also installed. The software lid switch trigger implemented in ps1digital is not installed. Neither is psnee or any other modchip installed. I also desoldered, opened and deoxidized the lid switch internally as an intermittent contact was my first guess. Things did not change.

Can anyone test this please :)

[mathieulh]

I may have found the check in the NTSC-U version of soul reaver:

TEXT:800361BC slti $v0, $a0, 0x45 # 'E' TEXT:800361C0 bnez $v0, loc_800361DC TEXT:800361C4 li $a2, 0x400 TEXT:800361C8 addiu $v0, $a0, -0x44 TEXT:800361CC srl $v1, $v0, 31 TEXT:800361D0 addu $v0, $v1 TEXT:800361D4 sra $v0, 1 TEXT:800361D8 addiu $a0, $v0, 0x44 # 'D'

This would make the location at 0x3A1C4 in the ISO, where you would want to change 0x45 to a value that's not a valid BIOS region string (like 0x00 for example)

Unfortunately I cannot test this patch since I am already using a NTSC-U BIOS. Legacy of Kain - Soul Reaver (USA) Region check patch.zip

EDIT: I have included a patch attempt, you need to apply it using deltapatcher https://www.romhacking.net/utilities/704/

Of course you can also use an hex editor to manually patch the iso, I just made this for convenience. Please test and tell me if it works.

[mathieulh]

So, someone with an unmodded console attempted to reproduce the freeze, with and without my patch and couldn't.

Do you have a reliable trigger for it? He tried the door in the drowned abbey, that's the right symbol to teleport to, right?

[mathieulh]

Ok, I reproduced the crash using this symbol even with NTSC-U BIOS

https://imgur.com/orfRdxd

It seems to indeed be a mechacon check, not a BIOS one.

UPDATE: The patch I did does not work.

[p-b-p]

Here some more info regarding this crash:

Tested with Legacy of Kain: Soul Reaver NTSC-U (SLUS-00708 with verified redump md5):

1. PAL setup: SCPH-5552 on XStation at 2.0.2 with no other modifications - the crash is reproducible as expected.
2. PAL setup: SCPH-9002 on CD-ROM with MM3 modchip and US BIOS installed (SCPH-9001 Version 4.1 12/16/97 A) - the crash is reproducible.
3. PAL setup: SCPH-7002 on CD-ROM with UniROM 8.0.K installed on Pro Action Replay 2 - tested it many times and I wasn't able to reproduce this crash. Everything runs smoothly.

Was it possible that UniROM handles differently a mechacon (MC68HC05) checks?

[ramapcsx2]

Just as an information, games have an easy time telling the Mech and the BIOS chip region. You may assume that a region lockout routine has both chip's region strings and can use those as wanted (or demanded by regulation via Sony regional stuff). Additionally, there are games known to check for the presence of expansion ROM carts, one example being an NTSC-J Cardcaptor Sakura game, which does not boot if any ROM is attached, no matter the contents of that ROM.

Now add to that the coder's likely desire to hide their region checks, and it becomes really hard to guesstimate what goes wrong.

At this point, someone really needs to dissect the binary, otherwise we'll never know :p

[ramapcsx2]

I scrolled back a bit and found that @mathieulh already has found part of the check, which is indeed the BIOS chip region byte. This should say E, I or U, depending on the region of the game, of course. But this is just one source, there could be many more things.

A crazy dev could even find out the GPU clock speed by measuring a GPU counter, and tell a 50Hz GPU from a 60Hz one ;p

Edit: Ah, and the way to patch the region byte is probably not going to be replacing the byte the game looks for, but providing the correct one... Otherwise it's easy to mess up a check routine, if part of the source data is garbage.

[p-b-p]

Yes, but is still interesting why it works with PSIO and UniROM, but not on real CD-ROM or XStation. Maybe UniROM behave similarly as PSIO like mentioned here

So, in order to investigate further I've ordered a spare PU-18 NTSC-U board and the next test I plan is to replace the mechacon on the the XStation console and see if that will resolve this issue :)

[AndreasV81]

Thank you very much for diving into a 2 year old issue :) It would be really great if this could be resolved somehow.

[p-b-p]

I've played for a while with no$psx debugger and found a very interesting mark about this check :) Standing right before the door the game on the real hardware hangs. When I simulate it on no$psx you can see on the image below that CDROM sends 3xGetID commands... I asume that here things get broken because it receives 'E' from the PAL mech, but expects 'A' to continue. However, receiving 'I' also pass the check because as reported NTSC-J consoles don't have this issue.

Apart from that, back in the days when I played that title on my modded SCPH-9002 PAL console I'm pretty sure that the disk was NTSC-U version of the game and I don't remember having such random hang issues during the full gameplay. Guess I have to thank to my dump modchip which was constantly sending 'SCEE' while the console is turned on :D

UPDATE: by the way on that spot GetID commands are missing in the PAL version (SLES-01301) of the game.

[AndreasV81]

So the correct modchip would solve this issue?

[p-b-p]

So the correct modchip would solve this issue?

Probably :) I have to test it. Unfortunately the console is sold long time ago and don't remember what exactly was flashed on that chip. Only remember that it was 4 wire mod wired as shown here:

[ramapcsx2]

GetID is most certainly unusual in a scene where the game just has to load new data. You have definitely found the region check routine here :p

[AndreasV81]

I also have the ps1digital in that console with an ... i think onboard? psnee... is it possible to combine the "power ;)" of the two? that would be really nice :) Or somehow patch out the region check?

[AndreasV81]

The only modchips i had back then were all stealth modchips, which will not work in this case.

[p-b-p]

GetID is most certainly unusual in a scene where the game just has to load new data. You have definitely found the region check routine here :p

For sure :) I will try to debug it further with no$psx. Here is the responce:

CDROM Command: 1A - GetID
IRQ2 CDROM INT03(22) from CMD=1A
IRQ2 CDROM INT02(22,00,20,00,53,43,45,41) from CMD=1A

5th-8th byte: SCEx region (eg. ASCII "SCEE" = Europe) (0,0,0,0 = Unlicensed)

'53 43 45 41' => SCEA

But still I don't know why it works with UniROM as No$Unlock is applied when it boots the CD, but then the weird thing is:

"In the unlocked state, ReadN/ReadS are working for unlicensed CD-Rs, and for imported CDROMs from other regions (both without needing modchips). However there are some cases which may still cause problems: The GetID command (1Ah) does still identify the disc as being unlicensed."

It could be that region check routine get untriggered or break if GetID command return (0,0,0,0 = Unlicensed) as it looks for '45' E in the 8th byte, but gets 0 :)

[mathieulh]

My take on this is that Soul Reaver and a few other games will (indirectly) check the region string stored in mechacon registers (read from the disc wobble) and will trigger some lock mechanism if the last string = E (from the SCEE code expected by mechacon in PAL consoles).

The reason they let empty values pass the check is because DTL-H units (with the exception of the Net Yaroze which checks and stores the wobble but only checks a match for the first 3 characters) do not check the wobble at all and thus never store the SCEx string in mechacon registers, so triggering the security mechanism on units with empty mechacon registers would prevent legit developers from testing builds. I believe this is the same reason why they specifically check for the E string rather than only allow the A one. I am not sure why they didn't add a case for the I string. Perhaps unauthorized imports of US titles to Japan wasn't that much of an issue.

Modchipped units still need to pass the mechacon check so a value matching the mechacon region will be stored in registers if mechacon status is unlocked.

[AndreasV81]

@mathieulh does the pal version converted to ntsc, run equally fast as the original ntsc version? i patched the pal version and teleported to the abbey and it does not hang, yet it seems it runs faster than my pal-expectation...

[p-b-p]

@mathieulh does the pal version converted to ntsc, run equally fast as the original ntsc version? i patched the pal version and teleported to the abbey and it does not hang, yet it seems it runs faster than my pal-expectation...

@AndreasV81 Here is an info from UniROM usage. It should be the same as PAL2NTSC disk image patch. You will get a PAL game running at 60Hz (PAL60):

PAL/NTSC switch / region override Allows you to force the video mode of a game to PAL or NTSC.

PAL->NTSC will be the slightly higher PAL resolution with the slightly higher NTSC framerate. NTSC-PAL will be the slightly lower NTSC resolution with the slightly lower PAL framerate. There's a clear winner there.

[AndreasV81]

Thanks :) except the resolution of the game is identical just squashed between two black borders, like most ported games ;) I guess i will stick to the patched pal version until the ntsc version can be patched somehow. Maybe disassambling the game, removing all getid commands and recompiling? But i dont know if this is even possible or within reasonable effort... or would even be enough... Still thank you all for you effort so far :)

[mathieulh]

Wouldn't it work on Unirom because it uses the undocumented mechacon unlock command (aka backdoor)? (To force mechacon to unlock the drive regardless of the state of the wobble registers). I believe they use this in unirom by default to allow out of region games and backups on unmodified consoles.

If Unirom doesn't read the wobble when booting the disc, it would never contain the SCEE string in the registers.

[ramapcsx2]

So yeah, if the Mech is properly unlocked, it will report back the region byte of A, E or I in GetID. It's very hard to avoid "getting unlocked", if you have a modchip installed or with the xStation. Unlocking is part of the CD init routines that the Mech runs automatically. This is why even with Unirom, the Mech will be unlocked.. However, the backdoor hack might let the region check routine pass for whatever other reason.. or the PSX that got tested with Unirom did not have a modchip/xStation :)

[p-b-p]

Wouldn't it work on Unirom because it uses the undocumented mechacon unlock command (aka backdoor)? (To force mechacon to unlock the drive regardless of the state of the wobble registers). I believe they use this in unirom by default to allow out of region games and backups on unmodified consoles.

If Unirom doesn't read the wobble when booting the disc, it would never contain the SCEE string in the registers.

@mathieulh It works via UniROM! I've mentioned it here and here :)

[ramapcsx2]

So, it might work to somehow lock the Mech again, somehow after the game has booted :p

[p-b-p]

So yeah, if the Mech is properly unlocked, it will report back the region byte of A, E or I in GetID. It's very hard to avoid "getting unlocked", if you have a modchip installed or with the xStation. Unlocking is part of the CD init routines that the Mech runs automatically. This is why even with Unirom, the Mech will be unlocked.. However, the backdoor hack might let the region check routine pass for whatever other reason.. or the PSX that got tested with Unirom did not have a modchip/xStation :)

For sure the PAL SCPH-7002 unit I have tested with UniROM is without any modifications. Otherwise I will not waste time to investigate :)

Even tested a workflow with UniROM (no$unlock boot) which was then booting xStation last CD (Soul Reaver NTSC-U) fast. That didn't worked as expected and still the game hangs :D

[mathieulh]

So, it might work to somehow lock the Mech again, somehow after the game has booted :p

If you lock mechacon again the disc will not be read. Mechacon needs to be unlocked to read discs (even audio cds just set another unlock status).

To not trigger the protection, the mechacon wobble registers must NOT contain the SCEE string.

It works with Unirom because unirom uses the undocumented mechacon unlock backdoor sequence (which works on non Japanese mechacons) which sets mechacon into an unlock state without reading the wobble, thus the wobble strings are not stored in mechacon registers and the E string used for the compare is not present.

On xstation a way to bypass that issue would be to relock mechacon (there is a command that does this as far as I am aware of) from the main menu (if the registers do not contain SCEI) and issue the unlock backdoor command before booting a game.

Perhaps if you are concerned about compatibility issues of any kind (not that there should be) only perform this behavior on SCEE units on known titles using this protection (which are only Soul Reaver and Wipeout 3 (US) I think).

Another way might be to just patch executables in memory upon running/booting them. (Assuming you find a way to patch the check)

[ramapcsx2]

The idea is to look for a way to write over the 'E' byte in Mech RAM only :p

[mathieulh]

@mathieulh does the pal version converted to ntsc, run equally fast as the original ntsc version? i patched the pal version and teleported to the abbey and it does not hang, yet it seems it runs faster than my pal-expectation...

Yes, unfortunately that is expected because the PAL unit would run at either 59.2hz or 60.1hz (depending on the patch applied) whereas the NTSC units would run at 59.94hz, that is unless you use a Dual Frequency Oscillator mod.

[p-b-p]

@mathieulh I think UniROM is using these commands to unlock the mechacon. Guess it uses: "World wide" ;for Yaroze

[mathieulh]

The idea is to look for a way to write over the 'E' byte in Mech RAM only :p

I don't know if that's even possible. It wouldn't change the lock status if it was, but those registers contain the strings located in the wobble data so they are probably read only from the ps1 side (it would be a serious lapse in security if they allowed them to be overwritten), so the only way to technically overwrite them would be to issue the wobble read command again (like the modchip detection titles (MGS integral and co) do?

This would read the content of the wobble and copy it to those registers (I am unsure whether or not the string has to pass the compare check in mechacon for it to be copied). I also assume you would need to relock or reset mechacon somehow before reading the wobble again?

As long as you can relock and somehow wipe the registers that would be enough, you wouldn't actually need to write arbitrary data there, then you'd just need to unlock mechacon with the backdoor.

[mathieulh]

@mathieulh I think UniROM is using these commands to unlock the mechacon: https://problemkaputt.de/psxspx-cdrom-secret-unlock-commands.htm

Maybe use: "World wide" ;for Yaroze

Yes, that's exactly what it does, this is the unlock backdoor I was talking about. It is well known and documented since the mechacon rom has been dumped and reverse engineered.

About the Yaroze, unlike real debugging/testing units, it performs the wobble check but does not compare the SCEx wobble string against a hardcoded value (for region checks) like the consumer units do. As a result it does need legit ps1 discs but it will play out of region discs, the mechacon registers will just contain whatever string is present in the disc's wobble data.

Real debug units ignore/skip the wobble check entirely and therefore do not display a SCEx string at the bottom of the playstation logo during boot. As such the wobble string registers on these units are just always empty.

The SCEW region does exist in Net Yaroze discs wobble (it is indeed legit wobble data), but there is no specific SCEW "encoded" mechacon. As such no mechacon other than Yaroze (which ignores the SCEx string compare) and dev/debug/test units will read those discs.

[ramapcsx2]

The backdoor unlock does not set any "scex_string", it cannot do that.. The region in Mech RAM will have whatever landed there, very likely just "0,0,0,0" though, as nothing ever wrote to it. The string is at 0x01D0 to 0x01D3 in Mech RAM, and there's these test commands to write to RAM, but they are restricted to where you can write.. If this area is included though, it would probably work to write '0' to 0x01D3.

[ramapcsx2]

Ah, I'm sorry but too bad, writing to Mech RAM is not supported via test commands.. That was only for some registers.

[ramapcsx2]

There's apparently a bug in the processing of subcode Q byte 2 "index", where a malformed byte would have the Mech write to 0x0100h .. 0x01D1, so close! :p

[AndreasV81]

How does actually the Xstation alter or defeat region lockout? When it boots up it says scei instead of scee. And this would be read from the bios or am i mistaken? Isnt then the bios write protected? Or do you inject hardware-wise the proper flag to the bios. If so wouldnt it be possible with some leg-lifting and additional soldering to inject the same or similar data to the mechacon? Or to write to that region you mentioned and provoking a similar error that happens when opening the lid if the getid command is streamed?

[mathieulh]

How does actually the Xstation alter or defeat region lockout? When it boots up it says scei instead of scee. And this would be read from the bios or am i mistaken? Isnt then the bios write protected? Or do you inject hardware-wise the proper flag to the bios. If so wouldnt it be possible with some leg-lifting and additional soldering to inject the same or similar data to the mechacon? Or to write to that region you mentioned and provoking a similar error that happens when opening the lid if the getid command is streamed?

The xstation just spams all the different wobble data sequentially fast enough that one of them gets picked up by mechacon. Mechacon will ignore the non matching wobble data.

The menu "disc" is SCEI encoded because the Japanese bios enforces an additional region check.

[ramapcsx2]

Right, there is only one way to unlock the Mech with symbol data (ie properly, not by relying on the backdoor), and that involves sending it the SCEX symbols that it looks for. It will then only accept the symbols meant for its region. You cannot have it unlock on out of region symbols.

[p-b-p]

@AndreasV81

How does actually the Xstation alter or defeat region lockout? When it boots up it says scei instead of scee. And this would be read from the bios or am i mistaken? Isnt then the bios write protected? Or do you inject hardware-wise the proper flag to the bios. If so wouldnt it be possible with some leg-lifting and additional soldering to inject the same or similar data to the mechacon? Or to write to that region you mentioned and provoking a similar error that happens when opening the lid if the getid command is streamed?

For example here I can see SCEE when using UniROM as bootloader for XStation Soul Reaver CD. It is a PAL BIOS of SCPH-5552.

[ramapcsx2]

I think Unirom just displays "SCEE" no matter what the actual unlock string was.. But your PAL Mech can only be unlocked by "SCEE", so it's right.

Edit: To repeat that and clarify once more .. No software (game, ROM cart, etc) can unlock the Mech through the regular SCEX method. It only unlocks if a CD (or chip etc) sends it the SCEX string that it expects. So whenever you see Unirom enabling reads, that is through the backdoor method, which a game can detect easily, if it was looking for it. There will be no valid GetID result, for example.

[AndreasV81]

Is it posdible to patch out these getid commands with a hex editor? Just thinking because the pal version does not have any, at least not at the scene in the drowned abbey... or make use of the backdoor method as a specific per game option in xstation?

[ramapcsx2]

It is entirely possible and ideal to just develop a patch for the game, though how that looks, I wouldn't know. Has anyone looked for a classic patch yet btw? :p

Does using the backdoor method on a locked Mech fix the problem though? That's not clear to me atm.

[AndreasV81]

The pal2ntsc patch which @mathieulh made works fine, with ps1digital you can also stretch out the pal borders. I dont know of any other patches yet. I tried the following today: -put unirom in the same folder on the xstation sdcard as the game itself and started the unirom as disc1 and soulreaver ntsc as disc2 -got into unirom itself, chose to switch disks, opened and closed the lid, so the game would be started and pressed play -unirom said nocash unlock was activated -game started normally -game freezes at the same spot that we all have seen too many times so far;)

so at least running unirom from diskimage and doing the unlock does not work this way. but this may also be because i had to open and close the lid to switch disks, maybe it works differently with a flashed actionreplay cartridge. unfortunately i cannot test this.

[ramapcsx2]

No worries, I wouldn't expect using a CD-R with backdoor hack on a PAL console would actually pass this check fine. It would be a weird coincidence if it did. The fix here is to have a region lockout defeat patch for this game.

[p-b-p]

It is entirely possible and ideal to just develop a patch for the game, though how that looks, I wouldn't know. Has anyone looked for a classic patch yet btw? :p

Does using the backdoor method on a locked Mech fix the problem though? That's not clear to me atm.

As far as I know the classic patches for the NTSC game are to convert it to PAL, embedded gameshark codes, etc. Only for PAL version of the game there are paches to remove the LibCrypt protection which is not present in the NTSC version:

https://problemkaputt.de/psxspx-cdrom-protection-libcrypt.htm - see Example (Legacy of Kain) I have tested it and it is exactly as described :p

@ramapcsx2 I think when UniROM use the backdoor to unlock mechacon on a console without modchip and boot the game from real CD-ROM drive no matter if UniROM is executed via expansion cartridge, FreePSXBoot on memory card or disk swap method the protection is defeated as GetID returns: (0,0,0,0 = Unlicensed) instead SCEE

https://problemkaputt.de/psxspx-cdrom-status-commands.htm

@AndreasV81 using UniROM as 1st disk to boot the game as 2nd disk on XStation would not work as XStation overrides the backdoor unlock commands and unlocks the mechacon in the proper way and then GetID returns SCEE. Even to boot the UniROM CD first the mechacon has to be unlocked with SCEE by XStation.

[ramapcsx2]

Do you think CD-R + backdoor works or could you verify it? As I said, it would be a curious oversight if that passes the region check routine. It's possible, but weird, if so :)

[p-b-p]

Do you think CD-R + backdoor works or could you verify it? As I said, it would be a curious oversight if that passes the region check routine. It's possible, but weird, if so :)

This is what I have tested here -> Test case 3: SCPH-7002 PAL console (without modchip or any other modifications) + UniROM loaded from Pro Action Replay 2 + backup CD-R of Soul Reaver NTSC.

Anyway, to be 100% sure I will test that case again and in addition will try to see whether that would work if I boot the CD-R via UniROM loaded from memory card 2 (with FreePSXBoot).

[p-b-p]

I confirm that the game does not hang when Soul Reaver CD-R is booted via UniROM (installed on cartridge or memory card):

[p-b-p]

I could try to develop a patch for that check, but sadly I do not how to debug the GetID check with no$psx. If anyone have a clue from where I should start let me know :)

[AndreasV81]

I do not know if this is even possible.... but can the Xstation in theory use the backdoor method to boot images instead of the current method which rather resembles that of modchips?

It might be possible, that some regions are only available to write to from the parallel port ... Is that so?

[ramapcsx2]

Okay, sorry for not picking it up earlier. It happens when chats get long and varied :) So okay, they coded a really bad region check routine. I wonder whether it is any hard to come up with a defeat patch that can be applied to the image? It could be done on the xStation side, where a special case would be made for this ID when it runs on a PAL Mech, but oh gee, it is really not the place where it should be done..