DiscImageCreator can't dump RVT-R/NR with Plextors due to not knowing the end PSN from the PFI data

[ehw]

Version 20230606

Describe the bug DiscImageCreator can dump commercial Wii/GC discs by using the READ BUFF command in DVD Plextor drives. This method also works for RVT-R and NR discs but there's a caveat.

Due to unknown reasons at the moment, the PFI data returned by the drive is incomplete. All drives have a hard time with the PFI data in any software. The possible reason for this is that the sector data that has the DMI/PFI data stored is scrambled with a unique algorithm that's not DVD standard, so it can't properly descramble all the parts of the PFI.

The PFI contains the starting PSN (00030000) but not the ending PSN value. So DiscImageCreator doesn't know how big the NR/RVT-R discs are, which results in a "INVALID FIELD IN CDB" error. If you tell DiscImageCreator the size of the disc, bypassing the need of checking the PFI data, DIC will dump the discs just fine. So there needs to be an alternate solution to determine the size of NR/RVT-R discs.

A.) The first logical solution, which should be the standard solution for making raw dumps, is to descramble the raw DMI/PFI data returned from the cache itself.

When you execute the 0xAD 00 opcode (READ DVD STRUCTURE), the PFI data gets returned to the root of the drive's cache in DRAM. This data is scrambled with some algorithm, but just like other sectors, it has EDC that can be validated against. So since drives don't have a natural ability to descramble the PFI data of a RVT/NR disc, you can descramble the PFI data at the user level instead to determine the actual end sector size. To do this, you do the following:

1.) User inserts the RVT-R/NR disc 2.) DiscImageCreator runs the READ DVD STRUCTURE opcode 3.) At the root of the cache, the first 0x810 or 0x950 bytes written is the PFI's raw scrambled data. Unscramble/bruteforce this data to get the right seed so that you can read the entire PFI data. 4.) At 0x8 in the unscrambled PFI data is the end PSN value. 0x4 contains the start PSN value 5.) Pass this off to DIC's read commands, and you should be able to return data.

The only problem with this solution is that DIC currently does the bruteforcing/unscrambling in a separate process. In this scenario, it assumes DIC can do some of the unscrambling itself. But this is the safest most reliable way to get the end PSN every time.

B.) Look at the unique string stored in the DMI. GC, Wii, RVT-R, and NR discs all have unique vendor strings stored in the DMI area at 0x10.

"Nintendo NNGC Emu. Disk" <- RVT "Nintendo Game Disk" <- Retail GameCube "Nintendo NNGC Disk" <- Retail Wii "Nintendo Emulation Disk" <- NR

If the DMI matches one of the strings above, you can assume the start and end PSN. According to a valid dump of the PFI data, here are the starting and ending PSN numbers for Gamecube and Wii discs (both Retail and Dev): Wii/RVT: Start PSN: 00030000, End PSN: 0026127F GC/NR: Start PSN: 00030000, End PSN: 00DE0AF

There is an issue with this. Dual layer RVT-Rs can't be be dumped this way as the single layer and dual layer RVT-Rs use the same string in the DMI. So this isn't very reliable...

Log file Here are the log files along with the DMI/PFI data as returned by a DVD Plextor for a NR and RVT disc. rvtr.zip nr.zip

[saramibreak]

the DMI/PFI data as returned by a DVD Plextor for a NR and RVT disc.

How about non plextor drive? If possible, please upload all logs of the non plextor drive.

Wii/RVT: Start PSN: 00030000, End PSN: 0026127F

Is it typo? My wii disc (single) have 0x26047f.

    StartingDataSector:  196608 (0x30000)
         EndDataSector: 2491519 (0x26047f)

[ehw]

the DMI/PFI data as returned by a DVD Plextor for a NR and RVT disc.

How about non plextor drive? If possible, please upload all logs of the non plextor drive.

All the drives I've tested returned the PFI/DMI data this way. Plextors are the only drives that can read these discs without disc swapping, which I would assume would mess with the PFI/DMI data being returned. The only other drive that can read RVT-Rs and NR discs are RVT-R Writers, which are Matshita SW-9585-C drives flashed with special firmware that can descramble the RVT-Rs that use a nonstandard algorithm. This drive can return the PFI/DMI of the disc in full without anything missing. Check below for logs. The Matshita drives can read RVT-R and NR discs with Read12, but Plextors need to use READ BUFF. I haven't tested if the Matshita can do raw reads.

The only way for other drives to retrieve the PFI in full is to read the raw data that's returned onto the cache and bruteforce a seed to descramble it. I don't think drives can descramble it entirely by themselves.

Wii/RVT: Start PSN: 00030000, End PSN: 0026127F

Is it typo? My wii disc (single) have 0x26047f.

  StartingDataSector:  196608 (0x30000)
       EndDataSector: 2491519 (0x26047f)

Not a typo, but something I actually overlooked. The EndDataSector is different for RVT-Rs in comparison to commercial discs. I didn't realize this. A single layer RVT-R returns 0x26127F as the end PSN. Commercial games return 0x26047F.

Here are some logs, let me know if you need to see anything specific:: Sonic and the Secret Rings (Retail single layer Wii disc, completely dumped with a PX-760A with READ BUFF) Sonic and the Secret Rings (retail wii disc - raw dump with plextor).zip

RVT-R disc dumped with a RVT-R Writer. This was dumped with normal DVD operations due to the firmware being made to handle these discs' scrambling algorithm: Muscle March Game Disc (rvt wii dev disc - normal dump with rvt writer).zip

Sonic Gems Collection (Retail GC disc, completely dumped with a PX-760A with READ BUFF): Sonic Gems Collection (retail gc disc - raw dump with plextor).zip

[ehw]

Here's a cache dump of what raw DMI/PFI data looks like from an RVT-R. The READ DVD STRUCTURE command was executed which put the DMI/PFI onto the cache. The cache was dumped using READ BUFF, and descrambled using unscrambler. The result, in the .iso, is what the data looks like. It consists of both the PFI and DMI in its entirety, which includes the end PSN too. The same drive didn't return all the data for the PFI when using the READ DVD STRUCTURE. It was only when descrambling the raw data from the cache that you could get everything: cache.zip

[saramibreak]

DiscImageCreator_test.zip

• added: generate _PFI.raw

Please check the end PSN of your RVT-R. If it's 0, I use a fixed size (0x26127F) as a magic number.

[ehw]

DiscImageCreator_test.zip

• added: generate _PFI.raw

Please check the end PSN of your RVT-R. If it's 0, I use a fixed size (0x26127F) as a magic number.

Doesn't seem like it worked, it read the first block (16 sectors) from the cache that was on the disc and not the PFI/DMI. issue 221 - raw pfi test 1.zip

[saramibreak]

Retail disc can get it correctly. test.zip

If RVT-R can't get it, I use a fixed size (0x26127F) as a magic number.

[ehw]

I think I found the issue. You need to put something on the cache first before you put the PFI/DMI on there with READ DVD STRUCTURE or else the data gets immediately replaced. You need to read from cache to get the PFI at a different point.

Try doing this: 1.) Read LBA 0, this gets LBA 0 onto the root of DRAM/Cache. a8 00 00 00 00 00 00 00 00 01 00 00

2.) Next, read format code 04 (return DMI. this will return PFI also because PFI and DMI are part of the same 'block'. drives return things onto the cache by block) ad 00 00 00 00 00 00 04 00 04 00 00

3.) Part of the lead in area is now in cache, now you can read it with 3C 02 00: 3c 02 00 00 00 00 00 08 10 00

Unscrambler needs 16 sectors worth of data to unscramble.

Here's the output data: The raw data looks like raw_pfi_dmi.bin when stored onto the cache. The data looks like raw_pfi_dmi.iso when successfully unscrambled (it used seed 00d0 for us).

raw_pfi_dmi.zip

And here's the data that DIC needs, see circled. That's the end PSN. This for some reason is not part of the PFI data returned directly with 0xAD.

[saramibreak]

Try doing this

I implemented below in test version. 1) read format code 00 2) read cache (3C 02 00) 3) got 16 blocks with PFI and DMI

This for some reason is not part of the PFI data returned directly with 0xAD.

Please try sg_raw using 0xAD to check the end PSN.

[ehw]

Try doing this

I implemented below in test version.

1. read format code 00
2. read cache (3C 02 00)
3. got 16 blocks with PFI and DMI

Ah I see, using format code 00 doesn't pull the data from the disc, it pulls it straight from various parts of RAM it seems. It's immediately returned when you run it. Using format code 04 for some reason makes the Plextor read from the disc, storing the data that's read onto the cache.

Can you make a new test version with format code 04 instead? That might do it.

The one thing to note, RVT-Rs and NR discs are not like commercial Wii/GC discs at all. For some reason, there are drives that can read commercial discs fine but almost all drives have a problem with the PFI of RVT-R and NR. So even though retail discs work for you with format code 00, having a RVT-R or NR in the drive might make the drive do different things.

This for some reason is not part of the PFI data returned directly with 0xAD.

Please try sg_raw using 0xAD to check the end PSN.

Using "ad 00 00 00 00 00 00 00 00 0F 00 00": 08 02 00 00 25 00 02 00 00 03 00 00 00 00 00 00 ....%...........

Start PSN is defined, but not End PSN. This is seen in the PFI.bin as well.

[saramibreak]

DiscImageCreator_test.zip

According to your rvtr.zip, "FormatCode: 04 failed" occurs. I implemented to generate _PFI.raw after this message.

[ehw]

DiscImageCreator_test.zip

According to your rvtr.zip, "FormatCode: 04 failed" occurs. I implemented to generate _PFI.raw after this message.

Yep, that did it: test5.zip

I didn't unscramble it but it's definitely a perfect match from our manual dump.

So you just have to unscsramble this and use the last psn value and DIC should be able to read RVT-Rs and NRs.

BTW, just as a reference:

The left file is the RVT-R's raw PFI/DMI data that's still scrambled. The right is the raw PFI/DMI data from a blank DVD-R. There is almost no similarity between them despite carrying a lot of the same data when unscrambled. This proves that the scrambling algorithm for RVT-Rs and NRs is different and isn't the DVD standard.

[saramibreak]

DiscImageCreator_test.zip

changed: _PFI.raw to _CDZ.raw (CDZ is ControlDataZone)

I don't have NR and RVT-R, so test please.

[ehw]

Good work! We're almost there...

test6.zip

LBA[000000, 0000000]: [F:ReadDiscStructure][L:1576] Opcode: 0xad ScsiStatus: 0x02 = CHECK_CONDITION SenseData Key-Asc-Ascq: 05-24-00 = ILLEGAL_REQUEST - INVALID FIELD IN CDB FormatCode: 01 failed cdb: ad, 00, 00, 00, 00, 00, 00, 01, 00, 04, 00, 00 LBA[000000, 0000000]: [F:ReadDiscStructure][L:1576] Opcode: 0xad ScsiStatus: 0x02 = CHECK_CONDITION SenseData Key-Asc-Ascq: 05-24-00 = ILLEGAL_REQUEST - INVALID FIELD IN CDB FormatCode: 04 failed cdb: ad, 00, 00, 00, 00, 00, 00, 04, 00, 04, 00, 00 GOD/WOD unscrambler 0.5.1 (xt5@ingenieria-inversa.cl)

This program is distributed under GPL license, see the LICENSE file for more info.

caching seed 00d0 image successfully unscrambled. time elapsed: 0.00 seconds. LBA[000000, 0000000]: [F:ReadDiscStructure][L:1576] Opcode: 0xad ScsiStatus: 0x02 = CHECK_CONDITION SenseData Key-Asc-Ascq: 05-24-00 = ILLEGAL_REQUEST - INVALID FIELD IN CDB FormatCode: 0c failed cdb: ad, 00, 00, 00, 00, 00, 00, 0c, 00, 04, 00, 00 LBA[000000, 0000000]: [F:ReadDiscStructure][L:1576] Opcode: 0xad ScsiStatus: 0x02 = CHECK_CONDITION SenseData Key-Asc-Ascq: 05-21-00 = ILLEGAL_REQUEST - LOGICAL BLOCK ADDRESS OUT OF RANGE FormatCode: 0d failed cdb: ad, 00, 00, 00, 00, 00, 00, 0d, 00, 04, 00, 00 LBA[000000, 0000000]: [F:ReadDiscStructure][L:1576] Opcode: 0xad ScsiStatus: 0x02 = CHECK_CONDITION SenseData Key-Asc-Ascq: 05-24-00 = ILLEGAL_REQUEST - INVALID FIELD IN CDB FormatCode: 10 failed cdb: ad, 00, 00, 00, 00, 00, 00, 10, 00, 04, 00, 00 Rawdump command [0]:0x3c [1]:0x02 [2]:0000 LBA[000000, 0000000]: [F:ReadDVDRaw][L:962] Opcode: 0xa8 ScsiStatus: 0x02 = CHECK_CONDITION SenseData Key-Asc-Ascq: 03-11-05 = MEDIUM_ERROR - L-EC UNCORRECTABLE ERROR

EndTime: 2023-07-09T11:47:25-0400

So the drive read the raw CDZ and unscrambled it successfully, and it seems like it did apply the correct end LBA to start dumping (if you got this far?). But when it tried to read the disc it returned a L-EC error and quit. This might be because DIC didn't detect the disc as Gamecube/Wii and so instead of ignoring the L-EC errors, it quit on the first one?

Remember, the discs use a non standard DVD scrambling algorithm. The drive will read the sectors and return it onto the cache but since it can't descramble the sector to check against EDC it'll return a L-EC ERROR sense. This is fine for us because we can brute force and descramble it later. So we want to still keep reading the disc, putting the sectors onto the cache raw, and just descramble it when we get to the end. :)

EDIT: Somewhat unrelated side request - can you output a log file when unscrambler is run like inputfilename_unscramble.txt? I noticed the unscrambler verbose isn't recorded anywhere.

[saramibreak]

DiscImageCreator_test.zip

added: RVT_SL_SIZE

BOOL IsNintendoDisc(
    PDISC pDisc
) {
    if (pDisc->SCSI.nAllLength == GAMECUBE_SIZE ||
        pDisc->SCSI.nAllLength == WII_SL_SIZE ||
        pDisc->SCSI.nAllLength == RVT_SL_SIZE ||
        pDisc->SCSI.nAllLength == WII_DL_SIZE) {
        return TRUE;
    }
    return FALSE;
}

can you output a log file when unscrambler is run like inputfilename_unscramble.txt? I noticed the unscrambler verbose isn't recorded anywhere.

added: output .txt using redirect

[ehw]

DiscImageCreator_test.zip

added: RVT_SL_SIZE

BOOL IsNintendoDisc(
  PDISC pDisc
) {
  if (pDisc->SCSI.nAllLength == GAMECUBE_SIZE ||
      pDisc->SCSI.nAllLength == WII_SL_SIZE ||
      pDisc->SCSI.nAllLength == RVT_SL_SIZE ||
      pDisc->SCSI.nAllLength == WII_DL_SIZE) {
      return TRUE;
  }
  return FALSE;
}

can you output a log file when unscrambler is run like inputfilename_unscramble.txt? I noticed the unscrambler verbose isn't recorded anywhere.

added: output .txt using redirect

Almost there!

RVT-Rs are working now. :) test8 - rvtr working.zip

NR discs aren't. It looks like a similar issue from before. test9 - nr fail.zip The end PSN for NR discs is 000DEEAF.

I'll recheck retail in a bit. Retail still works.

[ehw]

Side note. I don't have a RVT-DL on hand but I know someone that does.

The end PSN for a RVT-DL should be 0042A7BF.

[saramibreak]

DiscImageCreator_test.zip added: NR_SIZE, RVT_DL_SIZE

[ehw]

DiscImageCreator_test.zip added: NR_SIZE, RVT_DL_SIZE

RVT-R and NR are dumping now. :) nr and rvt good.zip

Feel free to merge and close.

BTW, have you looked into how you might be able to speed things up with dumping Wii/GC with Plextor? We were thinking messing with the READ error page 01h with the MODE SELECT command might help make it work.

[saramibreak]

BTW, have you looked into how you might be able to speed things up with dumping Wii/GC with Plextor?

No.

We were thinking messing with the READ error page 01h with the MODE SELECT command might help make it work.

You can see mmc3r10g.pdf p.291 for more information.

[ehw]

BTW, have you looked into how you might be able to speed things up with dumping Wii/GC with Plextor?

No.

We were thinking messing with the READ error page 01h with the MODE SELECT command might help make it work.

You can see mmc3r10g.pdf p.291 for more information.

Looked at it a little bit, doesn't seem to help. I believe we were able to turn off ECC correction and enable streaming mode but no change in behavior at all. It's possible that one of the drive's processing functions are slowing it down and not anything mechanic per se. Might need more of a firmware patch at this point unless you have some theories...

Anyway, thanks for fixing the bug. :)