Last entry of Full-TOC is improperly truncated

[tomthumb876]

On many discs, there is corruption in the Full-TOC due to improper truncation of the last entry (PFRAME and sometimes also PSEC of last entry are mistakenly set to 00 as a result). Many CD analysis programs have this issue, not just DIC; the cause is usually that the length of data returned for Full-TOC has been underestimated. It looks like some attempt was already made to deal with this (lines 266-270 in execScsiCmd.cpp with "wFullTocLenFix"), but it seems further correction is needed. This may be a simple matter of requesting or allocating a few more bytes for Full-TOC.

The truncation leads to unusable CCD files as well (since CCD is written from Full-TOC), so I'd say it should be prioritized. Please take a look, hopefully it will be an easy fix.

Note: Full-TOC is the ONLY place where this truncation occurs, all other TOC reading modes are fine.

[saramibreak]

there is corruption in the Full-TOC due to improper truncation of the last entry (PFRAME and sometimes also PSEC of last entry are mistakenly set to 00 as a result)

By what drive does it occur? "wTocEntriesAll" is always the multiples of 11. If it's not so, it's the firmware bug, I think.

[tomthumb876]

So far, I've tested PX-W4012 and PX-W5224. Anyway, no need for further guesswork, as I'm pretty sure I've identified the bug. In line 260 of execScsiCmd.cpp, you have:

    WORD wTocEntriesAll = (WORD)(wFullTocLen - sizeof(pFullTocData->Length));

This is wrong, because the returned data length (wFullTocLen) is already the exact length of the Full-TOC entries themselves. See the official MMC-3 documentation "TOC/PMA/ATIP Response Data Format 0010b" section, which says (emphasis added):

The TOC Data Length specifies the length in bytes of the available TOC data. The
TOC Data Length value *DOES NOT* include the TOC Data Length field itself. This
value is not modified when the allocation length is insufficient to return all
TOC data available.

So, I think the problem can be solved by simply changing execScsiCmd.cpp line 260 to this:

    WORD wTocEntriesAll = (WORD)(wFullTocLen);

Which of course is equivalent to just removing line 260 altogether and change line 259 to this:

    WORD wTocEntriesAll = MAKEWORD(pFullTocData->Length[1], pFullTocData->Length[0]);

Can we try that and I'll test and see if it solves the problem? Theoretically, everything should be fine after this correction.

[saramibreak]

You misunderstood my code.

FULL TOC struct is here

typedef struct _CDROM_TOC_FULL_TOC_DATA_BLOCK { UCHAR SessionNumber; UCHAR Control : 4; UCHAR Adr : 4; UCHAR Reserved1; UCHAR Point; UCHAR MsfExtra[3]; UCHAR Zero; UCHAR Msf[3]; } CDROM_TOC_FULL_TOC_DATA_BLOCK, *PCDROM_TOC_FULL_TOC_DATA_BLOCK;

typedef struct _CDROM_TOC_FULL_TOC_DATA {

// // Header //

UCHAR Length[2]; // add two bytes for this field UCHAR FirstCompleteSession; UCHAR LastCompleteSession;

// // one to N descriptors included //

if !defined(__midl)

CDROM_TOC_FULL_TOC_DATA_BLOCK Descriptors[0];

endif

} CDROM_TOC_FULL_TOC_DATA, *PCDROM_TOC_FULL_TOC_DATA;

The value of "length variable" of CDROM_TOC_FULL_TOC_DATA is the size of FirstCompleteSession and LastCompleteSession and Descriptors, while sizeof(pFullTocData->Length) is always 2 bytes. FirstCompleteSession and LastCompleteSession is also 2 bytes. To get the size of CDROM_TOC_FULL_TOC_DATA_BLOCK, it needs to reduce the value of "length variable" - 2 bytes. 2bytes are the size of FirstCompleteSession and LastCompleteSession, not the size of "length variable".

I should rewrite like this. WORD wTocEntriesAll = (WORD)(wFullTocLen - 2); // 2 is the size of FirstCompleteSession and LastCompleteSession

[tomthumb876]

Ah, I see. Thanks for clarifying. Even so, I'm not sure my solution is wrong, as the MMC-3 doc doesn't make it entirely clear whether or not the FirstCompleteSession and LastCompleteSession bytes are included in the returned Length value; if they're not, my suggestion should still work, simply for a different reason. Regardless, I keep running into this problem where the last entry in the Full-TOC is truncated by what appears to be 2 bytes, so no matter what the cause may be, increasing the wTocEntriesAll by 2 seems like a reasonable thing to try. You may very well be right that it's not the culprit, but I could confirm for sure in just a couple of minutes with a test version that disables the subtraction of 2 bytes.

[tomthumb876]

Update: I've now tested quite a few more Plextors, various different types (CD-RW, DVD-RW, etc.), including PX-716A and PX-760A. Always exactly the same truncation. While it's still technically possible that this is a firmware glitch, it's really seeming increasingly unlikely that ALL of these different models, from different model line-ups and time periods, would all be affected in exactly the same way.

Therefore, it's definitely worth testing to see if increasing wTocEntriesAll by 2 (i.e. disabling the current subtraction of 2 bytes) will solve this. It's a very serious problem that affects MANY discs and renders the CCD files unusable, so if there's even a 10% chance that such a simple solution can fix it, why not spend the couple of minutes to find out?

I hate to keep bothering you with this, so I decided to just go ahead and try to compile the test version myself. Unfortunately, I'm using Windows 8.1 which creates a lot of conflicts with Visual Studio, so it wouldn't make much sense for me to spend many hours trying to fix that when you could create the test version in just a minute or two. If you don't want to "pollute" your main test version with this change, you could simply compile a separate one-time-only test .exe file for this purpose. Any chance you'll be willing to do so? Please let me know, thanks! :)

[saramibreak]

added debug log in command-line screen. (Not change other code yet.) http://www.mediafire.com/file/eq80y20l9cwf48f/DiscImageCreator_test.7z/file

Let me know these values. FullTocLen: xxx, TocEntriesAll: xxx, TocEntries: xxx, FullTocLenFix (before padding): xxx FullTocLenFix (after padding): xxx, pPFullToc address: xxx, pFullToc address: xxx

[tomthumb876]

Thank you, I've gone ahead and tested this with two discs (this output is from PX-716A but seems to be the same with all drives). First, the output from a standard pressed CD-DA with 12 tracks:

FullTocLen: 167, TocEntriesAll: 165, TocEntries: 15, FullTocLenFix (before padding): 169
FullTocLenFix (after padding): 172, pPFullToc address: 0090D6C4, pFullToc address: 0090D6D8

As before, the last entry (track 12/point 0x0c) is truncated; PFRAME erroneously set to 00 as a result.

Then, here's the output from a standard DAO burned CD-DA with 31 tracks:

FullTocLen: 376, TocEntriesAll: 374, TocEntries: 34, FullTocLenFix (before padding): 378
FullTocLenFix (after padding): 380, pPFullToc address: 0019D300, pFullToc address: 0019D314

As before, the last entry (track 31/point 0x1f) is truncated; PSEC and PFRAME erroneously set to 00 00 as a result.

What do you conclude from this?

[saramibreak]

CD-DA with 12 tracks: FullTocLen: 167, TocEntriesAll: 165, TocEntries: 15

This is correct. TocEntries are always tracknum + 3 (0xa0, 0xa1, 0xa2). TocEntries * 11 = 165. it matches TocEntriesAll. TocEntriesAll + 2 = FullTocLen

CD-DA with 31 tracks: FullTocLen: 376, TocEntriesAll: 374, TocEntries: 34

This is also correct.

My tested disc log _disc.txt Session 1, Ctl 0, Adr 1, Point 0x13, Track 19, AMSF 59:37:70 (LBA[268195, 0x417a3]) ccd [Entry 21] Session=1 Point=0x13 ADR=0x01 Control=0x00 TrackNo=0 AMin=0 ASec=0 AFrame=0 ALBA=-150 Zero=0 PMin=59 PSec=37 PFrame=70 PLBA=268195 No truncated. Perhaps, this is your hardware problem (IDE SATA adapter? USB cable? motherboard? I don't know.)

[tomthumb876]

Yes, from the debug info I can see that your code is working as intended, in accordance with orthodox CD-reading procedures. However, my concern is that the orthodox CD-reading procedures may themselves be incorrect; perhaps there's some undocumented phenomenon that requires over-reading Full-TOC in order to avoid truncation. The fact that other CD reading programs have exactly the same problem would be consistent with such a possibility.

That said, your example of the non-truncated Full-TOC that you were able to read on your own setup is certainly solid evidence that my hypothesis is wrong after all. To help me hunt down the problem, can you please clarify for me:

(1) Are you absolutely certain that Point 0x13 is the VERY last Full-TOC entry on the example disc you used? (Neither any subsequent tracks, nor session pointers or structural info etc. after Point 0x13)

(2) Which Plextor model did you use for this? If I have it or can get it, then I can test with that model and confirm that it's indeed the firmware of all other drives causing this problem. Otherwise I'll have to start chasing down other possibilities (adapter/cable/motherboard/etc.), as you say.

[saramibreak]

(1) FullTocLen: 244, TocEntriesAll: 242, TocEntries: 22, FullTocLenFix (before padding): 246 FullTocLenFix (after padding): 248, pPFullToc address: 0079D46C, pFullToc address: 0079D480 ========== FULL TOC ========== FirstCompleteSession: 1 LastCompleteSession: 1 Session 1, Ctl 0, Adr 1, Point 0xa0, FirstTrack 1, Format: CD-DA or CD-ROM Session 1, Ctl 0, Adr 1, Point 0xa1, LastTrack 19 Session 1, Ctl 0, Adr 1, Point 0xa2, Lead-out, AMSF 62:07:15 (LBA[279390, 0x4435e]) Session 1, Ctl 0, Adr 1, Point 0x01, Track 1, AMSF 00:02:00 (LBA[000000, 0000000]) Session 1, Ctl 0, Adr 1, Point 0x02, Track 2, AMSF 01:52:53 (LBA[008303, 0x0206f]) Session 1, Ctl 0, Adr 1, Point 0x03, Track 3, AMSF 06:17:30 (LBA[028155, 0x06dfb]) Session 1, Ctl 0, Adr 1, Point 0x04, Track 4, AMSF 08:52:40 (LBA[039790, 0x09b6e]) Session 1, Ctl 0, Adr 1, Point 0x05, Track 5, AMSF 10:10:27 (LBA[045627, 0x0b23b]) Session 1, Ctl 0, Adr 1, Point 0x06, Track 6, AMSF 11:56:65 (LBA[053615, 0x0d16f]) Session 1, Ctl 0, Adr 1, Point 0x07, Track 7, AMSF 17:00:15 (LBA[076365, 0x12a4d]) Session 1, Ctl 0, Adr 1, Point 0x08, Track 8, AMSF 19:27:33 (LBA[087408, 0x15570]) Session 1, Ctl 0, Adr 1, Point 0x09, Track 9, AMSF 23:53:10 (LBA[107335, 0x1a347]) Session 1, Ctl 0, Adr 1, Point 0x0a, Track 10, AMSF 27:37:02 (LBA[124127, 0x1e4df]) Session 1, Ctl 0, Adr 1, Point 0x0b, Track 11, AMSF 30:56:40 (LBA[139090, 0x21f52]) Session 1, Ctl 0, Adr 1, Point 0x0c, Track 12, AMSF 34:11:65 (LBA[153740, 0x2588c]) Session 1, Ctl 0, Adr 1, Point 0x0d, Track 13, AMSF 37:36:32 (LBA[169082, 0x2947a]) Session 1, Ctl 0, Adr 1, Point 0x0e, Track 14, AMSF 41:58:27 (LBA[188727, 0x2e137]) Session 1, Ctl 0, Adr 1, Point 0x0f, Track 15, AMSF 45:31:03 (LBA[204678, 0x31f86]) Session 1, Ctl 0, Adr 1, Point 0x10, Track 16, AMSF 49:11:44 (LBA[221219, 0x36023]) Session 1, Ctl 0, Adr 1, Point 0x11, Track 17, AMSF 52:44:03 (LBA[237153, 0x39e61]) Session 1, Ctl 0, Adr 1, Point 0x12, Track 18, AMSF 56:12:13 (LBA[252763, 0x3db5b]) Session 1, Ctl 0, Adr 1, Point 0x13, Track 19, AMSF 59:37:70 (LBA[268195, 0x417a3])

(2) Tested by PX-4824A, 5224A, 755SA (fw is all latest) and ASUS BW-16D1HT (fw is 3.02) These results are all the same.

Again, this is not either the problem of my (and other) app or your drive. I recommend changing the adapter if you are using.

[tomthumb876]

Thank you, that's very helpful indeed! So it's not the drive firmware after all, nor a DIC bug. I was beginning to suspect a Windows/IOCTL (etc.) bug, but I've now managed to rule that out: there is NO truncation when I use my laptop's internal DVD drive (not DIC compatible) to read Full-TOC with other programs, even when those same programs DO show the truncation with all of my Plextors on the same laptop. So, by process of elimination, logically, the issue must be somewhere in the USB chain. I'll investigate further and probably won't need to bring it up anymore with you. Thanks again and sorry for wasting so much of your time on it!

[periodic1236]

@tomthumb876 As I see this issue is still open and I recently encountered the same truncation syndrome, I wonder if you reached any conclusion in the last 4.5 years.

I only have one drive to test with, but in my own testing, some discs never experience this yet about 2/3 of them do and it appears to be deterministic/reproducible. I don't know if there should be anything particular about those discs which would lead to it. All my rips are of commercially pressed CDs. In a given rip session (where the drive is continuously powered and connected to the host), one disc may be fine while others are not.

I am using a USB bridge/adapter, so should I be investigating that? Still, to me this is thus far insufficient a rationale to explain why DIC cannot be equipped to handle the truncation where the regular TOC reading does not match. Although, I appreciate that just having a workaround without fully understanding the underlying cause is undesirable.

I'll have to decide how I can move forward. In any case, now that I know other programs can give a similarly erroneous result, I am more convinced the issue is something happening from the hardware side.