search

KBNLresearch / iromlab

Loader software for automated imaging of optical media with Nimbie disc robot
Apache License 2.0
33 stars 5 forks source link

Data ISO sessions from CD-Extra / Blue Book discs fail Isolyzer check b/c sector numbers relative to CD, not ISO! #49

Closed bitsgalore closed 7 years ago

bitsgalore commented 7 years ago

All 'enhanced' audio CDs fail because ISO image does not pas Isolyzer check. Encountered same behaviour before. TODO: identify cause. Try with other imaging tools (e.g. ddrescue). Ask if others have similar experiences. Check if ISO images are readable at all.

bitsgalore commented 7 years ago

Additional tests:

Tried one failed CD both IsoBuster and ddrescue

Isobuster: 31.8 MB ISO, fails Isolyzer check.

Ddrescue command:

ddrescue -b 2048 -r4 -v /dev/sr0 mydisk.iso mydisk.log

After > 12 mins 32 MB recovered with errsize = 549 MB.

BUT errsize (and diff between Isolyzer's expected and actual size) corresponds roughly to the space occupied by the audio content; in any case:

size(audio) = size(expected data) > max capacity of a CD (which is about 700 MB). So maybe the image as extracted by Isobuster is OK? Perhaps the meaning of the PVD fields is somewhat different for this type of disc?

See also: CD Writing Howto - MIXING AUDIO AND DATA ON A CD and CD Writing Howto - MULTISESSION CD-ROMS, esp. -C parameter in mkisofs call - seems to do something superficially similar? TODO test if this gives similar results for ISO image. Adapt script:

https://github.com/KBNLresearch/cdtestcorpus/blob/master/audioAndDataMultiSession.sh

bitsgalore commented 7 years ago

Turns out issue is NOT that the image is incomplete, but that the sector numbers in an ISO from an enhanced CD are relative to the start of the CD, and not relative to the ISO image!! In most cases these images also cannot be mounted or opened by archive manager software!

See:

https://lists.debian.org/debian-user/2005/01/msg02339.html

the sector numbers in the file system refer to sectors of the original CD rather than sectors of session2.iso. I don't know of a utility for rewriting them so that the file can be loop-mounted or written to an ordinary CD, but you can at least get a directory listing by using isoinfo with an offset:

isoinfo -i session2.iso -N 204345 -l

https://lists.gnu.org/archive/html/libcdio-devel/2010-02/msg00048.html

Esp.:

Remember, the path table and directory structure of the iso reflect the fact that the ISO filesystem starts on sector 222145 (49:23:70) of the CD. If it is burned to another CD at a different position, it won't work. Likewise, any program that reads the iso will need to be able to compensate for the offset. Try, for example: isoinfo -N 222145 -d -i '8mm-songs_to_love_and_die_by.iso'

Also (from same thread):

https://lists.gnu.org/archive/html/libcdio-devel/2010-02/msg00053.html

Solution (iromlab QA only):

  1. Modify cd-info wrapper so that it also returns the sector number where the data session starts. Can be extracted from:

    cd-info

Then look for this bit:

CD-Plus/Extra   
session #2 starts at track  8, LSN: 21917, ISO 9660 blocks:  25309
ISO 9660: 25309 blocks, label `DISC

So start sector of the data session is 21917.

  1. I added a new --offset option to isolyzer.

  2. Now we can feed the start sector into isolyzer:

    isolyzer.py --offset 21917 DISC.iso

This tells isolyzer to subtract (offset sectors * sector size) from the expected size. The offsets should also be recorded in the metadata (because we will most likely need them for access). Probably best to save the fulll cd-info output as well and import that into the SIP's carrier-level technical metadata.

This does NOT solve the larger problem that these ISOs cannot be readily mounted / read because of the offset issue. needs further investigation. Some interesting suggestions here (mounting in Linux):

https://lists.gnu.org/archive/html/libcdio-devel/2010-02/msg00050.html

If this works it might be possible to rewrite the ISO using method here:

https://lists.gnu.org/archive/html/libcdio-devel/2010-02/msg00053.html

TODO check isolyzer behaviour in case of Apple partitions, may need further tweaking: https://github.com/KBNLresearch/isolyzer/issues/4

bitsgalore commented 7 years ago

Update to the above: following the instructions here I can mount the image in Linux using the following steps (assuming again that start sector of the data session is 21917):

  1. Create placeholder for missing blocks: dd if=/dev/zero bs=2K count=1 seek=21916 of=cd_dummy.iso
  2. Append ISO image to placeholder: cat multisession.iso >>cd_dummy.iso
  3. Mount image; sudo mount -t iso9660 -o loop,sbsector=21916 cd_dummy.iso /media/johan
  4. Create new ISO from mounted filesystem: xorrisofs -r -J -o ./cd_new.iso /media/johan

New ISO can be mounted/extracted normally! Note that the sbsector value in the mount command is 21916, not 21917 (initially tried 21917, but this resulted in all files in the mounted fs being corrupted!).

bitsgalore commented 7 years ago

Other update: if ISO image contains an Apple partition map (see isolyzer field containsApplePartitionMap) the Apple partition will mount without any modifications. So if we have an enhanced CD with a data session that contains both an ISO/Joliet/Rock Ridge file system and an Apple HFS+ filesystem, we could do the following to ensure that the ISO image will work across all emulated target environments:

  1. Use the unmodified ISO for environments that rely on the Apple file system
  2. For environments that rely on ISO/Joliet/Rock Ridge, use the modified ISO

Modified ISO could be pre-generated as an access copy, or it could be created on the fly upon access. Either way, it's essential to keep a record of the sector layout of the original physical carrier.

TODO:

bitsgalore commented 7 years ago

Done so far: