search

storaged-project / udisks

The UDisks project provides a daemon, tools and libraries to access and manipulate disks, storage devices and technologies.
https://storaged.org/doc/udisks2-api/latest/
Other
345 stars 143 forks source link

udisks fails to mount small hybrid HFS/ISO9660 CD Medium #725

Open mwilck opened 4 years ago

mwilck commented 4 years ago

udisks fails to mount a hybrid hfs/iso9660 medium, both automatically and manually. I Observe this with udisks 2.8.4 and 2.6.5. It feels strange to report this, as I'd expect it to have been reported before, but despite extensive googling I haven't found a similar report yet.

# udisksctl mount -b /dev/sr0
Object /org/freedesktop/UDisks2/block_devices/sr0 is not a mountable filesystem.

The CD medium in question has been created with the "Rimage" tool. It seems to be a typical example of a hybrid Mac/Iso9660 CD, where the first 32kiB (unused by ISO9660) hold a Mac partition table. I suppose the behavior wouldn't be different with mkisofs-generated hybrid images. udisksd detects an empty MAC partition table on the medium, and doesn't look further:

# udisksctl info -b /dev/sr0
/org/freedesktop/UDisks2/block_devices/sr0:
  org.freedesktop.UDisks2.Block:
   Configuration:              []
   CryptoBackingDevice:        '/'
   Device:                     /dev/sr0
   DeviceNumber:               2816
   Drive:                      '/org/freedesktop/UDisks2/drives/TSSTcorp_CDDVDW_TS_H653J_R82P6GCC100KHL'
   HintAuto:                   true
   HintIconName:               
   HintIgnore:                 false
   HintName:                   
   HintPartitionable:          false
   HintSymbolicIconName:       
   HintSystem:                 false
   Id:                         
   IdLabel:                    
   IdType:                     
   IdUUID:                     
   IdUsage:                    
   IdVersion:                  
   MDRaid:                     '/'
   MDRaidMember:               '/'
   PreferredDevice:            /dev/sr0
   ReadOnly:                   true
   Size:                       641024
   Symlinks:                   /dev/cdrom
                                     /dev/cdrw
                                     /dev/disk/by-id/ata-TSSTcorp_CDDVDW_TS-H653J_R82P6GCC100KHL
                                    /dev/disk/by-path/pci-0000:00:1f.2-ata-5
                                    /dev/dvd
                                    /dev/dvdrw
   UserspaceMountOptions:
org.freedesktop.UDisks2.PartitionTable:
   Partitions:         []
   Type:               mac

blkid also sees a "mac" partition table. Interestingly, blkid also sees the iso9660 file system, but only if it's called without --probe:

# blkid  /dev/sr0
/dev/sr0: UUID="2019-12-19-08-36-50-00" LABEL="IT87" TYPE="iso9660" PTTYPE="mac"
# blkid --probe /dev/sr0
/dev/sr0: PTTYPE="mac"

Mounting this medium works just fine:

#  mount /dev/sr0 /mnt/cdrom
mount: /mnt/cdrom: WARNING: device write-protected, mounted read-only.
# grep cdrom /proc/self/mountinfo 
281 89 11:0 / /mnt/cdrom ro,relatime shared:105 - iso9660 /dev/sr0 ro
# ls /mnt/cdrom/
extras  liesmich.txt  r181224.txt  readme.txt

cdrecord sees the medium like this, with just a single data track:

# cdrecord -media-info dev=4,0,0
Cdrecord-ProDVD-ProBD-Clone 3.02a09 (x86_64-unknown-linux-gnu) Copyright (C) 1995-2016 Joerg Schilling
...
Device seems to be: Generic mmc2 DVD-R/DVD-RW/DVD-RAM.
...
Mounted media class:      CD
Mounted media type:       CD-R
Disk Is not erasable
data type:                standard
disk status:              complete
session status:           complete
BG format status:         none
first track:              1
number of sessions:       1
first track in last sess: 1
last track in last sess:  1
Disk Is not unrestricted
Disk type: CD-DA or CD-ROM
Disk id: 0x627689
last start of lead in: 716730
last start of lead out: 1166730

Track  Sess Type   Start Addr End Addr   Size
==============================================
    1     1 Data   0          310        311

Last session start address:         0
Last session leadout start address: 311

When I dump the CD contents with dd and import the resulting image with qemu-nbd, I actually see a partition table:

# parted /dev/nbd0 print
Model: Unknown (unknown)
Disk /dev/nbd0: 635kB
Sector size (logical/physical): 512B/512B
Partition Table: mac
Disk Flags: 

Number  Start   End     Size    File system  Name               Flags
 1      512B    32.8kB  32.3kB               Apple              type=Apple_partitio
 3      45.1kB  330kB   285kB   hfs          Rimage HFS Hybrid  type=Apple_HFS

The HFS partition can actually be mounted and has the same contents as the ISO9660 image. On the nbd device, udiskctl also sees partitions (unlike the CD, where the partition table was seen empty by udisksd, see above), and can mount partition3:

# udisksctl mount -b /dev/nbd0p1
Object /org/freedesktop/UDisks2/block_devices/nbd0p1 is not a mountable filesystem.
# udisksctl mount -b /dev/nbd0p2
Object /org/freedesktop/UDisks2/block_devices/nbd0p2 is not a mountable filesystem.
# udisksctl mount -b /dev/nbd0p3
Mounted /dev/nbd0p3 at /run/media/root/IT87.

Attaching udiskctl info output for the nbd device mac-partition-info.txt.

It's also possible to map partitions on /dev/sr0 using the kpartx command:

# kpartx -r -p -part -a /dev/sr0
device-mapper: reload ioctl on sr0-part1  failed: Invalid argument
create/reload failed on sr0-part1

Despite this error, /dev/mapper/sr0-part3 is mountable as HFS afterwards.

# udisksctl mount -b /dev/mapper/sr0-part3 
Mounted /dev/dm-30 at /run/media/root/IT87.

Of course partitions on the device are not automatically set up by the OS, as we have no standard udev rules for "partitions" on optical devices. We could think about doing this for hybrid CDs, but that's not the issue here.

As far as udisks is concerned, I believe it should be able to figure out that there is a valid iso9660 superblock at offset 32k, and mount it.

The medium contains copyrighted material, thus I can't simply attach an image here. But I can provide it privately if necessary for debugging purposes.

mwilck commented 4 years ago

Forgot to say, the results above were obtained with udisks2 using libblockdev 2.22.

vojtechtrefny commented 4 years ago

What udev sees on the device -- udevadm info /dev/sr0 ? I'm afraid udev uses blkid with the --probe option so it probably doesn't see the iso9660 filesystem, only the partition table (we use data udev database).

mwilck commented 4 years ago

Right. udev uses the builtin blkid, which provides this:

# udevadm test-builtin blkid /block/sr0
calling: test-builtin
probe /dev/sr0 raid offset=0
ID_PART_TABLE_TYPE=mac

The question is, what exactly is different with and without --probe. The man page is not very clear about that. Rather, it suggests that with --probe, additional information would be provided.

Note that low-level probing also returns information about partition table type (PTTYPE tag) and partitions (PARTENTRY* tags).

mwilck commented 4 years ago

FTR, libblkid1 installed version was from util-linux 2.33

mwilck commented 4 years ago

I found the issue. Check out https://github.com/karelzak/util-linux/commit/e3d8933aaa1786ed78f06e5c9792d2167801d48e (udev just copied that code, https://github.com/systemd/systemd/commit/d13394a88334441bf3092cf93804ba0f9c56d8e0). The point is that this CD is tiny, it has less then 1MiB data on it (important data though). For disks smaller than 1.44M floppies, libblkid skips certain tests. It doesn't test if the device is a CD (https://github.com/karelzak/util-linux/commit/a3ab71cfd3f2b2240e05df3e9cc0800ab4578744), and doesn't test for file systems any more if it finds a partition table.

So yeah, it's not an udisks problem after all. To make this work we'd need patches in both libblkid and udev.

mwilck commented 4 years ago

Admittedly, this is a rather extreme corner case. Yet as hardly anyone has a floppy drive any more these days, how else would such small data be shipped? I obtained the CD in question from a shop that sells printed color calibration charts for scanner color calibration. The CD contains the reference data for the specific charge of prints in question - not much data, but highly important for the calibration. I guess the shop could have also sent me a download link, but they preferred to do it this way. IMO I think it's wrong that media are not/wrongly detected because of old optimizations. I'll submit patches.

guss77 commented 4 years ago

I have a similar - though not identical issue - with an ISO image of a hybrid game installation disk. Unlike the original issue, when this image is attached to a loop device, udisks successfully mounts the HFS part.

The problem is that it refuses to mount the ISO9660 Joliet file system and this case it is a serious issue as the Joliet and HFS file systems expose different file sets - to hide the Mac specific files from Windows and the Windows specific files from Mac.

Here are a bunch of diagnostics that I ran:

$ udisksctl loop-setup -r -f 'XXXXXXXX.iso'
Mapped file XXXXXXXX.iso as /dev/loop36.
$ udisksctl info --block-device /dev/loop36
/org/freedesktop/UDisks2/block_devices/loop36:
  org.freedesktop.UDisks2.Block:
    Configuration:              []
    CryptoBackingDevice:        '/'
    Device:                     /dev/loop36
    DeviceNumber:               1828
    Drive:                      '/'
    HintAuto:                   false
    HintIconName:               
    HintIgnore:                 false
    HintName:                   
    HintPartitionable:          true
    HintSymbolicIconName:       
    HintSystem:                 true
    Id:                         
    IdLabel:                    INSTALL
    IdType:                     iso9660
    IdUUID:                     2004-08-12-14-36-55-00
    IdUsage:                    filesystem
    IdVersion:                  Joliet Extension
    MDRaid:                     '/'
    MDRaidMember:               '/'
    PreferredDevice:            /dev/loop36
    ReadOnly:                   true
    Size:                       654561280
    Symlinks:                   /dev/disk/by-label/INSTALL
                                /dev/disk/by-uuid/2004-08-12-14-36-55-00
    UserspaceMountOptions:      
  org.freedesktop.UDisks2.Loop:
    Autoclear:          false
    BackingFile:        /home/odeda/XXXXXXXX/XXXXXXXX.iso
    SetupByUID:         1000
  org.freedesktop.UDisks2.PartitionTable:
    Partitions:         ['/org/freedesktop/UDisks2/block_devices/loop36p1', '/org/freedesktop/UDisks2/block_devices/loop36p2']
    Type:               mac
$ udisksctl info --block-device /dev/loop36p1
/org/freedesktop/UDisks2/block_devices/loop36p1:
  org.freedesktop.UDisks2.Block:
    Configuration:              []
    CryptoBackingDevice:        '/'
    Device:                     /dev/loop36p1
    DeviceNumber:               66313
    Drive:                      '/'
    HintAuto:                   false
    HintIconName:               
    HintIgnore:                 false
    HintName:                   
    HintPartitionable:          true
    HintSymbolicIconName:       
    HintSystem:                 true
    Id:                         
    IdLabel:                    
    IdType:                     
    IdUUID:                     
    IdUsage:                    
    IdVersion:                  
    MDRaid:                     '/'
    MDRaidMember:               '/'
    PreferredDevice:            /dev/loop36p1
    ReadOnly:                   true
    Size:                       1024
    Symlinks:                   
    UserspaceMountOptions:      
  org.freedesktop.UDisks2.Partition:
    Flags:              0
    IsContained:        false
    IsContainer:        false
    Name:               MRKS
    Number:             1
    Offset:             512
    Size:               1024
    Table:              '/org/freedesktop/UDisks2/block_devices/loop36'
    Type:               Apple_partition_map
    UUID:               
$ udisksctl info --block-device /dev/loop36p2
/org/freedesktop/UDisks2/block_devices/loop36p2:
  org.freedesktop.UDisks2.Block:
    Configuration:              []
    CryptoBackingDevice:        '/'
    Device:                     /dev/loop36p2
    DeviceNumber:               66314
    Drive:                      '/'
    HintAuto:                   false
    HintIconName:               
    HintIgnore:                 false
    HintName:                   
    HintPartitionable:          true
    HintSymbolicIconName:       
    HintSystem:                 true
    Id:                         
    IdLabel:                    XXXXXXXX Install
    IdType:                     hfs
    IdUUID:                     dbd5a3ea-5614-384e-8d15-045117946523
    IdUsage:                    filesystem
    IdVersion:                  
    MDRaid:                     '/'
    MDRaidMember:               '/'
    PreferredDevice:            /dev/loop36p2
    ReadOnly:                   true
    Size:                       628582400
    Symlinks:                   /dev/disk/by-label/XXXXXXXX\x20Install
                                /dev/disk/by-uuid/dbd5a3ea-5614-384e-8d15-045117946523
    UserspaceMountOptions:      
  org.freedesktop.UDisks2.Filesystem:
    MountPoints:        
    Size:               0
  org.freedesktop.UDisks2.Partition:
    Flags:              0
    IsContained:        false
    IsContainer:        false
    Name:               Toast 5.2 HFS Optimizer
    Number:             2
    Offset:             25977856
    Size:               628582400
    Table:              '/org/freedesktop/UDisks2/block_devices/loop36'
    Type:               Apple_HFS
    UUID:               
$ blkid XXXXXXXX.iso
XXXXXXXX.iso: UUID="2004-08-12-14-36-55-00" LABEL="INSTALL" TYPE="iso9660" PTTYPE="mac"
$ blkid --probe XXXXXXXX.iso
XXXXXXXX.iso: SYSTEM_ID="APPLE COMPUTER, INC., TYPE: 0002" APPLICATION_ID="TOAST ISO 9660 BUILDER COPYRIGHT (C) 1997-2002 ROXIO, INC. - HAVE A NICE DAY" UUID="2004-08-12-14-36-55-00" VERSION="Joliet Extension" LABEL="INSTALL" TYPE="iso9660" USAGE="filesystem" PTTYPE="mac"
$ blkid --probe /dev/loop36
/dev/loop36: SYSTEM_ID="APPLE COMPUTER, INC., TYPE: 0002" APPLICATION_ID="TOAST ISO 9660 BUILDER COPYRIGHT (C) 1997-2002 ROXIO, INC. - HAVE A NICE DAY" UUID="2004-08-12-14-36-55-00" VERSION="Joliet Extension" LABEL="INSTALL" TYPE="iso9660" USAGE="filesystem" PTTYPE="mac"
$ blkid --probe /dev/loop36p1
/dev/loop36p1: PART_ENTRY_SCHEME="mac" PART_ENTRY_NAME="MRKS" PART_ENTRY_TYPE="Apple_partition_map" PART_ENTRY_NUMBER="1" PART_ENTRY_OFFSET="1" PART_ENTRY_SIZE="2" PART_ENTRY_DISK="7:36"
$ blkid --probe /dev/loop36p2
/dev/loop36p2: UUID="dbd5a3ea-5614-384e-8d15-045117946523" LABEL="XXXXXXXX Install" TYPE="hfs" USAGE="filesystem" PART_ENTRY_SCHEME="mac" PART_ENTRY_NAME="Toast 5.2 HFS Optimizer" PART_ENTRY_TYPE="Apple_HFS" PART_ENTRY_NUMBER="2" PART_ENTRY_OFFSET="50738" PART_ENTRY_SIZE="1227700" PART_ENTRY_DISK="7:36"
$ udevadm info /dev/loop36
P: /devices/virtual/block/loop36
N: loop36
L: 0
S: disk/by-label/INSTALL
S: disk/by-uuid/2004-08-12-14-36-55-00
E: DEVPATH=/devices/virtual/block/loop36
E: DEVNAME=/dev/loop36
E: DEVTYPE=disk
E: MAJOR=7
E: MINOR=36
E: SUBSYSTEM=block
E: USEC_INITIALIZED=6879590
E: ID_FS_SYSTEM_ID=APPLE\x20COMPUTER\x2c\x20INC.\x2c\x20TYPE:\x200002
E: ID_FS_APPLICATION_ID=TOAST\x20ISO\x209660\x20BUILDER\x20COPYRIGHT\x20\x28C\x29\x201997-2002\x20ROXIO\x2c\x20INC.\x20-\x20HAVE\x20A\x20NICE\x20DAY
E: ID_FS_UUID=2004-08-12-14-36-55-00
E: ID_FS_UUID_ENC=2004-08-12-14-36-55-00
E: ID_FS_VERSION=Joliet Extension
E: ID_FS_LABEL=INSTALL
E: ID_FS_LABEL_ENC=INSTALL
E: ID_FS_TYPE=iso9660
E: ID_FS_USAGE=filesystem
E: ID_PART_TABLE_TYPE=mac
E: DEVLINKS=/dev/disk/by-label/INSTALL /dev/disk/by-uuid/2004-08-12-14-36-55-00
E: TAGS=:systemd:

$ udevadm info /dev/loop36p1
P: /devices/virtual/block/loop36/loop36p1
N: loop36p1
L: 0
E: DEVPATH=/devices/virtual/block/loop36/loop36p1
E: DEVNAME=/dev/loop36p1
E: DEVTYPE=partition
E: PARTN=1
E: MAJOR=259
E: MINOR=9
E: SUBSYSTEM=block
E: USEC_INITIALIZED=10840129245
E: ID_PART_TABLE_TYPE=mac
E: ID_PART_ENTRY_SCHEME=mac
E: ID_PART_ENTRY_NAME=MRKS
E: ID_PART_ENTRY_TYPE=Apple_partition_map
E: ID_PART_ENTRY_NUMBER=1
E: ID_PART_ENTRY_OFFSET=1
E: ID_PART_ENTRY_SIZE=2
E: ID_PART_ENTRY_DISK=7:36
E: TAGS=:systemd:

$ udevadm info /dev/loop36p2
P: /devices/virtual/block/loop36/loop36p2
N: loop36p2
L: 0
S: disk/by-label/XXXXXXXX\x20Install
S: disk/by-uuid/dbd5a3ea-5614-384e-8d15-045117946523
E: DEVPATH=/devices/virtual/block/loop36/loop36p2
E: DEVNAME=/dev/loop36p2
E: DEVTYPE=partition
E: PARTN=2
E: MAJOR=259
E: MINOR=10
E: SUBSYSTEM=block
E: USEC_INITIALIZED=10840134366
E: ID_PART_TABLE_TYPE=mac
E: ID_FS_UUID=dbd5a3ea-5614-384e-8d15-045117946523
E: ID_FS_UUID_ENC=dbd5a3ea-5614-384e-8d15-045117946523
E: ID_FS_LABEL=XXXXXXXX_Install
E: ID_FS_LABEL_ENC=XXXXXXXX\x20Install
E: ID_FS_TYPE=hfs
E: ID_FS_USAGE=filesystem
E: ID_PART_ENTRY_SCHEME=mac
E: ID_PART_ENTRY_NAME=Toast\x205.2\x20HFS\x20Optimizer
E: ID_PART_ENTRY_TYPE=Apple_HFS
E: ID_PART_ENTRY_NUMBER=2
E: ID_PART_ENTRY_OFFSET=50738
E: ID_PART_ENTRY_SIZE=1227700
E: ID_PART_ENTRY_DISK=7:36
E: DEVLINKS=/dev/disk/by-label/XXXXXXXX\x20Install /dev/disk/by-uuid/dbd5a3ea-5614-384e-8d15-045117946523
E: TAGS=:systemd:
$ udisksctl mount -b /dev/loop36
Object /org/freedesktop/UDisks2/block_devices/loop36 is not a mountable filesystem.
$ udisksctl mount -b /dev/loop36p1
Object /org/freedesktop/UDisks2/block_devices/loop36p1 is not a mountable filesystem.
$ udisksctl mount -b /dev/loop36p2
Mounted /dev/loop36p2 at /media/odeda/XXXXXXXX Install.
$ parted /dev/loop36 print
WARNING: You are not superuser.  Watch out for permissions.
Model: Loopback device (loopback)
Disk /dev/loop36: 655MB
Sector size (logical/physical): 512B/512B
Partition Table: mac
Disk Flags: 

Number  Start   End    Size   File system  Name                     Flags
 1      512B    1535B  1024B               MRKS
 2      26.0MB  655MB  629MB  hfs          Toast 5.2 HFS Optimizer

Mounting /dev/loop36 using either sudo mount or fuseiso works well to access the ISO9660.

Is all this relevant to this issue, or should I open a new issue?

mwilck commented 4 years ago

My issue was specific to very small images, thus I believe yours is different.