Linux (and Windows?) fail to mount resultant UDF drive with bs=4096

[atompkins]

Linux Mint 17.1 Cinnamon sees the fake partition but not the UDF. Interestingly my Windows 7 installation says that the disk is not initialised. Let me know what diagnostics you need.

[JElchison]

thanks for the report, @atompkins! the fake partition is only in place to help Windows identify the UDF disk. if Linux cannot read the UDF partition, then it's likely not the fault of the fake partition. most likely, Mint is having an issue auto-mounting the disk. have you tried manually mounting?

example: sudo mount -v -t udf /dev/sdX /mnt/external, where X is the letter associated with your drive

info that might help us isolate the issue:

• is the issue reproducible?
  • what are the exact environment/steps that i can use to repro it?
• does the issue still occur if you use a different flash drive?
• does the issue still occur if you use a hard drive?
• does the issue still occur if you use a different distro of Linux to run format-udf?
• does the issue still occur if you use OS X to run format-udf?
• what version of udftools are you using?
• what version of format-udf are you using?

also, please post output from each of the following:

• the actual format-udf command you used
• sudo fdisk -l
  • is your drive in the list?
• sudo blkid -c /dev/null /dev/sdX, where X is the letter associated with your drive

[atompkins]

adrian@Tsunami ~ $ sudo mount -v -t udf /dev/sdf /media/adrian
[sudo] password for adrian: 
mount: wrong fs type, bad option, bad superblock on /dev/sdf,
       missing codepage or helper program, or other error
       In some cases useful info is found in syslog - try
       dmesg | tail  or so

I have tried one internal disk, one external disk on USB3 and one flash drive so far, all with the same result. I have tried with v1.1.2, v.1.1.1 & v1.1.0. v1.0.3 seems to work OK on Mint.

udftools version 1.0.0b3-14.2ubuntu1

adrian@Tsunami ~/Downloads/format-udf-1.1.2 $ ./format-udf.sh sdf label
[+] Testing dependencies...
[+] Looking for drive listing tool... using /sbin/blockdev
[+] Looking for unmount tool... using /bin/umount
[+] Looking for UDF tool... using /usr/bin/mkudffs
[+] Validating arguments...
[+] Gathering drive information...
[sudo] password for adrian: 
/dev/sdf: LABEL="Samsung500GB" TYPE="udf" 
2105            
RO    RA   SSZ   BSZ   StartSec            Size   Device
rw   256   512  4096          0    500107862016   /dev/sdf
rw   256   512  4096       2048    500104691712   /dev/sdf1
The above-listed drive (and partitions, if any) will be completely erased.
Type 'yes' if this is what you intend:  yes
[+] Detecting total size...
[*] Using total size of 500107862016
[+] Validating detected total size...
[+] Detecting physical block size...
[*] Using block size of 4096
[+] Validating detected block size...
[+] Unmounting drive...
umount: /dev/sdf: not mounted
[+] Zeroing out any existing partition table on drive...
4096+0 records in
4096+0 records out
16777216 bytes (17 MB) copied, 0.00715565 s, 2.3 GB/s
[+] Formatting /dev/sdf ...
start=0, blocks=8, type=RESERVED 
start=8, blocks=3, type=VRS 
start=11, blocks=245, type=USPACE 
start=256, blocks=1, type=ANCHOR 
start=257, blocks=16, type=PVDS 
start=273, blocks=1, type=LVID 
start=274, blocks=122096115, type=PSPACE 
start=122096389, blocks=1, type=ANCHOR 
start=122096390, blocks=239, type=USPACE 
start=122096629, blocks=16, type=RVDS 
start=122096645, blocks=1, type=ANCHOR 
[+] Writing fake MBR...
16+0 records in
16+0 records out
16 bytes (16 B) copied, 0.00101932 s, 15.7 kB/s
2+0 records in
2+0 records out
2 bytes (2 B) copied, 4.7879e-05 s, 41.8 kB/s
[*] Successfully formatted 
Please disconnect/reconnect your drive now.

adrian@Tsunami ~ $ sudo fdisk -l
[sudo] password for adrian: 

Disk /dev/sda: 2000.4 GB, 2000398934016 bytes
255 heads, 63 sectors/track, 243201 cylinders, total 3907029168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x5a8f0eb9

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *        2048  3907026943  1953512448    7  HPFS/NTFS/exFAT

Disk /dev/sdb: 2000.4 GB, 2000398934016 bytes
255 heads, 63 sectors/track, 243201 cylinders, total 3907029168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0xa0ef998f

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1            2048  3907026943  1953512448    7  HPFS/NTFS/exFAT

Disk /dev/sdc: 500.1 GB, 500107862016 bytes
255 heads, 63 sectors/track, 60801 cylinders, total 976773168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk identifier: 0xc56417a6

   Device Boot      Start         End      Blocks   Id  System
/dev/sdc1            2048   976769023   488383488    7  HPFS/NTFS/exFAT

Disk /dev/sdd: 1000.2 GB, 1000204886016 bytes
255 heads, 63 sectors/track, 121601 cylinders, total 1953525168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x5864bc50

   Device Boot      Start         End      Blocks   Id  System
/dev/sdd1            2048  1953521663   976759808    7  HPFS/NTFS/exFAT

Disk /dev/sde: 250.1 GB, 250059350016 bytes
255 heads, 63 sectors/track, 30401 cylinders, total 488397168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x42cf5445

   Device Boot      Start         End      Blocks   Id  System
/dev/sde1   *        2048      206847      102400    7  HPFS/NTFS/exFAT
/dev/sde2         1548288   244197375   121324544    7  HPFS/NTFS/exFAT
/dev/sde3       244199422   488396799   122098689    5  Extended
/dev/sde5       244199424   456396689   106098633   83  Linux
/dev/sde6       456398848   488396799    15998976   82  Linux swap / Solaris

Disk /dev/sdf: 500.1 GB, 500107862016 bytes
255 heads, 63 sectors/track, 60801 cylinders, total 976773168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk identifier: 0x00000000

   Device Boot      Start         End      Blocks   Id  System
/dev/sdf1               0   122096645    61048323    b  W95 FAT32

Disk /dev/sdf1: 62.5 GB, 62513482752 bytes
255 heads, 63 sectors/track, 7600 cylinders, total 122096646 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk identifier: 0x00000000

     Device Boot      Start         End      Blocks   Id  System
/dev/sdf1p1               0   122096645    61048323    b  W95 FAT32

adrian@Tsunami ~ $ sudo blkid -c /dev/null /dev/sdf
[sudo] password for adrian: 
adrian@Tsunami ~ $ 

[JElchison]

Thanks for the info. It's a busy week for me, but I will attempt to reproduce your issue sometime soon.

In the meantime, are there any helpful clues in your syslog?

Thanks!

[JElchison]

sorry for the scattered requests. can you also please provide output from:

• uname -a
• lsb_release -a

many thanks.

[JElchison]

it's looking like Mint isn't wanting to mount for some reason. please attempt mounting with both of the following and report output:

sudo mount -v -t udf /dev/sdf1 /media/adrian

sudo mount -v -t udf /dev/sdf1p1 /media/adrian

[JElchison]

also, please report output from mount command (just by itself)

[JElchison]

other ideas:

sudo mount -v -t udf -o noauto,nofail,uhelper=udisks2,nodev,nosuid,iocharset=utf8,umask=0077,comment=x-gvfs-show /dev/sdf /media/adrian

also, it could be that Mint is improperly detecting the block size:

sudo mount -t udf -o bs=4096 /dev/sdf /media/adrian

and, of course, the union of these two:

sudo mount -v -t udf -o bs=4096,noauto,nofail,uhelper=udisks2,nodev,nosuid,iocharset=utf8,umask=0077,comment=x-gvfs-show /dev/sdf /media/adrian

[JElchison]

fyi, i just downloaded/installed linuxmint-17.1-cinnamon-64bit.iso onto a brand new VM, and was successfully able to format a block device using format-udf commit ac39e16 (the current latest, and functionally equivalent to v1.1.2).

are you able to reproduce this issue on multiple machines other than your Tsunami machine? i'd be interested if you see success on either Ubuntu 14.04.2 LTS or Ubuntu 15.04.

[JElchison]

ran 2 separate tests inside the Mint VM, one with a virtual hard drive (succeeded) and one with a real flash drive i had laying around (also succeeded).

[atompkins]

Sorry for the delay in getting back to you. Busy here too! Just reproduced your test in VM. Worked OK. Only difference I can see is the block size is different. I have reproduced the failure on my laptop. Seems that the fake MBR is incorrect for 4K devices.

[JElchison]

Thanks for reporting back.

I'm trying to make sense of your data. You initially said that you tried one internal disk, one external disk on USB3, and one flash drive. Do all three of these have a block size of 4096?

Also, if you're willing to do a bit of legwork, it would be super-helpful to collect complete information about your environment in this thread. Would you be willing to report output from the following commands? (This is just a condensed list of what I've already asked for above.)

• syslog
• uname -a
• lsb_release -a
• mount
• sudo mount -v -t udf /dev/sdf1 /media/adrian
• sudo mount -v -t udf /dev/sdf1p1 /media/adrian
• sudo mount -v -t udf -o noauto,nofail,uhelper=udisks2,nodev,nosuid,iocharset=utf8,umask=0077,comment=x-gvfs-show /dev/sdf /media/adrian
• sudo mount -t udf -o bs=4096 /dev/sdf /media/adrian
• sudo mount -v -t udf -o bs=4096,noauto,nofail,uhelper=udisks2,nodev,nosuid,iocharset=utf8,umask=0077,comment=x-gvfs-show /dev/sdf /media/adrian

In the meantime, I'll attempt to find a block device with bs=4096 so I can attempt to reproduce using that.

Thanks!

[atompkins]

I have just been successful with a smaller flash drive that has a 512 block size. I'll try to look at your list as soon as I can. I don't have the first flash drive right now but I'll try to get it back and let you know. Both the HDDs are 4K.

[GreatEmerald]

Confirmed on openSUSE 13.2, Plasma 5, an Advanced Format USB3 SAT device. Mounting manually with -o bs=4096 works, but automounting doesn't. Also, without -o, mount fails and the kernel log says:

[14203.907566] UDF-fs: warning (device sdc): udf_load_vrs: No VRS found
[14203.907580] UDF-fs: warning (device sdc): udf_fill_super: No partition found (2)

This happens even if I format the drive manually with mkudffs --media-type=hd -b 4096. Same thing on Windows: with -b 4096, it doesn't show the contents but rather asks to format the drive.

When using 512 block size, it works everywhere fine. But obviously that's not ideal for performance, and doesn't scale past 2 TiB.

I'm thinking this may be related to the kernel issue where it misdetects block sizes then using SAT: http://nunix.fr/index.php/linux/7-astuces/65-too-hard-to-be-above-2tb Perhaps that confuses udisks2 too. As for Windows... Does it have the capability of mounting 4K UDF to begin with? I can't find a definitive answer...

[JElchison]

Note: See #13

[tkalicki]

I'll shed some light on this issue.

Recently, I bought WD Red 3TB drive. I wanted to have single partition for storage and maintain Windows 8.1 + Linux compatibility. I tried to set up UDF and but I've faced some obstacles:

• Windows requires partition table. MBR supports only partitions up to 2TB, so I had to use GPT. To be honest, I prefer GPT over MBR anyway. No problem here.
• Windows initiates GPT with useless/pointless 128MB "Microsoft Reserved Partition". I don't need it, so I used fdisk for GPT instead.

fdisk reading for this HDD:

Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes

For testing, I created 100MB "Microsoft basic data partition". Windows sees this partition without any issues. I initialized UDF via Windows format: format D: /FS:UDF. Then I switched to Linux and tried to mount this partition. I was able to mount it with bs=512.

BTW: for block size "discovery" I used:

mount -t udf -o bs=512 /dev/sda1 /mnt
mount -t udf -o bs=1024 /dev/sda1 /mnt
mount -t udf -o bs=2048 /dev/sda1 /mnt
mount -t udf -o bs=4096 /dev/sda1 /mnt

Only one will work.

Unfortunately, bs=512 is not good for me since max partition size is 2TB and sub-optimal performance on 4096B physical sector HDD.

• Them I tried Linux mkuddfs:

mkuddfs --blocksize=2048 --media-type=hd --utf8 /dev/sda1
mkuddfs --blocksize=4096 --media-type=hd --utf8 /dev/sda1

Those work on Linux. 2048 is default mount block size on Linux, but for 4096 you need to manually provide -o bs=4096.

• Anything with block size different than 512 doesn't work on Windows! On Windows, UDF block size needs to be equal to logical sector size (see fdisk reading).
• I'm using NTFS with block size 4096 right now.

[JElchison]

Performing some cleanup on open issues. Based on what @tkalicki reported above, I feel comfortable closing this issue with the clarifying text I've just added on the README. Essentially, there's not much that format-udf can do if OS X has trouble auto-mounting drives with certain block sizes.

Related discussion is still welcome in this thread. I'm always more that happy to revisit this issue if there's any way that format-udf can produce drives that are more compatible on target OSes (while respecting the device geometry, of course).