exfatprogs / exfatprogs

exFAT filesystem userspace utilities
GNU General Public License v2.0
199 stars 52 forks source link

Some disks formatted with mkfs.exfat are not recognized by Windows and corrupted by Windows #261

Open ohyeaah opened 3 months ago

ohyeaah commented 3 months ago

I always used to format my external 2TB HDD using mkfs.exfat and then use it on a Windows PC.

But it seems Windows changed something in their code and now some disks formatted with mkfs.exfat don't get recognized on Windows anymore. And Windows also changes the checksum (perhaps more) in the exfat header without any user interaction. This means the disk will no longer be recognized under Linux either after it was connected to a Windows PC once.

When I run dmesg I can see Linux says the checksum is no longer valid (after it was connected to a Windows PC).

!!! So again to anyone who has formatted a disk using mkfs.exfat don't connect it to a Windows PC right now because all your data could get lost. I could fix my disk by correcting the checksum manually using a PHP script. The correct checksum can be seen in dmesg.

I verified this issue using different disks and different Windows PCs. It only happens for some of my disks. The exact size of one of my affected disks is 2000365289472.

I am using Windows 10 and Debian bookworm with exfatprogs 1.2.0.

I am formatting my disks without partition table (sudo mkfs.exfat /dev/sd...).

namjaejeon commented 3 months ago

Can you share the checksum error print to us ?

namjaejeon commented 3 months ago

I am formatting my disks without partition table (sudo mkfs.exfat /dev/sd...).

If you create partition table, problem is still reproduced ?

namjaejeon commented 3 months ago

@ohyeaah Let me know if this problem could happen with another exfat's mkfs.

https://github.com/relan/exfat

ohyeaah commented 3 months ago

@namjaejeon I am glad to tell you that I isolated the problem.

There is a "removable" property for each USB disk/flash drive. On Linux the value can be read using /sys/class/block/sda/removable for sda. This value comes from the hardware itself and also Windows uses it.

For disks which have set this value to 1 (=removable) there is currently no problem.

However when this value is 0 (=fixed) there are currently two problems when using mkfs.exfat. Both problems only occur if you are formatting a disk directly without partition table.

The first problem is that mkfs.exfat does not set the disk signature at 0x1b8. This will cause Windows to write its own signature without updating the checksum which will damage the disk. This will happen by plugging in the disk and without any further user interaction.

The second problem is that the missing partition table will cause that Windows will not recognize the exfat partition. So the whole disk will show up as empty in the Windows disk manager.

The mkfs.fat command currently does everything properly the way I explained:

Create MBR partition depending on wether the disk is fixed

So when the disk is fixed and you are formatting a disk directly (without partition table) you have to create a disk signature and MBR partition.

ohyeaah commented 3 months ago

Perhaps it's easier to show an error that formatting a disk directly as exfat is not supported if removable=0.

namjaejeon commented 3 months ago

Okay, Can you contribute the patch for this problem ?

ohyeaah commented 3 months ago

Perhaps some day but currently I have no time.

herok94 commented 2 months ago

I had the same problem,mkfs.exfat format The exfat file system is not recognized on windows,but Formatted on windows, recognized on linux

namjaejeon commented 2 months ago

@herok94 Please format exfat after creating partition table. don't do format against /dev/sda. i.e. you need to format sda1,2,3, etc.. After all, you want that mkfs.exfat create partition table when formatting raw device(no partition table, /dev/sda).