linux-apfs / linux-apfs-rw

APFS module for linux, with experimental write support
GNU General Public License v2.0
536 stars 35 forks source link

mount : wrong fs type, bad option, bad superblock on /dev/sda2, missing codepage #39

Open LxHobbist opened 1 year ago

LxHobbist commented 1 year ago

Hi, I have MBP 2012 and I dual boot it with Manjaro Gnome DE and macOS High Sierra. When I try to access it High Sierra Drive from Manjaro, I get an error -

Error - File type APFS not configured in

So, I have installed linux-apfs-rw from the AUR packages using the following commands $ yay -S linux-apfs-rw-dkms-git $ yay -S apfsprogs-git

Now, I get new error as below

Error mounting /dev/sda2/ at /run/media/lx/ : wrong fs type, bad option, bad superblock on /dev/sda2, missing codepage or helper program or other error

Note: The macOS partition (APFS) drive I am trying to mount in Linux has a size of 416 GB

Can you please suggest on how to fix this ? Thanks

Luflosi commented 1 year ago

This module does not yet support encryption, which is the problem you are likely running into. The error message is misleading/very broad but this module can't do anything about that. When you look at the output of dmesg, you should see a warning message "volume is encrypted, may not be read correctly".

You could try your luck with https://github.com/sgan81/apfs-fuse, I think supports encryption.

Alternatively, to share data between both operating systems, you could create a new partition with a different filesystem that both OSs support such as FAT or exFAT (but they don't support the usual Unix metadata) or you could install the ZFS driver for macOS and Linux and use ZFS (but it can be a hassle to install if you've never used it before).

LxHobbist commented 1 year ago

Hi, My apfs partition is not encrypted, when I log into macOS and look at the APFS partition from Disk utility. It says "APFS volumne , APFS". Do you think it could be something else ?

Right now, I am sharing data between OSs by HFS+, but the HFS drive is only 2 GB and is read only when check in linux which I need to remount it in write mode.

Thank you for letting me know about the ZFS, I will try that as well.

Luflosi commented 1 year ago

I don't know what else it could be. Are there any messages in dmesg when you try to mount the disk in Linux?

Can you resize the HFS+ disk in macOS to give it more space?

To always mount the HFS+ disk as writable, you could add it to /etc/fstab with the correct mount flags you seem to be already using when you remount the disk.

Be aware that ZFS is usually not compatible with the very latest Linux kernel versions. Currently only Linux 6.2 and older are officially supported. If your distro uses LTS kernels, this is not a problem. And the macOS driver is not yet what I would call "production-ready" but I have used it without problems in the past and work is being done to combine the two versions into one project.

I think ZFS is the only modern filesystem with solid write support on both Linux and macOS thanks to a mostly shared codebase.

eafer commented 1 year ago

Can you send me your filesystem's superblock? That is, the output of hexdump -C -n 4096 /dev/sda2? That would let me check if this is a supported container.

My first guess here is that the driver was not inserted for some reason, so maybe try running modprobe apfs before the mount.

Like Luflosi said, the output of dmesg could also be useful.

LxHobbist commented 1 year ago

Hi Eafer,

Here is the hexdump output. HexDump

modprobe output modprobe

dmesg - query parameters

Any particular query parameters that I need to search for, Could you please suggest ?

eafer commented 1 year ago

Thanks for the hexdump, I'll look at it now.

modprobe output

But did you attempt a mount after running modprobe? What I wanted to confirm was that the error was not caused by the driver being absent.

dmesg - query parameters

If you can, just send me everything that shows up on dmesg from the moment you attempt the mount.

EDIT: to be clear, you can just run dmesg and look at the last few lines yourself, I think something should jump out to you.

LxHobbist commented 1 year ago

When the drivers were absent and I try to mount, I get the following error

Error - File type APFS not configured in Kernel

I got this error from terminal and GUI as well.

APFS_error

Linux-apfs-rw installation: Installed the relevant Linux headers as well, based on my kernel -Linux 6.1.23-1-MANJARO

$ sudo pacman -Syu base-devel --needed

$ yay -S linux-apfs-rw-dkms-git
$ yay -S apfsprogs-git

After Installation: The error that I get after installation in terminal and GUI apfs_error2

DMESG: Attached the dmesg as an attachment, I could not really find any thing. dmesg.txt

Your help is greatly appreciated.

Thank you Ab

eafer commented 1 year ago

Like you said, there is nothing about apfs in your dmesg. Notice for instance how you get a warning about taint for the wl driver:

[    6.122495] wl: loading out-of-tree module taints kernel.

But you don't get any such warning about apfs. I don't think the module is getting inserted at all. That's why I asked what happens if you run modprobe apfs and then attempt the mount, can you try that?

EDIT: I guess you would only get the warning once because the kernel was already tainted, heh. Either way, you should be getting some apfs messages, so I still think the driver is not inserted.

LxHobbist commented 1 year ago

Hi eafer,

When I try the modprobe apfs, I get this Does that mean, the driver is not inserted ?

modprobe

Thanks

eafer commented 1 year ago

What I want to know is if the mount works after running modprobe. And if it doesn't, what is the output in dmesg then?

eafer commented 1 year ago

Are you the author of this post? If so, the problem is rather self-explaining: your volume has some form of encryption (like Luflosi said) and I haven't implemented that yet. I intend to get to it next, but it may take a while to be done.

LxHobbist commented 1 year ago

Hi eafer,

What I want to know is if the mount works after running modprobe. And if it doesn't, what is the output in dmesg then?

After running modprobe, when I try to mount the apfs.

$ sudo modprobe apfs $ sudo mount -t apfs -o rw,force /dev/sda2 /mnt/mac/

$ mount: /mnt/mac: wrong fs type, bad option, bad superblock on /dev/sda2, missing codepage or helper program, or other error. dmesg(1) may have more information after failed mount system call.

DMESG: It does not list any messages, I have attached the output again. dmesg.txt

mk01 commented 1 year ago

I can just confirm, HighSierra created volumes are not readable. that was the behaviour already with linux-apfs (the first RO incarnation)

update: My apology! It works (it just has issue with PREBOOT volume - but others do work)

great/thanks

eafer commented 1 year ago

I plan to start adding the missing features to the driver soon. So, in the future, if anybody runs into a container that won't mount or causes other problems, please consider sending me an image and I will try to implement whatever is needed.

lsnider commented 9 months ago

Hello @eafer!

I'm trying out your tool and also ran into the above mentioned error:

mount /dev/sdb2 /mnt/oldhd/
mount: /mnt/oldhd: can't read superblock on /dev/sdb2.

With force:

mount -o force  /dev/sdb2 /mnt/oldhd/
mount: /mnt/oldhd: wrong fs type, bad option, bad superblock on /dev/sdb2, missing codepage or helper program, or other error

And trying some other options:

mount -o cknodes,vol=2  /dev/sdb2 /mnt/oldhd/
mount: /mnt/oldhd: cannot mount; probably corrupted filesystem on /dev/sdb2.

Here is my hexdump

hexdump -C -n 4096 /dev/sdb2
00000000  ea 7e f5 67 1b 6d 4c ac  01 00 00 00 00 00 00 00  |.~.g.mL.........|
00000010  0c 8a f0 00 00 00 00 00  01 00 00 80 00 00 00 00  |................|
00000020  4e 58 53 42 00 10 00 00  a2 7d 50 10 00 00 00 00  |NXSB.....}P.....|
00000030  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000040  02 01 00 00 00 00 00 00  03 3e 33 11 df e0 42 98  |.........>3...B.|
00000050  a3 0b 00 d1 14 37 14 5f  2e 67 a1 00 00 00 00 00  |.....7._.g......|
00000060  0d 8a f0 00 00 00 00 00  2c 01 00 00 6c 74 00 00  |........,...lt..|
00000070  76 af 01 00 00 00 00 00  58 d8 00 00 00 00 00 00  |v.......X.......|
00000080  05 01 00 00 1c 22 00 00  03 01 00 00 02 00 00 00  |....."..........|
00000090  f0 21 00 00 2c 00 00 00  70 63 01 00 00 00 00 00  |.!..,...pc......|
000000a0  11 2c 07 00 00 00 00 00  01 04 00 00 00 00 00 00  |.,..............|
000000b0  00 00 00 00 64 00 00 00  04 04 00 00 00 00 00 00  |....d...........|
000000c0  07 04 00 00 00 00 00 00  20 61 01 00 00 00 00 00  |........ a......|
000000d0  00 03 02 00 00 00 00 00  79 1e 14 00 00 00 00 00  |........y.......|
000000e0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
000003d0  00 00 00 00 00 00 00 00  ae a7 0c 3d 00 00 00 00  |...........=....|
000003e0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
000004f0  00 00 00 00 00 00 00 00  ca 00 d2 00 00 00 00 00  |................|
00000500  30 e9 42 68 35 6b 46 e5  ad bb c9 cf 66 da 27 2d  |0.Bh5kF.....f.'-|
00000510  b1 73 2b 01 00 00 00 00  01 00 00 00 00 00 00 00  |.s+.............|
00000520  01 00 04 00 08 00 00 00  00 00 00 00 00 00 00 00  |................|
00000530  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000540  00 00 00 00 00 00 00 00  52 0d 07 00 00 00 00 00  |........R.......|
00000550  02 04 00 00 00 00 00 00  80 6d 59 00 00 00 00 00  |.........mY.....|
00000560  00 00 20 00 00 00 00 00  1a 64 db b2 55 04 05 00  |.. ......d..U...|
00000570  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|

And dmesg:

[6994839.523950] APFS (800012g): bad checksum for object in block 0x72c11 (apfs_read_object_block:352)
[6994839.524002] APFS (800012g): unable to read container object map (apfs_map_volume_super:418)
[6994884.422066] APFS (800012g): failed to read object block 0x1d4e5422b6781a9d (apfs_read_object_block:344)
[6994884.422118] APFS (800012g): object read failed for bno 0x1d4e5422b6781a9d (apfs_read_node:102)
[6994884.422166] APFS (800012g): unable to read volume block (apfs_map_volume_super:431)

If you're up for looking into this, I'm happy to try compiling different branches or whatever tests we'd like to do. Thanks!

If it's helpful, here is more output from other tools about the drive:

/apfsutil /dev/sdb
Device /dev/sdb opened. Size is 1000204886016
Found partitions:
C12A7328-F81F-11D2-4BBA-00A0C93EC93B BE7C3E16-998C-4117-9D92-0102F257022E 0000000000000028 0000000000064027 0000000000000000 EFI System Partition
7C3457EF-0000-11AA-11AA-00306543ECAC 9720C2E3-3AEA-4C26-FB94-E8C45EBA4048 0000000000064028 0000000074706D87 0000000000000000 iMac HDD
First APFS partition is 1
parted /dev/sdb
GNU Parted 3.3
Using /dev/sdb
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) p
Model: APPLE HD D WDC WD10EALX-4 (scsi)
Disk /dev/sdb: 1000GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:

Number  Start   End     Size    File system  Name                  Flags
 1      20.5kB  210MB   210MB   fat32        EFI System Partition  boot, esp
 2      210MB   1000GB  1000GB               iMac HDD
 fdisk /dev/sdb

Welcome to fdisk (util-linux 2.34).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.

Command (m for help): p
Disk /dev/sdb: 931.53 GiB, 1000204886016 bytes, 1953525168 sectors
Disk model: D WDC WD10EALX-4
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 96C489FD-CD0A-4805-8F33-D2E3E0418040

Device      Start        End    Sectors   Size Type
/dev/sdb1      40     409639     409600   200M EFI System
/dev/sdb2  409640 1953525127 1953115488 931.3G unknown
eafer commented 9 months ago

Hi @lsnider, what did you get on dmesg when you attempted to mount without the cknodes option?

EDIT: never mind, no way that block number is correct.

eafer commented 9 months ago

That's a fusion drive, right? Those aren't supported yet. There is still a bug though, because the error message shouldn't be so confusing.

lsnider commented 8 months ago

It is a fusion drive. Do they have some type of mirror layer on them or something?

Happy to test any updates to see if we can detect that it's a fusion drive.

eafer commented 8 months ago

It is a fusion drive. Do they have some type of mirror layer on them or something?

I've never worked with one, but my understanding is that they have two drives: a hard disk for storage, and a solid state drive as come sort of cache to make things faster.

Happy to test any updates to see if we can detect that it's a fusion drive.

Thanks. I just pushed a patch for that, let me know if you can try it out.

lsnider commented 8 months ago

We got an improved error messages and they match regardless of the mount options. But it doesn't mention fusion drive (not sure if that's the plan)?

I'm being overly verbose here in case I'm trying it wrong.

> git checkout development
> git pull
   41af006..7df84d2  development -> origin/development
Updating 41af006..7df84d2
Fast-forward
 README.rst |  3 ++-
 super.c    | 73 ++++++++++++++++++++++++++++++++++++++++++++-----------------------------
 2 files changed, 46 insertions(+), 30 deletions(-)
 > make
./genver.sh
make -C /lib/modules/5.4.0-167-generic/build M=/root/oldhd/linux-apfs-rw
make[1]: Entering directory '/usr/src/linux-headers-5.4.0-167-generic'
  CC [M]  /root/oldhd/linux-apfs-rw/super.o
  LD [M]  /root/oldhd/linux-apfs-rw/apfs.o
  Building modules, stage 2.
  MODPOST 1 modules
  CC [M]  /root/oldhd/linux-apfs-rw/apfs.mod.o
  LD [M]  /root/oldhd/linux-apfs-rw/apfs.ko
make[1]: Leaving directory '/usr/src/linux-headers-5.4.0-167-generic'
> rmmod apfs
> insmod apfs.ko
> mount  /dev/sdb2 /mnt/oldhd/
mount: /mnt/oldhd: wrong fs type, bad option, bad superblock on /dev/sdb2, missing codepage or helper program, or other error.

> mount -o cknodes,vol=2  /dev/sdb2 /mnt/oldhd/
mount: /mnt/oldhd: wrong fs type, bad option, bad superblock on /dev/sdb2, missing codepage or helper program, or other error.

> mount -o force  /dev/sdb2 /mnt/oldhd/
mount: /mnt/oldhd: wrong fs type, bad option, bad superblock on /dev/sdb2, missing codepage or helper program, or other error.
eafer commented 8 months ago

Sadly I don't control the error message you get in the terminal, that comes from the mount tool. The improved message about the fusion drive should show up on dmesg.

lsnider commented 8 months ago

Oh right, you nailed it then. Log line from dmesg

[7529698.628756] APFS (apfs): fusion drives are not supported
lwj1980s commented 1 month ago

I am encountering the same issue.

eafer commented 1 month ago

@lwj1980s Any output on dmesg?

lwj1980s commented 1 month ago

@lwj1980s Any output on dmesg?

Hi, i paste all the dmesg output in pictures above

eafer commented 1 month ago

@lwj1980s You are getting a "bad blocksize 0" error. I usually see this when I try to mount the whole disk instead of the apfs partition. What was the mount command that you ran, exactly?

lwj1980s commented 1 month ago

What was the mount command that you ran, exactly?

yes,I followed your suggestion and success to mount the disk,and thank you very very munch~~~