search

linux-apfs / linux-apfs-rw

APFS module for linux, with experimental write support
GNU General Public License v2.0
532 stars 35 forks source link

readwrite option not working #17

Closed muammar closed 10 months ago

muammar commented 3 years ago

Thanks for this effort you are making by enabling apfs on Linux. I used your other tool to format an external hard disk, and everything seems to work. When I mount the newly formatted partition with: 

sudo mount -o readwrite /dev/sdc1 Backup

And I try to create anything either with sudo or as a regular user; it does not let me do that.


mkdir: cannot create directory ‘hola’: Read-only file system

PS. This is on Debian Unstable.

eafer commented 3 years ago

That's weird. Any output on dmesg?

muammar commented 3 years ago 
[   66.504090] ucsi_acpi USBC000:00: PPM init failed (-110)
[   86.067204]  sdb: sdb1
[  105.562233]  sdb: sdb1
[  327.690887] EXT4-fs (sdb1): mounted filesystem with ordered data mode. Opts: (null)
[  423.904292]  sdb: sdb1
[  440.258117]  sdb: sdb1
[  530.870511] APFS (26g): experimental write support is enabled
[  535.085950] APFS (26g): large devices are not supported
[  535.085953] APFS (26g): aborting transaction

Large devices are not supported. That’s the problem. This is a 2TB hard drive.

eafer commented 3 years ago

2TB is not that big, that message is intended for containers above 7 TB or so. Did you say that you used my mkfs? The problem is most likely over there, I'll take a look.

eafer commented 3 years ago

I just pushed a patch to the mkfs, when you can let me know if it fixed your problem. One thing you should know: large containers above 1.4Tb or so may be usable with my module alone, but the official implementation of apfs will probably reject them. I'll fix this eventually, but I don't have the time now.

muammar commented 3 years ago

@eafer sorry that I got back to you late. I tried again, but it still does not work. 

[  149.340916] sd 10:0:0:0: [sdb] Spinning up disk...
[  150.362462] .....ready
[  154.458851] sd 10:0:0:0: [sdb] 3906963456 512-byte logical blocks: (2.00 TB/1.82 TiB)
[  154.459092] sd 10:0:0:0: [sdb] Write Protect is off
[  154.459097] sd 10:0:0:0: [sdb] Mode Sense: 47 00 10 08
[  154.459376] sd 10:0:0:0: [sdb] No Caching mode page found
[  154.459385] sd 10:0:0:0: [sdb] Assuming drive cache: write through
[  154.522900]  sdb: sdb1
[  154.524696] sd 10:0:0:0: [sdb] Attached SCSI disk
[  356.970552]  sdb: sdb1
[  515.548842] APFS (26g): experimental write support is enabled
[  518.284555] APFS (26g): unable to read container object map
[  518.284564] APFS (26g): aborting transaction
(base)

What could be happening in this case? I tried mounting the hard disk formatted with macOS, and also with your tools, and in both cases, I am unable to write. I will just check the disk and try again.

alexmyczko commented 3 years ago

@muammar I'm trying to create a debian package, if someone wants to help? https://mentors.debian.net/package/linux-apfs-rw/

muammar commented 3 years ago

@muammar I'm trying to create a debian package; if someone wants to help? https://mentors.debian.net/package/linux-apfs-rw/

I would be interested, and if it works, I can try to help you upload it to the Debian archives (I am a DD). I will install it on my Debian unstable and check if I get writing support.

alexmyczko commented 3 years ago

i guess it's far from ready, but after you install, i'm able to run dkms build linux-apfs-rw/0+git20210928 but somehow couldn't get a kernel module, to modprobe, any help is welcome.

alexmyczko commented 3 years ago

@muammar i think this one looks good already: https://mentors.debian.net/package/apfsprogs/ (except for the missing unicode license part, needs clarification with upstream)

eafer commented 3 years ago

@alexmyczko Thanks for your effort, it would be nice to see this packaged for Debian. I don't have any experience with that, but let me know if I can help. You should start a separate issue though, since Muammar's problem is not yet fixed.

AdityaGarg8 commented 2 years ago

Hello Do I need to use the apfsprogs tools as well to enable write support?

eafer commented 2 years ago

Do I need to use the apfsprogs tools as well to enable write support?

No, you need to use the readwrite mount option. The README explains it, but be careful, it's likely to cause some corruption.

AdityaGarg8 commented 2 years ago

Well I tried with readwrite option on a 4GB APFS formatted USB drive, but was not successful.

BTW, I noticed there was a demand for a Debian package for the same. I am hosting this driver in my APT repo. It shall be updated regularly since I have also set a scheduled job to look for updates from this repo of yours and publish a newer version if available.

AdityaGarg8 commented 2 years ago

Are there any logs that I can share to fix my write issue?

AdityaGarg8 commented 2 years ago

@eafer

This is what I get on mounting as readwrite

aditya@MacBook:~$ sudo mount -o readwrite /dev/sda2 /mnt
[sudo] password for aditya: 
mount: /mnt: WARNING: source write-protected, mounted read-only.

And my dmesg

aditya@MacBook:~$ sudo dmesg | grep apfs
[  101.590889] apfs: loading out-of-tree module taints kernel.
[  101.590937] apfs: module verification failed: signature and/or required key missing - tainting kerne
AdityaGarg8 commented 2 years ago

Also, the name of my APFS volume is not being displayed in Files app, and GNOME disks also marks my USB Drive as no mounted, in-spite of it actually being mount.

eafer commented 2 years ago 
***@***.***:~$ sudo mount -o readwrite /dev/sda2 /mnt
[sudo] password for aditya: 
mount: /mnt: WARNING: source write-protected, mounted read-only.

This doesn't seem to be an apfs problem. Your device at /dev/sda2 appears to be on read-only mode for some reason. Maybe it has a physical switch for that?

Also, the name of my APFS volume is not being displayed in Files app

APFS containers don't have a label themselves. You do have a label for each volume, but a single partition can hold multiple volumes, so I don't think there is any sane way for linux file managers to report that. If they wanted they could report the gpt partition label when it's available, but that doesn't involve APFS.

GNOME disks also marks my USB Drive as no mounted, in-spite of it actually being mount.

I have no idea how file managers tell apart mounted and unmounted devices, but it probably doesn't involve the driver directly. Maybe they check /proc/mounts. I just checked on thunar and it seems to work, but not every single time. I'll try to look into it at some point, but maybe you'll get faster results if you just ask the nautilus devs.

AdityaGarg8 commented 2 years ago

This doesn't seem to be an apfs problem. Your device at /dev/sda2 appears to be on read-only mode for some reason. Maybe it has a physical switch for that?

No it doesn't have a physical switch. I had formatted it using macOS 12 to APFS. Should I use apfsprogs?

eafer commented 2 years ago

No it doesn't have a physical switch

What do you get if you run blockdev --getro /dev/sda2? I think you'll get a 1, which means read-only. I don't know why, but maybe there's something on dmesg that explains it.

AdityaGarg8 commented 2 years ago

What do you get if you run blockdev --getro /dev/sda2? I think you'll get a 1, which means read-only. I don't know why, but maybe there's something on dmesg that explains it.

aditya@MacBook:~$ sudo blockdev --getro /dev/sda2
[sudo] password for aditya: 
0

I am getting a 0.

eafer commented 2 years ago

And it still doesn't mount? Weird. What about after you try to mount with readwrite, do you still get a 0?

AdityaGarg8 commented 2 years ago

Yeah, I am still getting a 0.

I just noticed that I am able to rename files after mounting as readwrite. But I am not able to copy new files or edit existing files.

AdityaGarg8 commented 2 years ago 
aditya@MacBook:/mnt$ sudo cp /media/aditya/"TOSHIBA EXT1"/ubuntu/iso-firmware.deb ./
cp: cannot create regular file './iso-firmware.deb': Read-only file system
AdityaGarg8 commented 2 years ago

I did manage to write one file luckily, but now its refusing to even rename.

eafer commented 2 years ago

You can rename files on a read-only mount? What do you get for your apfs filesystem on /proc/mounts? If you can also send me all your dmesg since the first mount.

eafer commented 2 years ago

I did manage to write one file luckily, but now its refusing to even rename.

But are you still getting mount: /mnt: WARNING: source write-protected, mounted read-only, or is that gone?

AdityaGarg8 commented 2 years ago

But are you still getting mount: /mnt: WARNING: source write-protected, mounted read-only, or is that gone?

I am not getting this now.

AdityaGarg8 commented 2 years ago

I reformatted the USB to APFS again using macOS and added a text file there. Now, I was able to rename, and even edit the file. But, as I tried to copy a debian package to the USB, it suddenly became read-only again.

Just in case you wish to know, my USB drive is 4Gbs.

AdityaGarg8 commented 2 years ago 
aditya@MacBook:~$ sudo dmesg | grep apfs
[  103.578879] apfs: loading out-of-tree module taints kernel.
[  103.578939] apfs: module verification failed: signature and/or required key missing - tainting kernel
aditya@MacBook:~$ sudo dmesg > apfs.log
aditya@MacBook:~$ sudo dmesg | grep APFS
[  186.334622] APFS (33g): experimental write support is enabled
[  203.507090] APFS: space manager is versioned
[  227.194568] APFS (33g): internal pool seems full
[  227.194572] APFS (33g): unable to read container object map
[  227.194574] APFS (33g): aborting transaction
[  227.194576] APFS (33g): failed to delete orphan inode 0x9e
AdityaGarg8 commented 2 years ago

Do you want the full dmesg?

AdityaGarg8 commented 2 years ago 
/dev/sda2 /mnt apfs ro,relatime 0 0

I am getting this in /proc/mounts

eafer commented 2 years ago

By any chance, is the volume close to being full? What's the size?

Do you want the full dmesg?

No need, thanks. It's a problem with the driver then, no idea why you were getting the source write-protected message before. The mount becomes read-only as soon as it finds something weird, to try to avoid corrupting your data. I'll look into it as soon as I'm done working on snapshots.

AdityaGarg8 commented 2 years ago

By any chance, is the volume close to being full? What's the size?

It's around 4Gbs. Not close to full since it has only 1 text file.

eafer commented 2 years ago

If you can send me an image for the freshly formatted disk it would help a lot.

AdityaGarg8 commented 2 years ago

If you can send me an image for the freshly formatted disk it would help a lot.

Image as in?

eafer commented 2 years ago

You can first wipe all data in the usb device with dd if=/dev/zero of=/dev/sda2. Then format the device for apfs the same way you always do. Then run dd if=/dev/sda2 of=img and you'll get a 4GB image file called img. You can send me that, or compress it first to make it easier.

AdityaGarg8 commented 2 years ago

I'll do it later when I shall get free.

eafer commented 2 years ago

Thanks, no rush. Make sure you understand what you are doing, if you zero the wrong device you'll lose data.

AdityaGarg8 commented 2 years ago

Thanks, no rush. Make sure you understand what you are doing, if you zero the wrong device you'll lose data.

Yeah I got it.

AdityaGarg8 commented 2 years ago

Will I have to do dd if=/dev/zero of=/dev/sda2, cause it's taking a lot of time.

eafer commented 2 years ago

Will I have to do dd if=/dev/zero of=/dev/sda2, cause it's taking a lot of time.

That's just in case you have private data left on the device, that was deleted but can be recovered. It should also make the final compressed file much smaller. But you can skip it if you prefer.

eafer commented 2 years ago

I'm thinking, maybe /dev/sda2 is larger than 4GB, and you just made a smaller container inside of it? Because otherwise I can't see why this would take so long. If you don't know, you can try blockdev --getsz /dev/sda2

AdityaGarg8 commented 2 years ago

I'm thinking, maybe /dev/sda2 is larger than 4GB, and you just made a smaller container inside of it? Because otherwise I can't see why this would take so long. If you don't know, you can try blockdev --getsz /dev/sda2

/dev/disk3 (external, physical):
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:      GUID_partition_scheme                        *4.0 GB     disk3
   1:                        EFI ⁨EFI⁩                     209.7 MB   disk3s1
   2:                 Apple_APFS ⁨Container disk4⁩         3.8 GB     disk3s2

/dev/disk4 (synthesized):
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:      APFS Container Scheme -                      +3.8 GB     disk4
                                 Physical Store disk3s2
   1:                APFS Volume ⁨Untitled⁩                802.8 KB   disk4s1

This is my partition table just in case you wish to know.

AdityaGarg8 commented 2 years ago

You can first wipe all data in the usb device with dd if=/dev/zero of=/dev/sda2. Then format the device for apfs the same way you always do. Then run dd if=/dev/sda2 of=img and you'll get a 4GB image file called img. You can send me that, or compress it first to make it easier.

https://drive.google.com/file/d/1qV9NYrkb6yA-JPS9ugxdOaUiz7dV0BnT/view?usp=sharing

You can download the image from here. Please inform me when you get it cause I want to free up my drive's space so I shall delete it after you get it.

eafer commented 2 years ago

You can download the image from here. Please inform me when you get it cause I want to free up my drive's space so I shall delete it after you get it.

Got it, thanks. I can reproduce your issue, I'll look into it now.

AdityaGarg8 commented 2 years ago

I’ll delete the file from my end then.

eafer commented 2 years ago

@AdityaGarg8 I just pushed a patch, when you can let me know if it solves your problem.

AdityaGarg8 commented 2 years ago

I'll send you the result later today.

AdityaGarg8 commented 2 years ago

I am having a new issue now. Now it doesn't become read-only, but the file it copies to the USB actually doesn't get copied. Eg:- I copied a short 20Mb video and it failed to play. Infact, I ejected and replugged my USB, it was gone.

aditya@MacBook:~$ sudo dmesg | grep apfs
[sudo] password for aditya: 
[  933.979984] apfs: loading out-of-tree module taints kernel.
[  933.980029] apfs: module verification failed: signature and/or required key missing - tainting kernel
aditya@MacBook:~$ sudo dmesg | grep APFS
[ 1029.349915] APFS (31g): experimental write support is enabled
[ 1036.327987] APFS: space manager is versioned
[ 1192.982724] APFS (31g): transaction commit failed
[ 1192.982726] APFS (31g): aborting transaction
[ 1213.092607] APFS (31g): experimental write support is enabled
[ 1310.425643] APFS (31g): transaction commit failed
[ 1310.425644] APFS (31g): aborting transaction

This is what I got.

AdityaGarg8 commented 2 years ago

For some reason I also am not able to unmount my USB if it is mounted in readwrite mode.