Segmentation fault.

[Johnnie390]

Hello all,

I am running a 4.15 kernel on Debian Stretch. The machine is a Macbook Pro mid-2017 15 inch retina.

libfuse-dev (2.9.7-1) ibicu-dev (57.1-8) icu-devtools (57.1-8)

Apfs-fuse worked up to a week ago mounting APFS container(s) on linux in R/O mode. Compliments on your efforts.

A week is a long time in Linux, after a few apt-get update sessions, apfs-fuse is no longer working (segmentation fault) -

apfs-fuse /dev/nvme0n1p2 /uu Segmentation fault

GDB output:

gdb apfs-fuse

(gdb) set args /dev/nvme0n1p2 /uu (gdb) run

Starting program: /Transit/apfs-fuse-master/build/bin/apfs-fuse /dev/nvme0n1p2 /uu [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".

Program received signal SIGSEGV, Segmentation fault. 0x000055555556be3e in BTreeNode::level (this=0x0) at /Transit/apfs-fuse-master/ApfsLib/BTree.h:79 79 uint16_t level() const { return m_bt->level; }

(gdb) backtrace

0 0x000055555556be3e in BTreeNode::level (this=0x0) at /Transit/apfs-fuse-master/ApfsLib/BTree.h:79

1 0x000055555556a523 in BTree::Lookup (this=0x55555579d538, result=..., key=0x7fffffffdf30, key_size=16, func=0x555555560b0e <CompareNodeMapKey(void const, size_t, void const, size_t, void*)>,

context=0x55555579c538, exact=false) at /Transit/apfs-fuse-master/ApfsLib/BTree.cpp:211

2 0x0000555555560e63 in ApfsNodeMapperBTree::GetBlockID (this=0x55555579c538, info=..., nodeid=1027, version=54073) at /Transit/apfs-fuse-master/ApfsLib/ApfsNodeMapperBTree.cpp:89

3 0x000055555555ce8f in ApfsContainer::GetVolume (this=0x55555579be70, index=0) at /Transit/apfs-fuse-master/ApfsLib/ApfsContainer.cpp:98

4 0x000055555555ad10 in main (argc=3, argv=0x7fffffffe148) at /Transit/apfs-fuse-master/apfsfuse/ApfsFuse.cpp:594

Hope the above information is sufficient. Any further questions, please let me know.

Regards,

JC

PS

I ran the the apfs-dump-quick diagnostic (apfs-dump-quick /dev/nvme0n1p2 nvme0n1p2.txt), this itself went down with a segmentation fault. The file is too big to upload here, nvme0n1p2.txt here is an excerpt, hope we are lucky on what I have redacted.

[sgan81]

I uploaded a new commit. Can you try again? The error you got must have to do something with a corrupt volume structure (or me not having understood it properly ...). The apfs-dump-quick should work now. It will log an error, and I need the nodes around that error ... ;)

You may also compress big txt files with gz, bzip2 or xz, all fine with me ...

[sonvirgo]

no nothing?

$ ./apfs-fuse -d 3  /dev/sda4 /media/sun/osx
Device not found!

$ sudo fdisk -l
Disk /dev/sda: 223.6 GiB, 240057409536 bytes, 468862128 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 67614AA0-5681-03A3-A852-D859A0D5E800

Device         Start       End   Sectors   Size Type
/dev/sda1      34816    546815    512000   250M EFI System
/dev/sda2     546816 287157717 286610902 136.7G Microsoft basic data
/dev/sda3  287158272 288184319   1026048   501M Windows recovery environment
/dev/sda4  329596928 468862087 139265160  66.4G unknown

[sonvirgo]

sudo - ok How to make it work with Disk or GParted?

[sgan81]

I have to admit that the error message "Device not found!" might be a bit misleading. I have uploaded a new version which is a bit more verbose if things go wrong. Maybe you could pull, compile, and try again?

[Johnnie390]

Here we are (again) -

/usr/bin/apfs-fuse /dev/nvme0n1p2 /uu apfs-fuse[8237]: segfault at 48 ip 000055f636581430 sp 00007fff5ec648a0 error 4 in apfs-fuse[55f636568000+2e000]

Wuth debug on (-d 1):

root@Pezenas:/Transit/apfs-fuse-master# /usr/bin/apfs-fuse /dev/nvme0n1p2 /uu -d 1 Device /dev/nvme0n1p2 opened. Size is 200738316288 bytes. Segmentation fault

root@Pezenas:/Transit/apfs-fuse-master/build# blkid /dev/nvme0n1: PTUUID="9d0fdb56-526d-46d5-8063-bc1a754c351a" PTTYPE="gpt" /dev/nvme0n1p1: LABEL="EFI" UUID="5F66-17ED" TYPE="vfat" PARTLABEL="EFI System Partition" PARTUUID="f2711c8d-b354-4365-96b2-82a682a818b8" /dev/nvme0n1p2: PARTLABEL="NoName" PARTUUID="7fab3f1a-4a67-40d6-94a5-f73e07f76dcf" /dev/nvme0n1p3: UUID="ebe6e905-8677-40f9-ac5d-2bf2e30763cf" TYPE="ext4" PARTUUID="d9ae78e1-f01a-4505-a1de-8c8cdd59a1a2"

root@Pezenas:/Transit/apfs-fuse-master/build# fdisk -l Disk /dev/nvme0n1: 465.9 GiB, 500277788672 bytes, 122138132 sectors Units: sectors of 1 * 4096 = 4096 bytes Sector size (logical/physical): 4096 bytes / 4096 bytes I/O size (minimum/optimal): 4096 bytes / 4096 bytes Disklabel type: gpt Disk identifier: 9D0FDB56-526D-46D5-8063-BC1A754C351A

Device Start End Sectors Size Type /dev/nvme0n1p1 6 76805 76800 300M EFI System /dev/nvme0n1p2 76806 49085183 49008378 187G unknown /dev/nvme0n1p3 49085184 122138111 73052928 278.7G Linux filesystem

Linux Pezenas 4.16.0-rc4 #1 SMP Fri Mar 9 16:36:20 CET 2018 x86_64 GNU/Linux

The same as above, with GDB enabled:

Reading symbols from apfs-fuse...(no debugging symbols found)...done. (gdb) set args -d 1 /dev/nvme0n1p2 /uu (gdb) run Starting program: /usr/bin/apfs-fuse -d 1 /dev/nvme0n1p2 /uu [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". Device /dev/nvme0n1p2 opened. Size is 200738316288 bytes.

Program received signal SIGSEGV, Segmentation fault. 0x000055555556d430 in BTree::Lookup(BTreeEntry&, void const, unsigned long, int ()(void const, unsigned long, void const, unsigned long, void), void, bool) () (gdb) backtrace

0 0x000055555556d430 in BTree::Lookup(BTreeEntry&, void const, unsigned long, int ()(void const, unsigned long, void const, unsigned long, void), void, bool) ()

1 0x000055555555d53a in ApfsNodeMapperBTree::GetBlockID(node_info_t&, unsigned long, unsigned long) ()

2 0x000055555555b8e1 in ApfsContainer::GetVolume(int) ()

3 0x0000555555558ca8 in main ()

(gdb) quit

[Johnnie390]

Above is destined for Simon.

[Johnnie390]

drivedump.txt.gz

Above is output of apfs-dump-quick /dev/nvme0n1p2 drivedump.txt which itself abended with a segfault.

Any further information required, please let me know.

[sgan81]

Thanks. Line numbers would have helped, but I have a suspicion nonetheless ...

How much RAM do you have? Judging from your drivedump, you have quite a lot of files on the drive. You could try and open BTree.h, change #define BTREE_USE_MAP to #undef BTREE_USE_MAP. This will make the driver somewhat slower, but it shouldn't crash anymore if the memory runs out ... apparently I didn't think too hard about this optimization ...

The problem with BTREE_USE_MAP is that it caches all management blocks it ever read into memory. This takes up about 160MB on a drive with just OS X 10.13 on it. If there are more files, it takes more ... maybe you just ran out of memory.

I will commit an update with a better fix for this, but that will take some time. In the meantime, you can use the workaround I described above. If it still crashes, I would appreciate you rebuilding the driver in debug mode before running gdb. Thanks. And thanks for helping :-)

[sgan81]

Update: I uploaded a fix for this ... that means, if my suspicions are correct. If it still crashes, there is another problem ...

[Johnnie390]

Will give the fix a try at the weekend and report back.

[Johnnie390]

"How much RAM do you have?" - 16GB - Max configuration for this Macbook.

[Johnnie390]

In the meantime, running a 4.16 kernel with gcc version 7.3.0 (Debian 7.3.0-12).

Apfs-fuse, with no diagnostics/trace -

root@Pezenas:/Transit/apfs-fuse-master/build/bin# ./apfs-fuse /dev/nvme0n1p2 /uu -d 1 Device /dev/nvme0n1p2 opened. Size is 200738316288 bytes. ERROR: wrong header type 0x40000003 Segmentation fault

With GDB enabled -

root@Pezenas:/Transit/apfs-fuse-master/build/bin# gdb apfs-fuse GNU gdb (Debian 7.12-6+b1) 7.12.0.20161007-git Copyright (C) 2016 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-linux-gnu". Type "show configuration" for configuration details. For bug reporting instructions, please see: http://www.gnu.org/software/gdb/bugs/. Find the GDB manual and other documentation resources online at: http://www.gnu.org/software/gdb/documentation/. For help, type "help". Type "apropos word" to search for commands related to "word"... Reading symbols from apfs-fuse...(no debugging symbols found)...done. (gdb) set args /dev/nvme0n1p2 /uu (gdb) run Starting program: /Transit/apfs-fuse-master/build/bin/apfs-fuse /dev/nvme0n1p2 /uu [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". ERROR: wrong header type 0x40000003

Program received signal SIGSEGV, Segmentation fault. 0x0000555555572f0c in BTree::Lookup(BTreeEntry&, void const, unsigned long, int ()(void const, unsigned long, void const, unsigned long, void), void, bool) () (gdb) backtrace

0 0x0000555555572f0c in BTree::Lookup(BTreeEntry&, void const, unsigned long, int ()(void const, unsigned long, void const, unsigned long, void), void, bool) ()

1 0x0000555555563a5a in ApfsNodeMapperBTree::GetBlockID(node_info_t&, unsigned long, unsigned long) ()

2 0x0000555555561296 in ApfsContainer::GetVolume(int, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&) ()

3 0x000055555555e2f9 in main ()

(gdb) quit A debugging session is active.

Inferior 1 [process 5373] will be killed.

Quit anyway? (y or n) y

[Johnnie390]

apfs-dump-quick produces a dump file that is 147MB in size which cannot be uploaded here :(

Any more information you require, please let me know.

[sgan81]

Did you use the most recent version of apfs-fuse? Was the volume converted from HFS+ at some point?

Also, debugging helps quite a bit more if you edit CMakeLists.txt beforehand and rebuild the project, as described in Readme.md:

Change set(CMAKE_BUILD_TYPE Release) to set(CMAKE_BUILD_TYPE Debug) in CMakeLists.txt in the root folder. make clean make

Dump files are huge because they contain all the information about the filesystem. But gzip or xz should be able to compress them quite well. Like: xz dump-output.txt or gzip -9 dump-output.txt

[Johnnie390]

Simon,

you are of course, correct. I had forgotten the Debug parameter in CMakeLists.txt.

Here we are -

Type "apropos word" to search for commands related to "word"... Reading symbols from apfs-fuse...done. (gdb) set args -d1 /dev/nvme0n1p2 /uu (gdb) run Starting program: /Transit/apfs-fuse-master/build/bin/apfs-fuse -d1 /dev/nvme0n1p2 /uu [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". Device /dev/nvme0n1p2 opened. Size is 200738316288 bytes. ERROR: wrong header type 0x40000003

Program received signal SIGSEGV, Segmentation fault. 0x0000555555584dc6 in BTreeNode::level (this=0x0) at /Transit/apfs-fuse-master/ApfsLib/BTree.h:80 80 uint16_t level() const { return m_bt->level; } (gdb) backtrace

0 0x0000555555584dc6 in BTreeNode::level (this=0x0) at /Transit/apfs-fuse-master/ApfsLib/BTree.h:80

1 0x000055555558322f in BTree::Lookup (this=0x5555557b99a8, result=..., key=0x7fffffffde60, key_size=16, func=0x55555557964c <CompareNodeMapKey(void const, size_t, void const, size_t, void*)>,

context=0x5555557b89a8, exact=false) at /Transit/apfs-fuse-master/ApfsLib/BTree.cpp:215

2 0x0000555555579a17 in ApfsNodeMapperBTree::GetBlockID (this=0x5555557b89a8, info=..., nodeid=1027, version=54073) at /Transit/apfs-fuse-master/ApfsLib/ApfsNodeMapperBTree.cpp:96

3 0x00005555555753c9 in ApfsContainer::GetVolume (this=0x5555557b8280, index=0, passphrase="") at /Transit/apfs-fuse-master/ApfsLib/ApfsContainer.cpp:106

4 0x00005555555730cd in main (argc=4, argv=0x7fffffffe0d8) at /Transit/apfs-fuse-master/apfsfuse/ApfsFuse.cpp:635

(gdb) quit A debugging session is active.

Inferior 1 [process 8727] will be killed.

Quit anyway? (y or n) y

[Johnnie390]

nvme0n1p2.log.gz

root@Pezenas:/Transit/apfs-fuse-master/build/bin# ./apfs-dump-quick /dev/nvme0n1p2 nvme0n1p2.log ERROR: wrong header type 0x40000003 Segmentation fault

Dump file size reduced to 94MB with gzip.

[Johnnie390]

Correction dump file is ca. 9MB when gzipped. Latest version of apfs-fuse used (downloaded today).

I have not attempted to perform ANY file conversions on the HFS+ file system.

[sgan81]

Thanks for the log. Although I'm now more confused than before.

What I meant by HFS+ conversion is, has the drive been converted from HFS+? If you had macOS 10.12 or earlier installed before and your drive is an SSD, this conversion is done automatically when installing macOS 10.13. The logfile looks to me like it has been converted from HFS+.

I also noted that the version numbers are somewhat strange. Did you unmount the volume properly before trying apfs-fuse? If yes, I have to do some investigations, which might take some time ...

[Johnnie390]

Morning Simon,

"If you had macOS 10.12 or earlier installed before and your drive is an SSD, this conversion is done automatically when installing macOS 10.13." - Yes, this has happened, without giving me the chance to make a choice. This is one of the many reasons I abhore apple.

"Did you unmount the volume properly before trying apfs-fuse?" - Yes, I did.

Bonne chance.

[sgan81]

I just uploaded a new version. It shouldn't crash anymore, but it still might fail to mount the volume. Can you try it and tell me if it works now? Thanks a lot for your help.

[sgan81]

Judging from the fact that I didn't hear any complaints anymore, I'm assuming the segfault has been fixed. Therefore I'm going to close this issue.

[Johnnie390]

Simon, have been away, thus no time to test. I see the new version of apfs-fuse uses lzfse, are there build instructions for cmake?

Regards,

JC

[sgan81]

Just clone the repository as described in the readme, the rest is done automagically ...

And no, lzfse is not supported yet, I just use the library for lzvn decompression.

[fredcooke]

Any updates on this @Johnnie390 ? Thanks to both of you! Reverse engineering this can't be too much fun :-)

[sgan81]

Well, for me, reverse engineering is actually kind of fun ... yes, it can get quite hard at times, but it's always satisfying when things work out in the end. And you learn a lot in the process ;-)

[fredcooke]

Agreed, can be fun, but in this case you're dealing with Apple, and that means a moving target :-)