Using Mach-O's for Darling itself

[LubosD]

The following idea is currently tested in the using-machos-experiment branch.

• Build all libraries & frameworks in Darling as Mach-O objects (.dylibs). This actually means reverting many changes made in assembly code.
• Build all (prefix) executables in Darling as Mach-O executables.
• Try using original dyld for loading binaries. This would sadly mean dropping our own libdyld.
  • Provide a very simple (ELF) executable capable of loading dyld itself (dyld would already be a Mach-O) and handing control over to it.
  • This thin loader would provide pointers to ELF loader's APIs (dlopen, dlclose, dlsym, ...) somewhere.
  • We would also need to be able to build libc & libsystem_kernel specifically as a static library for dyld. There is a special define to be used for the libc build.
• Once we need to call "foreign APIs" (e.g. ALSA, OpenGL, ...), provide a generator that creates a Mach-O wrapper for any given ELF library, creating simple stubs that automatically invoke ELF dlsym on first use and forward calls to the real implementation. Basically something similar to dyld_stub_binder, except this would be dyld_elf_stub_binder.
• We will also need to drop libobjc2 and use Apple's runtime instead. There have already been some tests of this in the apple-objc4-experiment branch.

[LubosD]

Now I've built most of the stuff, except libobjc (will use Apple's objc4) and frameworks in general.

Most things work, but /bin/bash and /bin/stat crash due to an incorrect free() call. The biggest problem is debugging, because gdb is unable to load debug info for any of the Mach-Os, so it's a nightmare.

Therefore, the next step is to get gdb working.

• Multitarget builds (USE=multitarget on Gentoo) of gdb include support for Darwin and Mach-O. They look for _dyld_all_image_infos to communicate with dyld.
  • Debian's gdb-multiarch doesn't include this support, but we can of course build gdb manually (and provide build instructions).
  • I haven't yet successfully used it, so maybe vanilla code will not work. In that case we'll just fork gdb.

UPDATE: I previously though gdb support was removed from recent dyld releases, but it hasn't. So now I just need to bind it into mldr (our new simple Mach-O loader that just hands over control to dyld).

[LubosD]

Current status: Mach-O Bash finally starts, but fork() doesn't work well, so it cannot spawn any subprocesses :-D They SIGABRT immediately. It seems a mig_reply_port() is not getting reset across fork(), so the forked process is using an invalid Mach port right. I wonder what's different, because it works in master (and I think I remember solving this in the past).

GDB status: it works, but debug info is sometimes not loaded for some libraries and no info at all is loaded when attaching to a running process. This is likely because I haven't yet hooked onto all of the callbacks used by GDB. But it shouldn't be too hard to get this sorted, the most difficult part is already done.

[yan12125]

From descriptions above I assume this is far from a stable state, so I just write some observations here instead of opening new issues. First the afed3b3b634232e46c6360469e7d164a6e419c7d checkout leads to some compilation failures. Here's partial log:

In file included from /home/yen/Projects/tmp/darling/src/external/corefoundation/NSArray.m:17:                                          
In file included from /home/yen/Projects/tmp/darling/src/external/foundation/src/NSFastEnumerationEnumerator.h:2:
In file included from /home/yen/Projects/tmp/darling/src/external/corefoundation/ForFoundationOnly.h:587:
In file included from /home/yen/Projects/tmp/darling/src/external/foundation/include/Foundation/Foundation.h:68:
In file included from /home/yen/Projects/tmp/darling/src/external/foundation/include/Foundation/NSURLCredential.h:2:
/home/yen/Projects/tmp/darling/src/external/security/include/Security/Security.h:54:10: fatal error: 'Security/SecAccessControl.h' file
      not found
#include <Security/SecAccessControl.h>
         ^
226 warnings and 1 error generated.                                                                                                     
make[2]: *** [src/external/corefoundation/CMakeFiles/CoreFoundation.dir/build.make:1983: src/external/corefoundation/CMakeFiles/CoreFoundation.dir/NSArray.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:8890: src/external/corefoundation/CMakeFiles/CoreFoundation.dir/all] Error 2
make: *** [Makefile:128: all] Error 2

I guess some include paths are not correctly set, so I patch src/external/corefoundation/CMakeLists.txt:

diff --git a/CMakeLists.txt b/CMakeLists.txt
index 1e2b080..9586909 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -21,6 +21,7 @@ include_directories(${CMAKE_CURRENT_SOURCE_DIR}
    ${CMAKE_SOURCE_DIR}/src/libmalloc/include/malloc
    ${CMAKE_SOURCE_DIR}/src/libffi/include
    ${CMAKE_SOURCE_DIR}/src/external/security/include
+   ${CMAKE_SOURCE_DIR}/src/external/security/OSX/sec
    ${CMAKE_SOURCE_DIR}/src/libc/stdtime/FreeBSD
    ${CMAKE_SOURCE_DIR}/src/external/syslog/libsystem_asl.tproj/include
    ${CMAKE_SOURCE_DIR}/src/external/objc4/runtime

There's a similar problem for src/external/foundation/CMakeLists.txt and the patch is also similar.

After building the whol project, running darling shell leads to another error:

$ darling shell
/bin/mldr: error while loading shared libraries: libpthread.so.0: cannot open shared object file: No such file or directory

Here's the strace log for the same command: https://gist.github.com/yan12125/766c2ef45e1f10c4c7e1855bebfcb784. Seems /bin/mldr also gets into darling's root directory so it can't find Linux libraries like libpthread.so?

I hope I can help in this experiment but little knowledge to advanced Linux kernel APIs and Darwin internals is a huge obstacle. Simple works like building and testing are more feasible for me. After all, thanks for giving me a new look of the world with such a innovative project!

Environment: Arch Linux x86_64, clang 3.9.1, Linux kernel 4.10.1

[bugaevc]

Hi,

the description above is very outdated. The Mach-O branch works quite well and is getting close to being merged.

Unfortunately, Arch does not quite conform to Filesystem Hierarchy Standard, causing us problems. See #236 and #237. I've made a hack in 73eb79b1f872954bcddda1270d7ad4d7dccbef1d that makes it work on Arch with the old design, but the it won't work with Mach-O. This should probably be worked out by the time the branch is merged (thanks for reminding me!); for now, you can use some other distro.

[yan12125]

The Mach-O branch works quite well and is getting close to being merged.

Great to hear this and congratulations! Actually I got another error before looking into this branch. There are compilation errors regarding mixing host (Darwin) and build (Linux) headers. I believe using Mach-O is the way to go so I gave it a try.

See #236 and #237

Thanks for that info. The workaround turns to be quite similar to the previous one:

diff --git a/src/dyld/darling.c b/src/dyld/darling.c
index b989d0f1..0db1d204 100644
--- a/src/dyld/darling.c
+++ b/src/dyld/darling.c
@@ -354,6 +354,8 @@ void setupChild(const char *curPath)
        setenv("PWD", buffer2, 1);
        chdir(buffer2);
    }
+
+   setenv("LD_LIBRARY_PATH", "/lib", 1);
 }

 void showHelp(const char* argv0)

With that I can run darling's bash.

Then comes another story. At first all external commands (those who require fork()) crash with SIGSEGV. After I attach the bash (mldr) process under strace:

sudo strace -fp $(pgrep mldr) >& /dev/null

All commands run fine. That sounds magic to me.

[bugaevc]

Regarding build errors, just to be sure, you should do git submodule init && git submodule update update every time you check out a commit. Otherwise, please do report all the build errors you're getting -- and feel free to open separate issue tickets :wink:

[yan12125]

You're right the build failure is already fixed in https://github.com/darlinghq/darling-security/commit/46ac6633b2239b08a2c33bbc9d7d426c8e39034e#diff-67254e321905596642d48c900ba8e5dc. The only issue left is:

Darling [~/Projects/tmp/darling/build/x86-64]$ ls
Segmentation fault: 11 (core dumped)

Let me check it more to see if it's an Arch-specific issue or a general Darling bug.