Closed LubosD closed 7 years ago
Now I've built most of the stuff, except libobjc (will use Apple's objc4) and frameworks in general.
Most things work, but /bin/bash
and /bin/stat
crash due to an incorrect free()
call. The biggest problem is debugging, because gdb
is unable to load debug info for any of the Mach-Os, so it's a nightmare.
Therefore, the next step is to get gdb working.
USE=multitarget
on Gentoo) of gdb include support for Darwin and Mach-O. They look for _dyld_all_image_infos
to communicate with dyld.
UPDATE: I previously though gdb support was removed from recent dyld releases, but it hasn't. So now I just need to bind it into mldr
(our new simple Mach-O loader that just hands over control to dyld).
Current status: Mach-O Bash finally starts, but fork() doesn't work well, so it cannot spawn any subprocesses :-D They SIGABRT immediately. It seems a mig_reply_port() is not getting reset across fork(), so the forked process is using an invalid Mach port right. I wonder what's different, because it works in master (and I think I remember solving this in the past).
GDB status: it works, but debug info is sometimes not loaded for some libraries and no info at all is loaded when attaching to a running process. This is likely because I haven't yet hooked onto all of the callbacks used by GDB. But it shouldn't be too hard to get this sorted, the most difficult part is already done.
From descriptions above I assume this is far from a stable state, so I just write some observations here instead of opening new issues. First the afed3b3b634232e46c6360469e7d164a6e419c7d checkout leads to some compilation failures. Here's partial log:
In file included from /home/yen/Projects/tmp/darling/src/external/corefoundation/NSArray.m:17:
In file included from /home/yen/Projects/tmp/darling/src/external/foundation/src/NSFastEnumerationEnumerator.h:2:
In file included from /home/yen/Projects/tmp/darling/src/external/corefoundation/ForFoundationOnly.h:587:
In file included from /home/yen/Projects/tmp/darling/src/external/foundation/include/Foundation/Foundation.h:68:
In file included from /home/yen/Projects/tmp/darling/src/external/foundation/include/Foundation/NSURLCredential.h:2:
/home/yen/Projects/tmp/darling/src/external/security/include/Security/Security.h:54:10: fatal error: 'Security/SecAccessControl.h' file
not found
#include <Security/SecAccessControl.h>
^
226 warnings and 1 error generated.
make[2]: *** [src/external/corefoundation/CMakeFiles/CoreFoundation.dir/build.make:1983: src/external/corefoundation/CMakeFiles/CoreFoundation.dir/NSArray.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:8890: src/external/corefoundation/CMakeFiles/CoreFoundation.dir/all] Error 2
make: *** [Makefile:128: all] Error 2
I guess some include paths are not correctly set, so I patch src/external/corefoundation/CMakeLists.txt
:
diff --git a/CMakeLists.txt b/CMakeLists.txt
index 1e2b080..9586909 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -21,6 +21,7 @@ include_directories(${CMAKE_CURRENT_SOURCE_DIR}
${CMAKE_SOURCE_DIR}/src/libmalloc/include/malloc
${CMAKE_SOURCE_DIR}/src/libffi/include
${CMAKE_SOURCE_DIR}/src/external/security/include
+ ${CMAKE_SOURCE_DIR}/src/external/security/OSX/sec
${CMAKE_SOURCE_DIR}/src/libc/stdtime/FreeBSD
${CMAKE_SOURCE_DIR}/src/external/syslog/libsystem_asl.tproj/include
${CMAKE_SOURCE_DIR}/src/external/objc4/runtime
There's a similar problem for src/external/foundation/CMakeLists.txt and the patch is also similar.
After building the whol project, running darling shell
leads to another error:
$ darling shell
/bin/mldr: error while loading shared libraries: libpthread.so.0: cannot open shared object file: No such file or directory
Here's the strace log for the same command: https://gist.github.com/yan12125/766c2ef45e1f10c4c7e1855bebfcb784. Seems /bin/mldr
also gets into darling's root directory so it can't find Linux libraries like libpthread.so
?
I hope I can help in this experiment but little knowledge to advanced Linux kernel APIs and Darwin internals is a huge obstacle. Simple works like building and testing are more feasible for me. After all, thanks for giving me a new look of the world with such a innovative project!
Environment: Arch Linux x86_64, clang 3.9.1, Linux kernel 4.10.1
Hi,
the description above is very outdated. The Mach-O branch works quite well and is getting close to being merged.
Unfortunately, Arch does not quite conform to Filesystem Hierarchy Standard, causing us problems. See #236 and #237. I've made a hack in 73eb79b1f872954bcddda1270d7ad4d7dccbef1d that makes it work on Arch with the old design, but the it won't work with Mach-O. This should probably be worked out by the time the branch is merged (thanks for reminding me!); for now, you can use some other distro.
The Mach-O branch works quite well and is getting close to being merged.
Great to hear this and congratulations! Actually I got another error before looking into this branch. There are compilation errors regarding mixing host (Darwin) and build (Linux) headers. I believe using Mach-O is the way to go so I gave it a try.
See #236 and #237
Thanks for that info. The workaround turns to be quite similar to the previous one:
diff --git a/src/dyld/darling.c b/src/dyld/darling.c
index b989d0f1..0db1d204 100644
--- a/src/dyld/darling.c
+++ b/src/dyld/darling.c
@@ -354,6 +354,8 @@ void setupChild(const char *curPath)
setenv("PWD", buffer2, 1);
chdir(buffer2);
}
+
+ setenv("LD_LIBRARY_PATH", "/lib", 1);
}
void showHelp(const char* argv0)
With that I can run darling's bash.
Then comes another story. At first all external commands (those who require fork()
) crash with SIGSEGV. After I attach the bash (mldr) process under strace:
sudo strace -fp $(pgrep mldr) >& /dev/null
All commands run fine. That sounds magic to me.
Regarding build errors, just to be sure, you should do git submodule init && git submodule update
update every time you check out a commit. Otherwise, please do report all the build errors you're getting -- and feel free to open separate issue tickets :wink:
You're right the build failure is already fixed in https://github.com/darlinghq/darling-security/commit/46ac6633b2239b08a2c33bbc9d7d426c8e39034e#diff-67254e321905596642d48c900ba8e5dc. The only issue left is:
Darling [~/Projects/tmp/darling/build/x86-64]$ ls
Segmentation fault: 11 (core dumped)
Let me check it more to see if it's an Arch-specific issue or a general Darling bug.
The following idea is currently tested in the using-machos-experiment branch.
dyld_stub_binder
, except this would bedyld_elf_stub_binder
.