port to Mac OS X

[derekbruening]

From derek.br...@gmail.com on February 26, 2009 12:58:15

container issue for general task of porting to Mac OS X

we should expand our feature-based *NIX support: HAVE_PROC_MAPS, etc., to make further ports (e.g., to BSD) and support of wider range of Linux distros easier

for /proc/maps, should investigate vm_region()

Original issue: http://code.google.com/p/dynamorio/issues/detail?id=58

[derekbruening]

From bruen...@google.com on July 09, 2012 04:57:23

Note that I now have access to a Mac and I started toying with this

Status: Started
Owner: bruen...@google.com

[derekbruening]

From bruen...@google.com on April 14, 2013 19:54:28

I had a tree from a while back that is now committed, along with some recent work in r2056 , r2059 , r2060 , r2061 , r2062 , r2063 . Still quite early though: haven't even gotten to the bulk of the os-specific parts of the code, but I have a plan for how to solve many of the issues. Probably I will use libSystem's mach wrappers initially as they are pretty bare and we can replace w/ raw syscalls in the future.

[derekbruening]

From bruen...@google.com on February 24, 2014 12:14:04

I have put a lot of work into this and can now run small single-threaded apps that do not use signals on both Lion and Mavericks. If I get a chance I will put a summary of the noteworthy issues here. A number of the more recent issues have been filed separately, but a lot of the core things are still under this issue.

[derekbruening]

64-bit support has a major roadblock in #1568 as there is no simple TLS solution

[sidkshatriya]

In #2078 @zhaoqin mentioned that DR supports macOS partially. I'm interested in the macOS 64bit port -- Is there a wiki entry somewhere on how to build for macOS and what can be accomplished right now e.g. no signals, single threaded apps work properly out of the box?

[zhaoqin]

Not much progress on macOS 64bit. We are short on manpower for Mac work, unfortunately. We would welcome contributions.

[derekbruening]

A subset of the 32-bit test suite (51 tests) works on MacOSX today. You can see it on our Travis runs: https://travis-ci.org/DynamoRIO/dynamorio/jobs/179528229. Basic signals and threads do work today but we will not be surprised if larger apps (e.g., big commercial packages) do not work. We also have Dr. Memory running, again on small programs.

The biggest problem for tools and the limiting factor for Dr. Memory is the lack of a private loader (#1285) which means there is not proper resource isolation today: this is the reason we do not advertise the Mac port of DR very much and do not even provide a binary package, while we do provide a DrM package.

For 64-bit the blocking issue is as mentioned above: #1568.

Unfortunately there is no public write-up in either the DR or DrM wikis but I have substantial notes and if publishing them will facilitate contributors I can try to make them accessible.

[sidkshatriya]

Unfortunately there is no public write-up in either the DR or DrM wikis but I have substantial notes and if publishing them will facilitate contributors I can try to make them accessible.

I think it would be awesome if notes on the DR codebase were shared in whatever imperfect form in the Wiki -- both notes relevant to macOS and the project in general. They could be titled "Getting started on the DynamoRIO codebase" (or similar). Some open source projects tend to have these resources as a way of on-boarding would be contributors and they are definitely useful.

DynamoRIO has an abundance of research publications and a nicely written thesis about it -- this is already a great resource -- now just some code guidance would be great!

[sidkshatriya]

@derekbruening Also I saw the slides of https://github.com/DynamoRIO/dynamorio/releases/download/release_6_1_0/DynamoRIO-tutorial-mar2016.pdf

It looks like a comprehensive and awesome presentation. Sadly I was not able to understand many slides as there was no audio/video. Might audio/video be available for this? If not, are there any presentations you're going to give in the near future on this slide set that could be recorded?

[zhaoqin]

There will be a CGO tutorial in early Feb 2017.

[sidkshatriya]

Please request a video recording of the CGO tutorial!

[derekbruening]

To get links to the commits:

2013-04-14 e61c6e23 i#58 MacOS: first steps toward building on MacOS 2013-04-14 e25aa1e3 i#58 MacOS: rename LINUX define to UNIX. 2013-04-15 121b3748 i#58 MacOS: further steps toward building on MacOS 2013-04-15 48f8d6ac i#58 MacOS: assembly support 2013-09-28 10546ebf i#58 MacOS: rename exception_type_t to dr_exception_typet to avoid conflict with Mac type. 2013-10-01 a520f1f5 i#58 MacOS: refactor Linux clone and vfork code 2013-10-01 18770699 i#58 MacOS: rename thread{yield,sleep,suspend,resume,terminate} to osthread* to avoid name conflicts with Mac Mach routines. 2013-10-01 1b7c6a79 i#58 MacOS: misc build fixes: 2013-10-01 455007cc i#58 MacOS: stdout, etc. are FILE with _file, not _IO_FILE with _fileno 2013-10-08 17b1855b i#58 MacOS: use sysctlbyname initially for get_numprocessors 2013-10-08 039886e6 i#58 MacOS, i#1270 part 5: mac memquery 2013-10-08 49fa0212 i#58 MacOS, i#1277 part 3: mac ksynch implementation using Mach semaphores 2013-10-08 dac617c6 i#58 MacOS: tls for mac, part 1: mostly just stubs 2013-10-08 944e3980 i#58 MacOS: Mac signal handling code, part 1: 2013-10-08 53676225 i#58 MacOS: fix several minor build issues: 2013-10-08 ee7d9903 i#58 MacOS: fix several minor linker issues: 2013-10-08 d4f8fe18 i#58 MacOS: nasm requires on everything 2013-10-08 896ac63b i#58 MacOS: rename {is,was}_clone_thread_syscall to {is,was}_thread_create_syscall 2013-10-08 530c996c i#58 MacOS: add module_macho.c with NYI stubs 2013-10-08 eff51a89 i#58 MacOS: DR library bounds. 2013-10-08 04893764 i#58 MacOS: injector for MacOS, part 1. 2013-11-25 02fdd659 i#58 MacOS: rpath on Mac 2013-11-25 922915f6 i#58 MacOS: misc build fixes 2013-11-25 9b03d33f i#58 MacOS: initial Mach-O support 2013-12-05 33bb5db5 i#58 MacOS: mmap and lseek invocations by DR 2013-12-05 7bb27f2e i#58 MacOS: syscalls monitored by DR, part 1 2013-12-05 112fe5ba i#58 MacOS: syscalls monitored by DR, part 2 2013-12-05 6726ebc8 i#58 MacOS: get_application_name() 2013-12-05 c766779e i#58 MacOS: get_uname() and get_num_processors() 2013-12-05 1f8da27b i#58 MacOS: injector for MacOS, part 2: 2014-01-09 22c316d3 i#58 MacOS: query_time_seconds() and SYS_gettimeofday quirk 2014-01-09 4c27927c i#58 MacOS: os_thread_yield() 2014-01-09 78b82432 i#58 MacOS: thread_signal() 2014-01-09 a81a3312 i#58 MacOS: os_thread_sleep() 2014-01-09 a14805c2 i#58 MacOS: no SYS_getdents on MacOS: not needed for now so disabled 2014-01-09 c0353344 i#58 MacOS: os_get_current_dir() 2014-01-09 cfef3880 i#58 MacOS: handle system call invocation features unique to MacOS 2014-01-09 ce12b142 i#58 MacOS: avoid zero-sized kernel_sigset_t on Mac 2014-01-09 22d0837f i#58 MacOS: add diagnostic for RLIMIT_NOFILE failure 2014-01-09 194986f0 i#58 MacOS: remove the last of the Linux syscall numbers 2014-01-09 130459fb i#58 MacOS: client interface 2014-01-09 22cb1371 i#58 MacOS: support old nasm with -fmacho instead of -fmacho32 2014-01-09 fa320969 i#58 MacOS: fix memory iterator 2014-01-09 1fdd7604 i#58 MacOS: fix Mac build error from r2429 2014-01-09 5af2c1a1 i#58 MacOS: determine the file backing a memory region 2014-01-09 2f0732c8 i#58 MacOS: library bounds 2014-01-09 4f109b98 i#58 MacOS: add support for 7th arg to dynamorio_syscall 2014-01-09 a529a058 i#58 MacOS: use SYS_thread_selfid for thread id 2014-01-09 4c385318 i#58 MacOS: Mach-O segment walking 2014-01-09 0d6abe2b i#58 MacOS: more Mach-O support 2014-01-09 81d18e2a i#58 MacOS: Mach-O module data 2014-01-09 43387cb0 i#58 MacOS: tls implementation: tls_get_fs_gs_segment_base() 2014-01-09 16cd0048 i#58 MacOS: 32-bit TLS implementation 2014-01-09 ed456adf i#58 MacOS: do not set SS_ONSTACK for the new sigaltstack as it's an invalid argument there and is only used for the old output stack. 2014-01-09 9e587f6a i#58 MacOS: fix bug in semaphore_create call 2014-01-09 c83e3d77 i#58 MacOS: implement module_entry_point() 2014-01-09 77c0e598 i#58 MacOS: skip the __PAGEZERO segment completely during segment walks, as it's hard to identify as part of a module during a memory query walk. 2014-01-09 8ba53b91 i#58 MacOS: os_list_threads() NYI for now: fail gracefully and continue 2014-01-09 27ba5785 i#58 MacOS: vsyscall hook is Linux-only 2014-01-13 1215489a i#58 MacOS: use the _nocancel versions of all system calls that DR makes in order to defer app-initiated thread termination. 2014-01-13 78d8c67c i#58 MacOS: monitor the _nocancel versions of app syscalls 2014-01-13 e93a0523 i#58 MacOS: app system call args and return value 2014-01-21 6875bcc6 i#58 MacOS: thread id type is 64-bit 2014-01-21 517e92af i#58 MacOS: fix misc bugs now that we can run an app to completion 2014-01-21 99d50510 i#58 MacOS: properly handle rebased library in module_walk_program_headers() 2014-01-21 e2c2aaf0 i#58 MacOS: memquery iterator, even with primed start, still requires iteration with sub-regions 2014-01-28 3fc59732 i#58 MacOS: 32-bit syscall return values 2014-01-28 1a9abc32 i#58 MacOS: client syscall results 2014-01-28 7d795a01 i#58 MacOS: retain control on app sysenter 2014-01-28 3b7505e0 i#58 MacOS: handle syscall number encodings 2014-01-28 d76e8ca7 i#58 MacOS: set syscall args properly for final exit syscall 2014-02-06 34bab0d3 i#58 MacOS: stack alignment 2014-02-06 d8aa520d i#58 MacOS: support clang 2014-02-11 c4741e14 i#58 MacOS: run dsymutil to generate symbols on all targets 2014-02-19 92a70aac i#58 MacOS: dyld shared cache 2014-02-19 8cf6c969 i#58 MacOS: for the Mach-O module name "module_name", ensure it's a basename and not a full path. 2014-02-19 459cc12e i#58 MacOS: fix Mac mem query bug 2014-02-19 220a8b29 i#58 MacOS: intra-DR signal handling 2014-02-19 29d49ee4 i#58 MacOS: fix stack aligment bugs 2014-02-19 fc87a532 i#58 MacOS: normalize system call number passed to pre-syscall event as well as post-syscall event. 2014-02-19 eb6dedd3 i#58 MacOS: AVX state for signals 2014-02-19 bda49648 i#58 MacOS: drwrap stack alignment 2014-02-19 9e666ef4 i#58 MacOS: dyld shared cache on 10.9 2014-02-19 fae559b9 i#58 MacOS: fixes for Mavericks (10.9) 2014-02-24 522bf5ce i#58 MacOS: fix exit-time crash due to stack align shifting dcontext param to cleanup_and_terminate 2014-02-24 ed52157d i#58 MacOS: handle LC_MAIN, introduced in 10.9 to replace LC_UNIXTHREAD to hold executable entry point 2014-02-24 6e77bcf0 i#58 MacOS: add version check 2014-02-24 d58a8d96 i#58 MacOS: improve asm setup on Mac by using CMake 2.8.3's NASM support 2014-02-24 9963afbe i#58 MacOS: fix detection of cores on Mac OSX 10.9 where the output from "system_profiler" is too large for CMake 2014-02-24 2db1527b i#58 MacOS: binary compatibility 2014-02-24 2761a240 i#58 MacOS: fix identification of the app name from the stack: turns out it's below the args, which are below environ. 2014-02-24 e9db67ea i#58 MacOS: for libs in the dyld shared cache, we need to add the base, and not the load_delta, to exports found in the trie, as the segments have all been updated but not the trie offsets. 2014-03-17 765f6d2b i#58 MacOS signals: add delivery of signals to the app 2014-03-25 6e37284e i#58 MacOS: retrieve sigcontext from the pointer and not the inlined field in the frame, as kernel padding makes the latter unreliable. This fixes crashes when clients suppress signals. 2014-03-25 25044d08 i#58 MacOS: pass 2 slots to SYS_lseek for 64-bit arg in 32-bit mode 2014-03-30 16d33e4d i#58 MacOS: initial thread handling 2014-03-30 677730ed i#58 MacOS: handle post-sysenter continuation on a skipped syscall 2014-03-30 4e214205 i#58 MacOS: signaling a specific thread 2014-03-30 b9d438ae i#58 MacOS: thread synch and termination 2014-03-30 1ea24099 i#58 MacOS: handle thread exit 2014-03-30 0d1d4ca0 i#58 MacOS: Mach syscall success 2014-03-30 49606de1 i#58 MacOS: install debug info files on Mac 2014-11-21 52df3151 i#58 MacOS: properly mark BSD syscall in dynamorio_sys_exit for 64-bit. 2015-10-15 3dfd4ad7 i#58 MacOS: start porting test suite to build on Mac 2015-10-15 be4882b8 i#58 MacOS: add os_public.h to share context defines 2015-10-16 fadac17a i#58 MacOS: port test suite, part 2 2015-10-16 5248c8d4 i#58 MacOS: port test suite, part 3 2015-10-16 26ca68f4 i#58 MacOS: enable initial test suite 2015-10-21 0f90cf98 i#58 MacOS: only build 32-bit for release packages 2015-10-21 4c261c3b i#58 MacOS: set name for release package 2017-03-05 a7ff3fa3 i#58: properly parse the executable path on recent OSX kernels (#2266)