Add iOS, TVOS, and WatchOS support to LDC

[smolt]

I thought we could discuss approaches here on how to add iPhoneOS support to LDC. Much of it is prototyped and glued together in https://github.com/smolt/ldc-iphone-dev. Not everything in the repo is needed to support iPhoneOS. Most up-to-date branch is ios. I imagine a bunch of smaller pulls to incrementally add it in.

I am going to start with a rough overview of the changes and an approach can follow. I may come back and edited this overview as I review things.

Inventory of changes

1. LDC
   1. add iPhoneOS target
      • new version(iOS), version(TvOS), and version(WatchOS) - [in upstream]
      • expanded some isMacOSX code to include iOS (isOSDarwin())
   2. add iOS ABIs (abi-ios-arm and abi-ios-arm64)
   3. cross compiling
      1. real type (both iOS arm/arm64 use 64-bit real) [see #1317]
      2. clang style -arch flag to select darwin target, include x86-based iOS simulator
   4. Generate embedded bitcode. This is probably needed so App Store will accept submissions.
2. phobos
   1. update unitttests for iOS
   2. some math changes for 64-bit real type
      • tweaks to ARM and AArch64 inline assembly [done]
      • potential fix for #888
   3. std.parallelism fix for iOS
   4. and a couple other changes that have already gone upstream [done]
3. druntime
   1. iOS support in core.stdc. Code common to OSX and iOS would use version(Darwin) [in upstream]
   2. Thread,Fiber support (ARM and AArch64 tweaks to existing support)
      • includes workaround for Issue #666 [Fiber done]
   3. SjLj exceptions (only 32-bit ARMs use) [done]
   4. add missing AArch64 support
   5. thread local changes to use support code below
   6. and a couple other changes that are already upstream
4. thread local support via patched LLVM and support code in https://github.com/smolt/iphoneos-apple-support [no longer needed as of LLVM 3.8.0 and iOS 9]
5. build support
   1. cmake cross compile, and in particular build universal libs
   2. process for making releases

      Version iOS

      • Currently in fork, version(IPhoneOS) version(iOS) is set when triple ends in "ios". version(OSX) is also set because iOS is a kind of OS X and often the OSX code is the correct code for iOS too.
      • Decision: use version(iOS) when triple OS is "ios" instead of IPhoneOS and don't set version(OSX). This aligns with Swift too. Also set version(Darwin) for both.

        Cross compile to target real type

Currently using something I cooked up last year that works when targeting ARM (-DUSE_OSX_TARGET_REAL), but it shouldn't be used. Other approaches are being worked such as #1317.

Support SjLj Exceptions for ARM 32-bit

personality function ldc-developers/druntime#39 Fiber support: [done]

Non-standard LLVM

Yeah! Stock LLVM can be used as of LLVM 3.8.0 and iOS 8 (arm64), iOS 9 (armv7). For earlier iOS version iPhoneOS targets need a small LLVM change to support thread locals. Even the i386 and x86_64 iPhone Simulator SDK doesn't support thread locals and ld complains if darwin TLS relocations show up in an object file. So LLVM AArch64, ARM, and X86 targets have a dab of code to generate a call to __tls_get_addr() which avoids using TLS relocations.

This may not work for app submissions because Apple seems to want to rebuild from LLVM bitcode. Best option here is to only support newest iOS.

Build/Test/Release support

Approach to make universal libs in cmake feels like a kludge, and not sure how to support testing. Still have not run dmd test-suite with cross-compiler.

[jacob-carlborg]

I'm not sure if I'm so happy how the versions are set. I think that iOS would be a better version identifier because that could then cover iPad and other iOS devices as well. Perhaps both iOS and iPhone should be set?

Previously we have only been able to identify the architecture and OS but now we will have use cases to identify the device as well. Should that be handled some other way than a version identifier?

I understand the reason to set OSX for iPhone but I think it's questionable. It will be more problematic to have different code for OS X and iOS. Example:

version (OSX) int a = 3; 
else version (IPhoneOS) int a = 4;

a will always bet set to 3.

While this will work as expected:

version (IPhoneOS) int a = 4;
else version (OSX) int a = 3; 

Ideally we would use darwin to identify iOS or OS X but Walter chose to go with OSX when DMD got support for OS X and deprecated darwin which already existed in GDC while everyone loudly complained :angry:.

[dnadlinger]

Isn't IPhoneOS slightly confusing because of the iPad, though? What about the watch? (Although that is bitcode-only, IIRC.)

Will get to the actual meat of the changes later.

[smolt]

I think version(iOS) would be fine as Apple considers it iOS synonymous with iPhone OS, and it is all the same OS for iPhone, iPad, iPod Touch, AppleTV, and iWatch. I work with two other "IOS" operating systems so I automatically want to differentiate, but they may never find their way into D-land (CISCO IOS and Island OS, a capabilities-based OS with a company I worked for).

Basically, same druntime/phobos lib binaries work on iPhone, iPad, iPod, AppleTV, no need to differentiate unless writing code at the UI or H/W capabilities level (e.g. does it have a camera).

For some more background, the version names and behavior was modeled on the SDK name and the definitions in include/TargetConditionals.h:

TARGET_OS_* 
These conditionals specify in which Operating System the generated code will
run. The MAC/WIN32/UNIX conditionals are mutually exclusive.  The EMBEDDED/IPHONE 
conditionals are variants of TARGET_OS_MAC. 

    TARGET_OS_MAC           - Generate code will run under Mac OS
    TARGET_OS_WIN32         - Generate code will run under 32-bit Windows
    TARGET_OS_UNIX          - Generate code will run under some non Mac OS X unix 
    TARGET_OS_EMBEDDED      - Generate code will run under an embedded OS variant
                              of TARGET_OS_MAC
    TARGET_OS_IPHONE        - Generate code will run under iPhone OS which 
                              is a variant of TARGET_OS_MAC.

where TARGET_OS_MAC SDK is named "MacOSX" and TARGET_OS_IPHONE SDK is named "iPhoneOS". For iPhoneOS SDK

    #define TARGET_OS_MAC               1
    #define TARGET_OS_WIN32             0
    #define TARGET_OS_UNIX              0
    #define TARGET_OS_EMBEDDED          1 
    #define TARGET_OS_IPHONE            1 
    #define TARGET_IPHONE_SIMULATOR     0 

I do think version(darwin) for the common version makes more sense to my brain. It would mean some churn in druntime and a little in phobos because most of the version(OSX) code would need to change to version(darwin).

[jacob-carlborg]

I work with two other "IOS" operating systems so I automatically want to differentiate

iOS != IOS :smiley:.

Basically, same druntime/phobos lib binaries work on iPhone, iPad, iPod, AppleTV, no need to differentiate unless writing code at the UI or H/W capabilities level (e.g. does it have a camera)

Yeah, that's mostly the reason I was thinking about. Perhaps most of these are better to handle at runtime?

For some more background, the version names and behavior was modeled on the SDK name and the definitions in include/TargetConditionals.h

The first version identifiers in D, like Win32 and linux were chosen from the corresponding if-defs in C. But that logic stopped with the introduction of the support for OS X. I think Walter said at one point that he chose the name based on what Apple is calling the OS (but at that point the OS was called Mac OS X). The if-defs for OS X is not very pretty __APPLE__.

TARGET_OS_UNIX - Generate code will run under some non Mac OS X unix

The Unix version identifier has traditional (D1, GDC) implied Posix systems, including OS X.

Conclusion: there is no logic behind the names chosen as version identifiers :smiley:.

I do think version(darwin) for the common version makes more sense to my brain. It would mean some churn in druntime and a little in phobos because most of the version(OSX) code would need to change to version(darwin)

I agree. But darwin is a deprecated version identifier in DMD. What should we do about that? Convince the DMD developers to undeprecated it?

[smolt]

Yeah, there are runtime calls to detect the device resolution, capabilities, things like that.

BTW, maybe you've seen this link before. I thought it was pretty handy:

http://nadeausoftware.com/articles/2012/01/c_c_tip_how_use_compiler_predefined_macros_detect_operating_system#OSXiOSandDarwin

Even though darwin makes sense to me, the one problem I see with using OS "darwin" as the common ground for "OSX" and "iOS" is this survey grep -i -e OSX -r ldc: "OSX" shows up in many C code and directory names and nearly every case is correct as is for iOS too. Would it be acceptable to change them to "darwin"?

[dnadlinger]

I'd rather change it to Darwin to fix the capitalization, since I don't think DMD defines darwin at all.

[jacob-carlborg]

I'd rather change it to Darwin to fix the capitalization, since I don't think DMD defines darwin at all

It does define darwin. According to the documentation [1] it's deprecated, but I don't get any deprecation warnings.

[1] http://dlang.org/version.html#predefined-versions at the end of this list there's another one with deprecate identifiers.

[dnadlinger]

Oh, you are right. It does not treat it as a reserved version identifier for whatever reasons, though, which is the table I must have looked at in the source last time.

One problem with [dD]arwin might be that Darwin does not actually include many of the user-facing APIs that are nevertheless shared between OS X and iOS.

[redstar]

Both deprecated identifiers darwin and Thumb are not present in latest DMD source - they are removed. Time to update the documentation.

[jacob-carlborg]

darwin is still present as far as I can see: https://github.com/D-Programming-Language/dmd/blob/master/src/mars.d#L349

[redstar]

Yes, but it is not in the list of reserved identifiers: https://github.com/D-Programming-Language/dmd/blob/master/src/cond.d#L161 up to line 250.

[jacob-carlborg]

Yeah, but as long as it's predefined it should be in the documentation. But I see now that the documentation is not entirely accurate since it claims that all the identifiers in that list are reserved, but that's not the case with darwin.

[smolt]

What about the "core/sys/osx" directory tree in druntime? Would this dir change to "darwin"? Or just retain "osx" name and change version(OSX) to version(Darwin)?

[jacob-carlborg]

It is/was called "darwin" in Tango, where druntime originated from.

[smolt]

Interesting. Naming always seems to be one of the hardest things.

I will do an experiment renaming a bunch of stuff in my clone from osx to darwin to get a feel for the impact and define version(Darwin) instead of version(OSX) for iOS. Definitely I will change version(IPhoneOS) to version(iOS).

[jacob-carlborg]

"There are only two hard things in Computer Science: cache invalidation and naming things." :wink:

[dnadlinger]

Definitely, and thanks for bearing with me. You might want to float this by @alexrp and all the others (@joakim-noah, …) who were involved in making the upstream version identifiers itself. I don't claim to be an expert for this particular kind of bikesheds myself. ;)

[smolt]

Ok. The good thing is that there is much else that can be done in parallel while names are worked out.

[smolt]

@redstar - is there any chance to get your long double work merged in to support cross-compiling?

I updated from Xcode 6 to Xcode 7.0 and ld is doing additional checks to weed out code with thread locals. This is a new problem going forward because Xcode 7 is the default download now. What is weird is that the new check in ld does allow link of objects to contain thread locals if arm64 >= iOS 8.0 but does not provide the support in dyld as far as I can tell. I may have to dig for a bit.

[redstar]

@smolt The problem with the long double stuff is that (good) support for math intrinsics is missing. cos(), sin() etc. have all (most likely) less precision in CTFE. IMHO this is a major blocker. If someone have some numerical algorithms at hand that would be cool.

[joakim-noah]

I have no opinion on the naming, both because I'd never use or develop for Apple products and I don't really care what names are chosen, only that there is a way to differentiate when necessary. However, I advise that you get a small PR for dmd in early with whatever name you have in mind, as it took me a month to get a similar PR merged last year, which only slightly renamed a C runtime and added other already defined C runtimes to the official list, followed by another two weeks of further comment. ;)

[tritao]

I've been doing some cleaning in the Mono codebase related to this so I thought I might share our approach in case it helps.

We've ended up with a Darwin define, that is common to all Apple platforms, an iOS define common to all iOS platforms, and a define specific for the iPhone, WatchOS, TVOS so we can handle all the subtle differences between them.

In our case we also need to do some runtime detection for different ARM CPUs so we can handle them differently in the compiler (we need to emulate some instructions that are lacking in certain variations).

I think version(iOS) would be fine as Apple considers it iOS synonymous with iPhone OS, and it is all the same OS for iPhone, iPad, iPod Touch, AppleTV, and iWatch.

This is not exactly true, the OS is not exactly the same between all devices. For instance on WatchOS Apple disallowed access to signals and a lot of Mach APIs. There are also differences in its CPU (ARMv7k) with slightly different calling conventions to regular ARMv7.

[jacob-carlborg]

@tritao sounds reasonable to me.

Question is, will we get any problems if someone tries to get D to work on GNU/Darwin? Do we need to care about that?

[dnadlinger]

@tritao: Thanks a lot for sharing your experience with us!

[smolt]

Sorry to have disappeared for a bit. I think @tritao's suggestion on versions seems good. It does conflicts with the SDK C TARGET macros TARGET_OS_IPHONE and TARGET_OS_IOS which mean exactly opposite. Will it cause confusion?

Xcode 7.1 SDKs for MacOSX, iPhoneOS, AppleTVOS, and WatchOS documents the TARGET_OS_ conditionals as

    TARGET_OS_* 
    These conditionals specify in which Operating System the generated code will
    run.  Indention is used to show which conditionals are evolutionary subclasses.  

    The MAC/WIN32/UNIX conditionals are mutually exclusive.
    The IOS/TV/WATCH conditionals are mutually exclusive.

        TARGET_OS_WIN32           - Generated code will run under 32-bit Windows
        TARGET_OS_UNIX            - Generated code will run under some Unix (not OSX) 
        TARGET_OS_MAC             - Generated code will run under Mac OS X variant
           TARGET_OS_IPHONE          - Generated code for firmware, devices, or simulator 
              TARGET_OS_IOS             - Generated code will run under iOS 
              TARGET_OS_TV              - Generated code will run under Apple TV OS
              TARGET_OS_WATCH           - Generated code will run under Apple Watch OS
           TARGET_OS_SIMULATOR      - Generated code will run under a simulator
           TARGET_OS_EMBEDDED       - Generated code for firmware

These end up being set by the SDKs as:

Macro	MacOSX	iPhoneOS	AppleTVOS	WatchOS
TARGET_OS_MAC	1	1	1	1
TARGET_OS_IPHONE	0	1	1	1
TARGET_OS_IOS	0	1	0	0
TARGET_OS_TV	0	0	1	0
TARGET_OS_WATCH	0	0	0	1

[smolt]

Swift has only #if os() checks for OSX, iOS, watchOS, and tvOS. They are all mutually exclusive and match the SDK you are building against. I also don't see support in LLVM Triple yet of WatchOS or TVOS, we can kick that can down the road. I think an upstream dmd PR to add iOS and Darwin predefined versions is good enough for now. I'll also start a discussion on changing druntime osx packages to darwin.

[smolt]

LLVM is getting WatchOS and TVOS (http://comments.gmane.org/gmane.comp.compilers.llvm.cvs/279954). Probably should add these versions too.

[smolt]

Just an update. It is important to get changes upstream first. Latest proposal for a versioning structure is in PR D-Programming-Language/druntime#1448. The Darwin family versions as of now are OSX, iOS, TVOS, and WatchOS with modules defining versions Darwin or DarwinEmbedded as needed based on the other OS versions.

Currently DarwinEmbedded only shows up in my phobos repo (https://github.com/smolt/phobos) and would be defined for iOS, TVOS, and WatchOS. It is helpful in cases such as disabling a unit test that relies on fork() because it fails for a regular user on iOS, tvOS, or watchOS.

[smolt]

Work on this should kick off again in July. Some iOS support is already upstream and iOS is now natively supporting thread locals as of LLVM 3.8.0 and iOS 8 (arm64), iOS 9 (armv7). Handling real as a cross-compiler is a must have and @kinke #1317 looks like a good start.

[smolt]

Work on this should kick off again in July

Ok, its way past July. Here is an update:

Started testing LDC targeting iOS, using native thread local support and stock LLVM 3.9.1. During testing a bug was uncovered in iOS TLS for armv7: codegen assumes r12 will not be clobbered by TLS lookup, but it is. I was able to recreate with a small C program. It could be either an iOS or LLVM issue, so I started by filing with Apple bug 30042849. Good news is that arm64 TLS implementation seems solid and druntime/phobos unit tests pass.

[kinke]

Slightly off-topic: AArch64 still needs some work though, right? #1931 (on Linux) shows at least a missing vararg implementation in druntime. Any more AArch64 issues you know about?

[smolt]

The iOS arm64 ABI just uses char* for varargs, so that doesn't help with aarch64 on linux. There are a number of difference between the abi-ios-arm64.cpp and abi-aarch64.cpp, but can't say if they are relevant without a test platform. I believe I can run AArch64 based linux on Pi 3. I'll look into it, but my D time is sparse currently.

[kinke]

I didn't know there were separate TargetABIs, as I thought your LDC-specific patches for iOS were already in LDC master. ;) What's your plan in this regard? Merge all at once as soon as it works sufficiently?

I believe I can run AArch64 based linux on Pi 3. I'll look into it, but my D time is sparse currently.

That'd be nice, but I didn't intend to put any pressure on you, I just thought both AArch64-Linux and -iOS shared the identical TargetABI class (abi-aarch64.cpp) and was thus wondering why you didn't mention any vararg issues.

[patrickkh7788]

hi @kinke

I didn't know there were separate TargetABIs, as I thought your LDC-specific patches for iOS were already in LDC master. ;)

What need to be done to make the the master ldc be able to used for IOS with -betterC ?

[joakim-noah]

You'd have to look at applying the patches in Dan's last release branch onto master, around 35 files that he still had to patch. Not a huge diff and you may not need all of it for betterC, but it will take some work.

[patrickkh7788]

@joakim-noah, thanks very much for all you help , I has merge the betterC branch into ios branch. but the old llvm is not working since a lot code updated.

I guess if I cherry pick all ABI and float change then it will be able to work without TLS.

@smolt said the LLVM already support TLS, but the still has TLS patch, I guess in the ldc master branch TLS patch is no need any more?

[joakim-noah]

the old llvm is not working since a lot code updated.

Not sure what you mean, the llvm 3.6.2 he patched isn't working because you updated his ldc with all the ltsmaster commits from my betterC branch? But ltsmaster and my branch work fine with llvm 3.6.2.

I guess if I cherry pick all ABI and float change then it will be able to work without TLS... I guess in the ldc master branch TLS patch is no need any more?

I don't know what he had to do to get TLS working, never looked at his patches.

I'm not sure what you mean by "code updated" above, my betterC branch adds very few commits. If you simply merged all of the ltsmaster branch that it's based on and that's somehow causing problems, maybe try applying the betterC diff alone to his iOS branch?

[joakim-noah]

Anybody with an iOS device to test on interested in getting Dan's patches into master? I took a look at the diff against ltsmaster, not that large, but I don't use Apple devices.

[smolt]

Just a note - Apple fixed bug 30042849 for 32-bit arm TLS. Which means stock LLVM should work. No more TLS patches!

[patrickkh7788]

@smolt but we still need modify ldc ABI to work with ldc ? or we can use AArch64 abi ?

[smolt]

@patrickkh7788 The iOS arm64 ABI is slightly different from AArch64 ABI and somewhat simpler. My LDC iOS fork did have a working iOS arm64 abi file. Let me refresh my memory this weekend and I'll try to help. I haven't looked at this much in a couple years now :turtle:

[patrickkh7788]

@smolt Thanks very much for the great work, This is really great news.

[patrickkh7788]

any update ?

[zoujiaqing]

How is it going?

[patrickkh7788]

Is any one can help me to cherry pick the -arch part into ldc master, I try myself but there is too much conflict , a lot file not even exists(removed from ldc).

If we have -arch and abi supported, I guess we can use with betterC for IOS app. I want this for years, please help me. or suggestion what should I do with about the cherry-pick process .

[s-ludwig]

https://www.bountysource.com/issues/26721376-add-ios-tvos-and-watchos-support-to-ldc

[jacob-carlborg]

@s-ludwig are you aware of https://www.flipcause.com/secure/cause_pdetails/NjU3MTY=?

There's also https://github.com/ldc-developers/ldc/pull/3288.

[s-ludwig]

I hadn't seen those, no, and kind of forgot about flipcause, although I knew that you were already working on this. Nice, it looks like this is all coming together! This and the Android effort should finally enable D to come out of its computational/web'ish niche.

[zoujiaqing]

https://www.bountysource.com/issues/26721376-add-ios-tvos-and-watchos-support-to-ldc

[etcimon]

https://www.bountysource.com/issues/26721376-add-ios-tvos-and-watchos-support-to-ldc