[SR-40] Port Swift to Linux on Raspberry Pi

[gribozavr]

Previous ID	SR-40
Radar	None
Original Reporter	@gribozavr
Type	New Feature
Status	Resolved
Resolution	Done

Attachment: Download

Additional Detail from JIRA

| | | |------------------|-----------------| |Votes | 24 | |Component/s | Compiler, Foundation, Package Manager, Standard Library, XCTest | |Labels | New Feature, NewPortRequest, arm | |Assignee | @hpux735 | |Priority | Medium | md5: 7062026e0685ad58b94ffcbb4be1f47b

Sub-Tasks:

• SR-387 Undef. symbols when linking SwiftPM
• SR-359 Undef. symbols when linking to Foundation (ARM)

Issue Description:

It would be good to get Swift compiler running on Raspberry Pi, as well as having a cross-compiler from x86 to Raspberry Pi.

[hpux735]

I've started playing around with compiling on ARMv7, using a BeagleBone, rather than a RasPi. I encountered a clang++ crash during build:

{{ [174/1922] Building CXX object lib/IR/CMakeFiles/LLVMCore.dir/LegacyPassManager.cpp.o FAILED: /usr/bin/clang++ -DGTEST_HAS_RTTI=0 -D_DEBUG -D_GNU_SOURCE -DSTDC_CONSTANT_MACROS -DSTDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -fPIC -fvisibility-inlines-hidden -Wall -W -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wmissing-field-initializers -pedantic -Wno-long-long -Wcovered-switch-default -Wnon-virtual-dtor -std=c++11 -fcolor-diagnostics -g -Ilib/IR -I/root/swift-build/llvm/lib/IR -Iinclude -I/root/swift-build/llvm/include -fno-exceptions -fno-rtti -MMD -MT lib/IR/CMakeFiles/LLVMCore.dir/Function.cpp.o -MF "lib/IR/CMakeFiles/LLVMCore.dir/Function.cpp.o.d" -o lib/IR/CMakeFiles/LLVMCore.dir/Function.cpp.o -c /root/swift-build/llvm/lib/IR/Function.cpp clang: error: unable to execute command: Killed clang: error: clang frontend command failed due to signal (use -v to see invocation) Ubuntu clang version 3.6.0-2ubuntu1~trusty1 (tags/RELEASE_360/final) (based on LLVM 3.6.0) Target: arm-unknown-linux-gnueabihf Thread model: posix clang: note: diagnostic msg: PLEASE submit a bug report to http://bugs.debian.org/ and include the crash backtrace, preprocessed source, and associated run script. clang: note: diagnostic msg: ********************

PLEASE ATTACH THE FOLLOWING FILES TO THE BUG REPORT: Preprocessed source(s) and associated run script(s) are located at: clang: note: diagnostic msg: /tmp/Function-8fe796.cpp clang: note: diagnostic msg: /tmp/Function-8fe796.sh clang: note: diagnostic msg:

******************** ninja: build stopped: subcommand failed. utils/build-script: command terminated with a non-zero exit status 1, aborting }}

The mentioned files are attached to this comment.

clang-crash-BeagleBoneBlack.tar.bz2

[gribozavr]

How much RAM does your device have? Building just LLVM and Clang requires more than 4 Gb, probably 8 Gb, IIRC. Building Swift requires 12 Gb if building with debug information (for the linking step).

I think you would want to try building a cross-compiler instead.

[hpux735]

Yep, you're right.

I found somewhere that using the release (-R) flag decreases memory requirements, and I added an moderate swapfile. I'm working on setting up a cross-compiling toolchain, but I've never had a good experience doing that, so I don't have high hopes. Currently it's about 1/2 finished compiling CLANG on the beaglebone. I expect that to finish perhaps tonight. 😉

I've also got a Jetson TK1 with 2GB of ram (the Beaglebone black has 512MB) that I can try.

[hpux735]

I think I'm on the right track with swift/utils/build-script-impl changes, and I tested that on Linux ARM and x86-64 (I haven't tested to make sure that changes don't hurt Darwin or FreeBSD yet). On x86-64, compilation completes successfully, and tests (1 unexpected failure, not sure if that means anything). On ARM, clang, cmark, and llvm build, swift doesn't; I'll start tackling that on monday.

https://github.com/hpux735/swift

[swift-ci]

Comment by Kody Holman (JIRA)

@hpux735 Would it be helpful to have an emulated raspberry pi running on a mac to get everything compiled?

[hpux735]

Yah, most likely. Can I give it a lot of ram? 🙂

[swift-ci]

Comment by Joe (JIRA)

@hpux735, I recommend looking into a Wandboard, particularly the Quad http://www.wandboard.org/. I have a a BeagleBoard X15 (http://www.elinux.org/Beagleboard:BeagleBoard-X15) that I'd be happy to test compiling and executing if you're using a specific branch.

It is the link steps that gobble the RAM, the X15 comes with 2GB and I had to add an 8G swapfile. Slow when it gets to those stages.

Let me know how I can help!

[swift-ci]

Comment by Kyle Frost (JIRA)

@hpux735 Looks like your GitHub says you have a working build. Does this mean you can actually run Swift on Raspberry Pi? Or build succeeds but it's broken.

[hpux735]

Thanks for those links, Joe. I've got a few Tegra TK1s with 2GB of ram, and they're doing alright. I'm looking into getting a Tegra TX1, which has 3GB. For building the debug variant, Nick (on the mailing list) reminded me that the 32 bit machines probably can't build it, no matter the swap size.

Kyle: I am able to build binaries for armv7l, but there is a problem in the linking stage somewhere. I'm getting an error "unexpected reloc type 0x03". So far, I haven't been able to figure out the root cause or a possible solution to that. I just don't have that great an understanding about how linking works at that level.

There was another pull request that solves an architecture dependency in the module.map for Glibc that I wanted to base my patch on, so I re-forked off the main apple repo, and I'm manually re-making the changes; it's still a little broken, but I expect that to be finished sometime today.

Ultimately, I think it's really, really close but I could use some help with that linking issue.

[swift-ci]

Comment by Félix Lapalme (JIRA)

I tried to build Swift on my Pi 2 about a week ago and ended up giving up because I always got a "memory exhausted" message when doing the final linking steps. I figured (after Googling a bit) it was because it was trying to work with files greater than 4GB and the Pi's 32 bit CPU couldn't handle that. You say the debug variant can't be built with a 32-bit processor, but can the "release" build be built with a 32 bit CPU?

I was using the -m option to use cmake instead of ninja. It worked quite well (with a few issues that I could easily work around) except for that 32-bit related memory issue. Did you try building it with that option ? Or is there some disadvantage to building it with cmake ?

[hpux735]

I have built using release mode on a BeagleBone Black, and on a Tegra TK1. Debug definitely doesn't work. I have tried on a Raspberry Pi, though with sufficient swap space, I don't see a reason why it wouldn't work.

I haven't tried using it with -m, so I don't know whether/how well it would work.

[swift-ci]

Comment by Mike Greiner (JIRA)

Has anyone tried cross-compiling the toolchain? That would certainly avoid the memory issues.

I've cross-compiled a few OSS projects before but I think this would be a bit beyond my expertise. If anyone's gotten a start on it though, I'd be happy to help out where I can.

[swift-ci]

Comment by Cody Krieger (JIRA)

mgreiner (JIRA User): I've started looking into reworking the build tooling to support cross-compilation in a generic way, but there are lots of assumptions that need to be changed in order to make that happen. No ETA yet.

[swift-ci]

Comment by Félix Lapalme (JIRA)

Someone already started to make cross compiling work for Android ( https://github.com/SwiftAndroid/swift ) maybe it would be a good starting point ?

With optimizations enabled and debugging disabled, my Pi seems to be compiling Swift quite well (98% done so far. I don't think it ever got this far). If I understood correctly, the 32-bit memory issues only occur when building Swift in debugging mode (so, without the -R argument for build-script)

[hpux735]

Thanks for reminding me about SwiftAndroid. That actually provided the key I needed to get swift linking correctly on ARMv7. I need to issue a pull request into the main repo, but for now, if you'd like to clone my repo, your compiler should be able to generate working executables.

https://github.com/hpux735/swift

[swift-ci]

Comment by Félix Lapalme (JIRA)

Wow nice job 

My Pi is now building Swift with your version (and using ninja this time). We'll see how it goes.. (currently building LLVM and it's at [1636/1921])

EDIT: Actually, I'll cancel this build and I'll try with tienex's fork ( https://github.com/apple/swift/pull/608 ), which looks better, as you've seen

[hpux735]

-With his current (at least as of now) repo, you'll need to edit line 162 in cmake/modules/AddSwift.cmake (I think) to be:

list(APPEND result "-lpthread" "-ldl" "-Wl,Bsymbolic")

Disregard all that. I was on the wrong branch of his repo.

[swift-ci]

Comment by Félix Lapalme (JIRA)

I'm getting a compile error from both repos once it's building Swift (after building LLVM):
Swift compile error.txt

I'm assuming the C++ code is fine, but is there some build setting that's missing that could potentially fix this issue ? Is it working on anyone else's Pi ? Maybe it has to do with my Pi configuration ?

[hpux735]

What clang compiler are you using?

[gribozavr]

I have seen that issue with older versions of Clang. Try this patch:

diff --git a/lib/IRGen/ClassMetadataLayout.h b/lib/IRGen/ClassMetadataLayout.h
index 7612218..5deb326 100644
--- a/lib/IRGen/ClassMetadataLayout.h
+++ b/lib/IRGen/ClassMetadataLayout.h
@@ -38,9 +38,11 @@ protected:
   using super::IGM;
   using super::asImpl;

+public:
   /// The most-derived class.
   ClassDecl *const Target;

+protected:
   ClassMetadataLayout(IRGenModule &IGM, ClassDecl *target)
     : super(IGM), Target(target) {}

diff --git a/lib/IRGen/EnumMetadataLayout.h b/lib/IRGen/EnumMetadataLayout.h
index 2dbd653..7ecad51 100644
--- a/lib/IRGen/EnumMetadataLayout.h
+++ b/lib/IRGen/EnumMetadataLayout.h
@@ -33,9 +33,11 @@ protected:
   using super::IGM;
   using super::asImpl;

+public:
   /// The Enum.
   EnumDecl *const Target;

+protected:
   EnumMetadataLayout(IRGenModule &IGM, EnumDecl *target)
     : super(IGM), Target(target) {}

diff --git a/lib/IRGen/GenMeta.cpp b/lib/IRGen/GenMeta.cpp
index eb8517a..77b075a 100644
--- a/lib/IRGen/GenMeta.cpp
+++ b/lib/IRGen/GenMeta.cpp
@@ -2067,7 +2067,7 @@ namespace {
   struct SEARCHER : MetadataSearcher<Scanner<SEARCHER<Scanner>>> {      \
     using super = MetadataSearcher<Scanner<SEARCHER<Scanner>>>;         \
     using super::Target;                                                \
-    using TargetType = decltype(Target);                                \
+    using TargetType = decltype(super::Target);                         \
     SEARCHER(IRGenModule &IGM, TargetType target)                       \
       : super(IGM, target) {}
 #define BEGIN_GENERIC_METADATA_SEARCHER_1(SEARCHER, TYPE_1, NAME_1)     \
@@ -2075,7 +2075,7 @@ namespace {
   struct SEARCHER : MetadataSearcher<Scanner<SEARCHER<Scanner>>> {      \
     using super = MetadataSearcher<Scanner<SEARCHER<Scanner>>>;         \
     using super::Target;                                                \
-    using TargetType = decltype(Target);                                \
+    using TargetType = decltype(super::Target);                         \
     TYPE_1 NAME_1;                                                      \
     SEARCHER(IRGenModule &IGM, TargetType target, TYPE_1 NAME_1)        \
       : super(IGM, target), NAME_1(NAME_1) {}
@@ -2085,7 +2085,7 @@ namespace {
   struct SEARCHER : MetadataSearcher<Scanner<SEARCHER<Scanner>>> {      \
     using super = MetadataSearcher<Scanner<SEARCHER<Scanner>>>;         \
     using super::Target;                                                \
-    using TargetType = decltype(Target);                                \
+    using TargetType = decltype(super::Target);                         \
     TYPE_1 NAME_1;                                                      \
     TYPE_2 NAME_2;                                                      \
     SEARCHER(IRGenModule &IGM, TargetType target,                       \
@@ -2114,7 +2114,7 @@ namespace {
   class MetadataSizer : public Scanner<MetadataSizer<Scanner>> {
     typedef Scanner<MetadataSizer<Scanner>> super;
     using super::Target;
-    using TargetType = decltype(Target);
+    using TargetType = decltype(super::Target);

     Size AddressPoint = Size::invalid();
   public:
diff --git a/lib/IRGen/StructMetadataLayout.h b/lib/IRGen/StructMetadataLayout.h
index fc798ad..081047c 100644
--- a/lib/IRGen/StructMetadataLayout.h
+++ b/lib/IRGen/StructMetadataLayout.h
@@ -33,9 +33,11 @@ protected:
   using super::IGM;
   using super::asImpl;

+public:
   /// The struct.
   StructDecl *const Target;

+protected:
   StructMetadataLayout(IRGenModule &IGM, StructDecl *target)
     : super(IGM), Target(target) {}

[swift-ci]

Comment by PeteVine (JIRA)

https://bugs.swift.org/browse/SR-333

[swift-ci]

Comment by Dallas Brown (JIRA)

I spent several days trying to get it working, both with local compiling and doing cross-compiling, and each time it would fail somewhere around 24hr's into compiling (local) and 12 hr's (cross). I even tried compiling new versions of clang instead of using whats in apt-get as its such an old version (that made it get a couple hours further along).
Unfortunately I don't remember what the errors I got were.
I started working on this like the day Swift was opened sourced and then kind of gave up.

[swift-ci]

Comment by Francesco Frison (JIRA)

Someone made it: https://twitter.com/armadsen/status/680877848043728896

[hpux735]

Yep, that's where the state of things have been for about a week, I think. However, there are several issues that prevent the port from being what I would call "minimally useful". I've added bug reports for these and added them to this bug as subtasks. See SR-387, SR-388, and SR-359

Joe Bell made a bunch of progress on SR-388 (getting the REPL to work). It launches now, but breaks:

{{
wdillon@tegra-ubuntu:\~$ swift -lldb-repl
Welcome to Swift version 2.2-dev (LLVM 3ebdbb2c7e, Clang f66c5bb67b, Swift 9bd3eb7bc2). Type :help for assistance.
1> print("Hello world")
lldb: /home/wdillon/swift/lib/AST/Module.cpp:1477: void performAutoImport(swift::SourceFile &, SourceFile::ImplicitModuleImportKind): Assertion `M && "unable to auto-import module"' failed.
Aborted
wdillon@tegra-ubuntu:\~$
}}

[swift-ci]

Comment by Tom Gall (JIRA)

Is there any updates? I'm in the midst of catching up. I have a full build using clang 3.8 + gcc 4.9.3 (on Gentoo) building native on arm-linux (32 bit on a Chromebook2)

On trying to run the first swift test program I'm getting :

/home/tgall/swift/build/Ninja-ReleaseAssert/swift-linux-armv7/test-linux-armv7/1_stdlib/Output/Bit.swift.tmp/a.out: error while loading shared libraries: /home/tgall/swift/build/Ninja-ReleaseAssert/swift-linux-armv7/lib/swift/linux/libswiftCore.so: unexpected reloc type 0x03

....

[hpux735]

Hi tgall (JIRA User). You shouldn't be getting that. What repos are you using? Here is a list of repos that should work (for armv7):

#!/bin/bash
git clone https://github.com/hpux735/swift.git swift
git clone https://github.com/apple/swift-llvm.git llvm
git clone https://github.com/apple/swift-clang.git clang
git clone https://github.com/hpux735/swift-lldb.git lldb
git clone https://github.com/apple/swift-cmark.git cmark
git clone https://github.com/apple/swift-llbuild.git llbuild
git clone https://github.com/apple/swift-package-manager.git swiftpm
git clone https://github.com/apple/swift-corelibs-xctest.git
git clone https://github.com/hpux735/swift-corelibs-foundation.git

[swift-ci]

Comment by Tom Gall (JIRA)

Ok! so following up and given some reading on the mailing list from back in December, I have a build that can run tests now.

This post: https://www.mail-archive.com/swift-dev@swift.org/msg00330.html

Leads me to this patch:

https://github.com/SwiftAndroid/swift/commit/364aa03cb423d875679096a8456f6c8399acf884?diff=split

You JUST need "-Wl,-Bsymbolic" added and things start to work...

[hpux735]

Yep, that's right. It should be in the repos already, though... hmm.

[swift-ci]

Comment by Tom Gall (JIRA)

yeah I "froze" my source for a bit as I was working through gentoo-isms and such as I was building. Time to catch up!

[hpux735]

Cool. I'm excited to see what you come up with. I'm building gentoo on my arm64 machine.

[swift-ci]

Comment by Lucas Goossen (JIRA)

I can't seem to find the answer. Has anyone got cross compiling working?

[hpux735]

lucasgoossen (JIRA User) I'm not aware of a stable project with a cross-compiler yet. The swift-andrioid (or is it android-swift) project is working on a swift cross compiler for android, but I don't think the compiler is cross-compiled.

People are investigating and working on it, but for now, it's native compilation of the compiler itself.

[swift-ci]

Comment by Lucas Goossen (JIRA)

By what I have read there isn't much chance of compiling on a B+ due to memory. Is this correct?

[hpux735]

It can be possible, I've done it before. But it takes release mode, a lot of patience and perseverance. (and lots of swap)

[swift-ci]

Comment by Michael Armstrong (JIRA)

lucasgoossen (JIRA User) the B+ is tricky due to all of the above but also that it won't run Ubuntu MATE (its not armv7). In Raspbian some of the dependencies in the stock repo's are a little old so you'll need to compile most from source. You also need to make a large swap file (circa 2GB) using dphys-swapfile.

[hpux735]

For all my work, I've been using Jessie Light (on both Armv6 and Armv7)

https://www.raspberrypi.org/downloads/raspbian/

[swift-ci]

Comment by Michael Armstrong (JIRA)

Ah great, i didn't realise you could get Jessie on the PI. In that case, the only thing you should need is a lot of swap 🙂

[hpux735]

With the merge of PR#1157, I think this bug has been substantially resolved. I'm going to close it for now, but it could be re-opened if the cross-compilation piece of the puzzle should stay here. Tom Birch is working on cross compilation at the moment, but it's more general so it might fit better in a general cross-compiler bug.

[swift-ci]

Comment by Isaac Arvestad (JIRA)

Accidentally changed rank when viewing in iOS browser. Can't figure out how to revert or what the previous rank was.

[swift-ci]

Comment by Chris Donnelly (JIRA)

This is marked closed but there still appears to be no build capability for x86 (32-bit). I'm assuming here it's only closed because there is now an armv? build capability.

[hpux735]

Excellent point. When I did that, I had forgotten that part of the bug was for cross-compilation. That kinda makes me want to re-open it. Are you interested in working on cross-compilation?

[swift-ci]

Comment by Chris Donnelly (JIRA)

I'd like to say yes but I'm really not skilled enough to help. I'm just getting started with learning swift and recently makefiles, using and updating from git. I could help compile on my 32 bit machine and provide feedback. I've read enough of the code to see the build process is not broad enough. Would of thought there would be a change of focus as of 3.0 to support as varied hardware as possible. SR124https://bugs.swift.org/browse/SR-124is related. Is it possible to rename the title so it better matches the contents of description.