Building i686-unknown-linux-gnu target on OS X fails

[alexchandel]

I'm trying to build the rust libraries for multiple targets on OS X for cross-compilation purposes. Building with,

./configure --target=x86_64-apple-darwin,i686-unknown-linux-gnu,arm-unknown-linux-gnueabi
make

gives me:

rustc: x86_64-apple-darwin/stage2/lib/rustlib/x86_64-apple-darwin/lib/libhexfloat
rustc: x86_64-apple-darwin/stage2/lib/rustlib/x86_64-apple-darwin/lib/libregex_macros
compile: i686-unknown-linux-gnu/rt/arch/i386/morestack.o
/Users/alex/GitHub/rust/src/rt/arch/i386/morestack.S:93:18: error: unable to emit symbol attribute
 .private_extern ___morestack
                 ^
make: *** [i686-unknown-linux-gnu/rt/arch/i386/morestack.o] Error 1

[alexchandel]

However, manually compiling src/rt/arch/*/morestack.S for all three of those targets succeeds.

[brson]

Can you paste the error from make VERBOSE=1?

On x86 we use the in-tree llvm-mc to compile assembly, and if that program is being invoked correctly, it should not be possible for an error here.

[alexchandel]

Here's the make VERBOSE=1 error:

DYLD_LIBRARY_PATH=/Users/alex/GitHub/rust/x86_64-apple-darwin/stage2/lib:$DYLD_LIBRARY_PATH   x86_64-apple-darwin/stage2/bin/rustc --cfg stage2  -O --cfg rtopt --cfg debug  --target=x86_64-apple-darwin  -D warnings -L "x86_64-apple-darwin/rt" -L "/Users/alex/GitHub/rust/x86_64-apple-darwin/llvm/Release+Asserts/lib" -L ""  --out-dir x86_64-apple-darwin/stage2/lib/rustlib/x86_64-apple-darwin/lib -C extra-filename=-4e7c5e5c /Users/alex/GitHub/rust/src/libhexfloat/lib.rs
info: now are following matches for libhexfloat-*.dylib libraries:
x86_64-apple-darwin/stage2/lib/rustlib/x86_64-apple-darwin/lib/libhexfloat-4e7c5e5c.dylib
info: now are following matches for libhexfloat-*.rlib libraries:
x86_64-apple-darwin/stage2/lib/rustlib/x86_64-apple-darwin/lib/libhexfloat-4e7c5e5c.rlib
MATCHES=""; if [ -n "$MATCHES" ] ; then echo "warning: removing previous" \'libregex_macros-*.dylib\' "libraries:" $MATCHES; rm $MATCHES ; fi
MATCHES=""; if [ -n "$MATCHES" ] ; then echo "warning: removing previous" \'libregex_macros-*.rlib\' "libraries:" $MATCHES; rm $MATCHES ; fi
DYLD_LIBRARY_PATH=/Users/alex/GitHub/rust/x86_64-apple-darwin/stage2/lib:$DYLD_LIBRARY_PATH   x86_64-apple-darwin/stage2/bin/rustc --cfg stage2  -O --cfg rtopt --cfg debug  --target=x86_64-apple-darwin  -D warnings -L "x86_64-apple-darwin/rt" -L "/Users/alex/GitHub/rust/x86_64-apple-darwin/llvm/Release+Asserts/lib" -L ""  --out-dir x86_64-apple-darwin/stage2/lib/rustlib/x86_64-apple-darwin/lib -C extra-filename=-4e7c5e5c /Users/alex/GitHub/rust/src/libregex_macros/lib.rs
info: now are following matches for libregex_macros-*.dylib libraries:
x86_64-apple-darwin/stage2/lib/rustlib/x86_64-apple-darwin/lib/libregex_macros-4e7c5e5c.dylib
info: now are following matches for libregex_macros-*.rlib libraries:
clang -E -E -MMD -MP -MT i686-unknown-linux-gnu/rt/arch/i386/morestack.o -MF i686-unknown-linux-gnu/rt/arch/i386/morestack.d /Users/alex/GitHub/rust/src/rt/arch/i386/morestack.S | /Users/alex/GitHub/rust/x86_64-apple-darwin/llvm/Release+Asserts/bin/llvm-mc -assemble -filetype=obj -triple=i686-unknown-linux-gnu -o=i686-unknown-linux-gnu/rt/arch/i386/morestack.o
/Users/alex/GitHub/rust/src/rt/arch/i386/morestack.S:93:18: error: unable to emit symbol attribute
 .private_extern ___morestack
                 ^
make: *** [i686-unknown-linux-gnu/rt/arch/i386/morestack.o] Error 1

[elexing]

@brson I have encountered the same problem many times, do you have a plan to solve it ?

[alexchandel]

Longer output. This is built with:

./configure --target=x86_64-apple-darwin,i686-unknown-linux-gnu,arm-unknown-linux-gnueabi
make

I believe this uses the clang version supplied with Xcode, but I have the LLVM head installed to /usr/local/opt/llvm. Would using that one help, and if so how do I change which clang the rust build uses?

[cmsd2]

I don't think there's anything wrong with the way llvm-mc is called. But the initial call to clang on the left of the pipe doesn't pass any target options. Clang seems to produce the private extern modifier only when targetting osx.

[cmsd2]

as a quick workaround i tried adding --target=i686-unknown-linux-gnu to the clang command above, then resuming the build with make. next it gets stuck on:

clang '-Wl,--as-needed' '-m32' '-L' '/Users/chris/Development/cmsd2/rust/x86_64-apple-darwin/stage2/lib/rustlib/i686-unknown-linux-gnu/lib' '-o' 'x86_64-apple-darwin/stage2/lib/rustlib/i686-unknown-linux-gnu/lib/libstd-4e7c5e5c.so' 'x86_64-apple-darwin/stage2/lib/rustlib/i686-unknown-linux-gnu/lib/std-4e7c5e5c.o' '-Wl,--whole-archive' '-lmorestack' '-Wl,--no-whole-archive' 'x86_64-apple-darwin/stage2/lib/rustlib/i686-unknown-linux-gnu/lib/std-4e7c5e5c.metadata.o' '-Wl,-O1' '-nodefaultlibs' '/Users/chris/Development/cmsd2/rust/x86_64-apple-darwin/stage2/lib/rustlib/i686-unknown-linux-gnu/lib/librand-4e7c5e5c.rlib' '/Users/chris/Development/cmsd2/rust/x86_64-apple-darwin/stage2/lib/rustlib/i686-unknown-linux-gnu/lib/libcollections-4e7c5e5c.rlib' '/Users/chris/Development/cmsd2/rust/x86_64-apple-darwin/stage2/lib/rustlib/i686-unknown-linux-gnu/lib/libunicode-4e7c5e5c.rlib' '/Users/chris/Development/cmsd2/rust/x86_64-apple-darwin/stage2/lib/rustlib/i686-unknown-linux-gnu/lib/liballoc-4e7c5e5c.rlib' '/Users/chris/Development/cmsd2/rust/x86_64-apple-darwin/stage2/lib/rustlib/i686-unknown-linux-gnu/lib/liblibc-4e7c5e5c.rlib' '/Users/chris/Development/cmsd2/rust/x86_64-apple-darwin/stage2/lib/rustlib/i686-unknown-linux-gnu/lib/libcore-4e7c5e5c.rlib' '-L' 'i686-unknown-linux-gnu/rt' '-L' '.' '-L' '.' '-L' '/Users/chris/Development/cmsd2/rust/.rust' '-L' '/Users/chris/Development/cmsd2/rust' '-Wl,--whole-archive' '-Wl,-Bstatic' '-lrust_builtin' '-lbacktrace' '-lrustrt_native' '-Wl,--no-whole-archive' '-Wl,-Bdynamic' '-ldl' '-lpthread' '-lrt' '-lgcc_s' '-lpthread' '-lc' '-lm' '-shared' '-lcompiler-rt’
ld: unknown option: --as-needed

i guessed again that there should be a --target option since it's trying to build a .so, not a .dylib. also added is the -v option to show the linker command:

clang --target=i686-unknown-linux-gnu '-Wl,--as-needed' '-m32' '-L' '/Users/chris/Development/cmsd2/rust/x86_64-apple-darwin/stage2/lib/rustlib/i686-unknown-linux-gnu/lib' '-o' 'x86_64-apple-darwin/stage2/lib/rustlib/i686-unknown-linux-gnu/lib/libstd-4e7c5e5c.so' 'x86_64-apple-darwin/stage2/lib/rustlib/i686-unknown-linux-gnu/lib/std-4e7c5e5c.o' '-Wl,--whole-archive' '-lmorestack' '-Wl,--no-whole-archive' 'x86_64-apple-darwin/stage2/lib/rustlib/i686-unknown-linux-gnu/lib/std-4e7c5e5c.metadata.o' '-Wl,-O1' '-nodefaultlibs' '/Users/chris/Development/cmsd2/rust/x86_64-apple-darwin/stage2/lib/rustlib/i686-unknown-linux-gnu/lib/librand-4e7c5e5c.rlib' '/Users/chris/Development/cmsd2/rust/x86_64-apple-darwin/stage2/lib/rustlib/i686-unknown-linux-gnu/lib/libcollections-4e7c5e5c.rlib' '/Users/chris/Development/cmsd2/rust/x86_64-apple-darwin/stage2/lib/rustlib/i686-unknown-linux-gnu/lib/libunicode-4e7c5e5c.rlib' '/Users/chris/Development/cmsd2/rust/x86_64-apple-darwin/stage2/lib/rustlib/i686-unknown-linux-gnu/lib/liballoc-4e7c5e5c.rlib' '/Users/chris/Development/cmsd2/rust/x86_64-apple-darwin/stage2/lib/rustlib/i686-unknown-linux-gnu/lib/liblibc-4e7c5e5c.rlib' '/Users/chris/Development/cmsd2/rust/x86_64-apple-darwin/stage2/lib/rustlib/i686-unknown-linux-gnu/lib/libcore-4e7c5e5c.rlib' '-L' 'i686-unknown-linux-gnu/rt' '-L' '.' '-L' '.' '-L' '/Users/chris/Development/cmsd2/rust/.rust' '-L' '/Users/chris/Development/cmsd2/rust' '-Wl,--whole-archive' '-Wl,-Bstatic' '-lrust_builtin' '-lbacktrace' '-lrustrt_native' '-Wl,--no-whole-archive' '-Wl,-Bdynamic' '-ldl' '-lpthread' '-lrt' '-lgcc_s' '-lpthread' '-lc' '-lm' '-shared' '-lcompiler-rt' -v
Apple LLVM version 6.0 (clang-600.0.56) (based on LLVM 3.5svn)
Target: i386-unknown-linux-gnu
Thread model: posix
Selected GCC installation: 
 "/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ld" --eh-frame-hdr -m elf_i386 -shared -o x86_64-apple-darwin/stage2/lib/rustlib/i686-unknown-linux-gnu/lib/libstd-4e7c5e5c.so crti.o crtbeginS.o -L/Users/chris/Development/cmsd2/rust/x86_64-apple-darwin/stage2/lib/rustlib/i686-unknown-linux-gnu/lib -Li686-unknown-linux-gnu/rt -L. -L. -L/Users/chris/Development/cmsd2/rust/.rust -L/Users/chris/Development/cmsd2/rust -L/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib -L/usr/lib --as-needed x86_64-apple-darwin/stage2/lib/rustlib/i686-unknown-linux-gnu/lib/std-4e7c5e5c.o --whole-archive -lmorestack --no-whole-archive x86_64-apple-darwin/stage2/lib/rustlib/i686-unknown-linux-gnu/lib/std-4e7c5e5c.metadata.o -O1 /Users/chris/Development/cmsd2/rust/x86_64-apple-darwin/stage2/lib/rustlib/i686-unknown-linux-gnu/lib/librand-4e7c5e5c.rlib /Users/chris/Development/cmsd2/rust/x86_64-apple-darwin/stage2/lib/rustlib/i686-unknown-linux-gnu/lib/libcollections-4e7c5e5c.rlib /Users/chris/Development/cmsd2/rust/x86_64-apple-darwin/stage2/lib/rustlib/i686-unknown-linux-gnu/lib/libunicode-4e7c5e5c.rlib /Users/chris/Development/cmsd2/rust/x86_64-apple-darwin/stage2/lib/rustlib/i686-unknown-linux-gnu/lib/liballoc-4e7c5e5c.rlib /Users/chris/Development/cmsd2/rust/x86_64-apple-darwin/stage2/lib/rustlib/i686-unknown-linux-gnu/lib/liblibc-4e7c5e5c.rlib /Users/chris/Development/cmsd2/rust/x86_64-apple-darwin/stage2/lib/rustlib/i686-unknown-linux-gnu/lib/libcore-4e7c5e5c.rlib --whole-archive -Bstatic -lrust_builtin -lbacktrace -lrustrt_native --no-whole-archive -Bdynamic -ldl -lpthread -lrt -lgcc_s -lpthread -lc -lm -lcompiler-rt crtendS.o crtn.o
ld: warning: directory not found for option '-L/Users/chris/Development/cmsd2/rust/.rust'
ld: unknown option: --eh-frame-hdr

I don't know how to fix that --eh-frame-hdr option. I tried messing with the ld command line to leave out the option to see what would happen. I also have a gcc cross compiler installed, so i try the linker from that. ld then whinges about the crt objects. I don't know but I suppose that's not surprising, so i remove the references to the first two that it complains about (crti.o and crtbeginS.o), again to see what happens:

/usr/local/cross/ia32/bin/i686-pc-elf-ld --eh-frame-hdr -m elf_i386 -shared -o x86_64-apple-darwin/stage2/lib/rustlib/i686-unknown-linux-gnu/lib/libstd-4e7c5e5c.so -L/Users/chris/Development/cmsd2/rust/x86_64-apple-darwin/stage2/lib/rustlib/i686-unknown-linux-gnu/lib -Li686-unknown-linux-gnu/rt -L. -L. -L/Users/chris/Development/cmsd2/rust/x86_64-apple-darwin/stage2/lib/rustlib/i686-unknown-linux-gnu/lib -L/Users/chris/Development/cmsd2/rust/.rust/lib/i686-unknown-linux-gnu -L/Users/chris/Development/cmsd2/rust/lib/i686-unknown-linux-gnu -L/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib -L/usr/lib --as-needed x86_64-apple-darwin/stage2/lib/rustlib/i686-unknown-linux-gnu/lib/std-4e7c5e5c.o --whole-archive -lmorestack --no-whole-archive x86_64-apple-darwin/stage2/lib/rustlib/i686-unknown-linux-gnu/lib/std-4e7c5e5c.metadata.o -O1 /Users/chris/Development/cmsd2/rust/x86_64-apple-darwin/stage2/lib/rustlib/i686-unknown-linux-gnu/lib/libcollections-4e7c5e5c.rlib /Users/chris/Development/cmsd2/rust/x86_64-apple-darwin/stage2/lib/rustlib/i686-unknown-linux-gnu/lib/librand-4e7c5e5c.rlib /Users/chris/Development/cmsd2/rust/x86_64-apple-darwin/stage2/lib/rustlib/i686-unknown-linux-gnu/lib/liballoc-4e7c5e5c.rlib /Users/chris/Development/cmsd2/rust/x86_64-apple-darwin/stage2/lib/rustlib/i686-unknown-linux-gnu/lib/libunicode-4e7c5e5c.rlib /Users/chris/Development/cmsd2/rust/x86_64-apple-darwin/stage2/lib/rustlib/i686-unknown-linux-gnu/lib/liblibc-4e7c5e5c.rlib /Users/chris/Development/cmsd2/rust/x86_64-apple-darwin/stage2/lib/rustlib/i686-unknown-linux-gnu/lib/libcore-4e7c5e5c.rlib --whole-archive -Bstatic -lrust_builtin -lbacktrace -lrustrt_native --no-whole-archive -Bdynamic -ldl -lpthread -lrt -lgcc_s -lpthread -lc -lm -lcompiler-rt crtendS.o crtn.o
i686-unknown-linux-gnu/rt/librust_builtin.a: member i686-unknown-linux-gnu/rt/librust_builtin.a(rust_builtin.o) in archive is not an object

weird. it's trying to produce a static library for the gnu platform but rust_builtin.o is a mach (osx) object file:

file i686-unknown-linux-gnu/rt/rust_builtin.o
i686-unknown-linux-gnu/rt/rust_builtin.o: Mach-O object i386

What on earth is supposed to happen?

[brson]

Oh, man. I didn't realize this was trying to cross from darwin to linux. I'm positive nobody has ever done that before, so certainly there will be issues.

[alexchandel]

@cmsd2 You get that error when clang invokes darwin's ld with arguments for linux's ld.

We need to invoke a ELF-compatible linker. Clang is infamous for completely ignoring its linker path argument, so I usually invoke clang with -c and link separately. This probably means configuring Rust with a separate linker path for i686-unknown-linux-gnu.

Alternatively, LLD can cross-link without any problems. However due to a bug in how it handles static libraries, LLD is not presently capable of statically linking Rust programs (the issue on LLVM), although it can dynamically link them. cc #9367 for linker issues

[cmsd2]

Right ok. In the short term I might have a go at replicating what the android port does with its platform-specific mk/cfg/ file. What I was really hoping for ultimately though was actually a bit different. I've written a kernel in C, which compiles using standard gcc tools. Developing on osx though, the target isn't sufficiently similar to the host, so that's why i started looking at rust as a cross compiler. A bigger problem though is that I don't yet have a libc hosted target environment. Gcc is fine with this because when building it as a cross compiler for e.g. i686-pc-elf, it knows that -elf is without a libc. Rust build doesn't have that option available, and tries building a complete runtime when all I currently need is #[no_std] support and the ability to compiler .rs to elf object files.

[ckkashyap]

I too am bitten by this problem. I am trying to build a xv6 translation to rust - https://github.com/ckkashyap/unix/ - looks like I'll have to hold off build on mac for now.

[ckkashyap]

For what it's worth I have a workaorund that works for me -

On my mac machine, do the following

1. Compile the rust file (that's supposed to emit the static library) using --target=x86_64-unknown-linux-gnu -L library_path (library_path is the location where I've copied libcompiler-rt.a and libmorestack.a from the linux rust build)
2. Unarchive the generated .a file (ar x ) and then use the .o files (only the necessary .o files) in the final linking

[cmsd2]

i tried just using the osx version early on but without having a libcompiler-rt.a to hand. i got stuck because i didn't have a core crate from which to import standard types etc.

no need to go as far as linking a static library though. just emit the object file you want.

for a mostly empty foo.rs:

#![no_std]
extern crate core;
using core::prelude::*;

this produces an elf object on my osx system using the osx build of rust:

rustc --target=i686-unknown-linux-gnu --emit=obj --crate-type=lib -L../rust-nightly-i686-unknown-linux-gnu/lib/rustlib/i686-unknown-linux-gnu/lib -o foo.o foo.rs

i made sure my osx rustc and target libraries were both from last night's nightlies too.

[alexchandel]

Couldn't morestack.S be rewritten in LLVM IR, like rust_try.ll?

[johalun]

I'm still getting this error.. No solution in sight?

[SirVer]

+1. seeing this error on 1b28ffa5216c845d1cef6b0cb3e5ac7db12025d0 while building a cross compiler from darwin to linux using ../configure --target=i686-unknown-linux-gnu && make -j3.

[alexcrichton]

This is unfortunately a known cross compilation triple that's just inherently quite difficult to target. Some of the bugs mentioned here have specifically been resolved, but in general this is never going to "just work" as it'll require a toolchain which can target 32-bit linux from OSX (which isn't available by default). If such a toolchain is installed, however, then most of our commands today should work just fine, but toolchains like this are pretty rare to come by.

As a result I'm going to close this for now, but we're always looking for more avenues to make cross compilation in Rust easier, so please feel free to file follow-up bugs!

[dongz9]

@alexcrichton I just found this thread of discussion. Just want to let you, there do exists toolchains targeting 32-bit linux from OS X, and I have a Homebrew formula for it.

https://github.com/dongz9/homebrew-devel

[alexcrichton]

@dongz9 oh nice! While probably rare to come across, good to know they exist :)

[dongz9]

@alexcrichton Is there any chance for rust to support this in the near future?

[alexcrichton]

So long as you've got the right linker locally Rust already supports this use case, it's up to you to make sure all the right libraries are in place.

[dongz9]

@alexcrichton How can I configure rust to use this toolchain?

[alexcrichton]

You'll need to make sure that your rustlib directory is populated with i686-unknown-linux-gnu libraries and then pass -C linker to the compiler to pass it the linux linker, not the OSX linker.

[dongz9]

I'm sorry but could you provide more details? Is this something I should do when calling ``configure'' before make?

[alexcrichton]

Unfortunately this kind of cross compilation is pretty unusual right now, so it's not really set up to "just work". It's unlikely that our configure script will be able to produce this kind compiler for this kind of cross compilation easily on OSX. In general you're mostly on your own in this territory right now. We certainly want to make this situation easier, but it will take some time and hasn't happened yet.

Specifically what I mean is that all Rust distributions have a rustlib directory with a bunch of directories inside of it, one per target triple the compiler has the standard library for. To cross compile from OSX to Linux you'll need to have target libraries for both architectures. This probably means that you'll download an OSX compiler, then you'll download a linux compiler, then you'll take the linux rustlib directory and merge it with the OSX rustlib directory. Then when you run rustc --target i686-unknown-linux-gnu the compiler will be able to find the standard library and such. Finally you'll need to pass -C linker=... to tell the compiler about the linux linker you have installed.

[yanns]

Just had the issue myself. As a workaround, I am using a docker container to build the executable:

docker pull clux/muslrust
docker run -v $PWD:/volume -w /volume -t clux/muslrust cargo build --release

(resource: https://github.com/clux/muslrust)