Cross-compile rust libs for ARMv5 target

[joerg-krause]

I want to cross-compile the rust libs, eg std, for an ARMv5 target. For now, I am able to cross-compile libcore with an armv5te-none-eabi.json target and cross-compile a no-std program without the libc crate by linking with a target libc from musl.

However, I am not sure how to cross-compile the std library for the ARMv5 target as the included target arm-unknown-linux-gnueabi builds for ARMv6.

[MagaTailor]

What if you modified the target to use v5? Together with arm-unknown-linux-gnueabi.mk.

[joerg-krause]

My first attempt was to create a new target armv5te-unknown-linux-gnueabi.mk, but this fails because my host (Arch Linux) rust compiler backend does not include such a target.

My second attempt was to replace the existing arm-unknown-linux-gnueabi.mk target with:

# arm-unknown-linux-gnueabi configuration
CROSS_PREFIX_arm-unknown-linux-gnueabi=arm-linux-gnueabi-
CC_arm-unknown-linux-gnueabi=gcc
CXX_arm-unknown-linux-gnueabi=g++
CPP_arm-unknown-linux-gnueabi=gcc -E
AR_arm-unknown-linux-gnueabi=ar
CFG_LIB_NAME_arm-unknown-linux-gnueabi=lib$(1).so
CFG_STATIC_LIB_NAME_arm-unknown-linux-gnueabi=lib$(1).a
CFG_LIB_GLOB_arm-unknown-linux-gnueabi=lib$(1)-*.so
CFG_LIB_DSYM_GLOB_arm-unknown-linux-gnueabi=lib$(1)-*.dylib.dSYM
CFG_JEMALLOC_CFLAGS_arm-unknown-linux-gnueabi := -D__arm__ -msoft-float $(CFLAGS)
CFG_GCCISH_CFLAGS_arm-unknown-linux-gnueabi := -Wall -g -fPIC -D__arm__ -msoft-float $(CFLAGS)
CFG_GCCISH_CXXFLAGS_arm-unknown-linux-gnueabi := -fno-rtti $(CXXFLAGS)
CFG_GCCISH_LINK_FLAGS_arm-unknown-linux-gnueabi := -shared -fPIC -g -msoft-float
CFG_GCCISH_DEF_FLAG_arm-unknown-linux-gnueabi := -Wl,--export-dynamic,--dynamic-list=
CFG_LLC_FLAGS_arm-unknown-linux-gnueabi :=
CFG_INSTALL_NAME_arm-unknown-linux-gnueabi =
CFG_EXE_SUFFIX_arm-unknown-linux-gnueabi :=
CFG_WINDOWSY_arm-unknown-linux-gnueabi :=
CFG_UNIXY_arm-unknown-linux-gnueabi := 1
CFG_LDPATH_arm-unknown-linux-gnueabi :=
CFG_RUN_arm-unknown-linux-gnueabi=$(2)
CFG_RUN_TARG_arm-unknown-linux-gnueabi=$(call CFG_RUN_arm-unknown-linux-gnueabi,,$(2))
RUSTC_FLAGS_arm-unknown-linux-gnueabi :=
RUSTC_CROSS_FLAGS_arm-unknown-linux-gnueabi := -C target-cpu=arm926ej-s -C target-feature="+v5te" -C soft-float
CFG_GNU_TRIPLE_arm-unknown-linux-gnueabi := arm-unknown-linux-gnueabi

... and the librustc target backend arm_unknown_linux_gnueabi.rs with:

use target::Target;

pub fn target() -> Target {
    Target {
        llvm_target: "arm-unknown-linux-gnueabi".to_string(),
        target_endian: "little".to_string(),
        target_pointer_width: "32".to_string(),
        arch: "arm".to_string(),
        target_os: "linux".to_string(),
        target_env: "gnueabi".to_string(),
        target_vendor: "unknown".to_string(),
        options: super::linux_base::opts()
    }
}

Cross-compiling rust:

$ ./configure --target=arm-unknown-linux-gnueabi --prefix=$HOME/rust-cross --disable-jemalloc
$ make
$ make install
$ LD_LIBRARY_PATH=$HOME/rust-cross/lib $HOME/rust-cross/bin/rustc --version
rustc 1.7.0-dev (4c111e990 2016-01-05)

I need to configure Rust with --disable-jemalloc otherwise building hello world will fail with:

note: /home/joerg/rust-cross/lib/rustlib/arm-unknown-linux-gnueabi/lib/liballoc_jemalloc-ca1c970e.rlib(jemalloc.pic.o): In function `secure_getenv':
/home/joerg/git/rust/src/jemalloc/src/jemalloc.c:734: undefined reference to `issetugid'

As building rust with --disable-jemalloc is currently broken (https://github.com/rust-lang/rust/issues/30592) I had to patch current HEAD revision (https://github.com/rust-lang/rust/commit/dbacacda8ac5dba72afd57b971c71a71242e7f67) with https://github.com/simartin/rust/commit/4c68110c68e520898a37853ae18e15b7435d401f and https://github.com/simartin/rust/commit/4c111e99069f88eef4bf21c3cafa1fc87ae6c460.

Cross-compile hello:

$ cat hello.rs 
fn main() {
    println!("Hello, world!");
}
$ LD_LIBRARY_PATH=$HOME/rust-cross/lib $HOME/rust-cross/bin/rustc --target=arm-unknown-linux-gnueabi -C linker=/usr/bin/arm-linux-gcc hello.rs
$ file hello 
hello: ELF 32-bit LSB shared object, ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux.so.3, for GNU/Linux 2.6.32, not stripped
$ size hello 
   text    data     bss     dec     hex filename
 115129    2452      68  117649   1cb91 hello

Running on target:

$ ./hello 
Hello, world!

Finally, it works!!!

I am very new to Rust and I will be happy about any comments or suggestions. Furthermore, I would be glad if we can have a ARMv5(te) target with soft float supported by Rust.

My next attempt will be to cross-compile using the musl library (instead of glibc).

[MagaTailor]

What host are you building the cross-compiler on? And btw, it's no longer necessary to set LD_LIBRARY_PATH I believe.

[joerg-krause]

Host is Arch Linux.

[MagaTailor]

I meant host architecture.

[joerg-krause]

Host architecture is x86_64.

[MagaTailor]

Alright, if you continue experimenting with custom targets, remember to add them to mod.rs before trying to bootstrap the compiler.

[joerg-krause]

I followed the explanations from https://doc.rust-lang.org/nightly/book/advanced-linking.html#linux to build a musl-enabled Rust. However, using my arm-unknown-linux-musl target (https://gist.github.com/joerg-krause/7a7351b6e4ae7267a0ae) fails with some mismatched types errors, like:

[..]
rustc: x86_64-unknown-linux-gnu/stage2/lib/rustlib/arm-unknown-linux-musl/lib/libstd
rustc: x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd
warning: dropping unsupported crate type `dylib` for target `arm-unknown-linux-musl`
../src/libstd/dynamic_lib.rs:111:37: 111:56 error: mismatched types:
 expected `*const i8`,
    found `*const u8`
(expected i8,
    found u8) [E0308]
[..]

Any idea what's wrong here?

[MagaTailor]

Ah, yes - I'm an expert on that ;)

https://github.com/rust-lang/rust/issues/29867

[joerg-krause]

Ouch! Does it mean I have to fix the std library?

[joerg-krause]

Okay, I guess https://github.com/rust-lang-nursery/libc/blob/master/src/unix/notbsd/linux/musl.rs#L9 is not true for ARM devices. The ARM architecture defines char as unsigned, wheras x86(_64) defines it as signed.

[MagaTailor]

You'll probably have to add some casts if you want to compile it from this machine - is it just this one instance or are there more?

[joerg-krause]

A lot more.

I'll try to patch libc by editing https://github.com/rust-lang-nursery/libc/blob/master/src/unix/notbsd/linux/musl.rs#L9:

#[cfg(target_arch = "arm")]
pub type c_char = u8;
#[cfg(not(target_arch = "arm"))]
pub type c_char = i8;

Unfortunatly, building rust takes really long ...

[joerg-krause]

After fixing libc, only one error is left:

rustc: x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd
rustc: x86_64-unknown-linux-gnu/stage2/lib/rustlib/arm-unknown-linux-musl/lib/libstd
warning: dropping unsupported crate type `dylib` for target `arm-unknown-linux-musl`
../src/libstd/sys/unix/fs.rs:188:9: 188:25 error: mismatched types:
 expected `u32`,
    found `u64`
(expected u32,
    found u64) [E0308]
../src/libstd/sys/unix/fs.rs:188         self.entry.d_ino
                                         ^~~~~~~~~~~~~~~~
../src/libstd/sys/unix/fs.rs:188:9: 188:25 help: run `rustc --explain E0308` to see a detailed explanation

[joerg-krause]

I got the last error fixed by appling this patch: https://github.com/brson/rust/commit/cfdce22799f5ef4bc7b858e70bb47236dc8606a1 (ino_t is u64 for musl).

[MagaTailor]

Hey, if you ever get bored you could attempt a bigger task of cross-bootstrapping a rust compiler for OpenPandora (sof-float, Cortex-A8).

You'd be the first to do it and probably become a hero in their forum too ;)

[joerg-krause]

Unfortunatly, I do not have an OpenPandora to play with...

What is the problem with this target? Because it's soft-float?

[MagaTailor]

The problem's the fact cross-bootstrapping is not the easiest thing at the moment and there's no stage0 snapshot available. Maybe you could just try building an OpenPandora x86_64 host cross-compiler for those guys seeing as you already have a working soft-float gcc toolchain?

[joerg-krause]

Looks like OpenPandora uses the NEON FPU with a softfp ABI. The question is which libc (glibc? version?) and which instruction set (ARM or Thumb2?) does it uses?

I propably could build a cross toolchain for Cortex-A8 with the corresponding setting and look if rust bootstraps. Unfortunatly, compiling rust is very time consuming...

[MagaTailor]

If I remember correctly from someone's request it has glibc 2.9. CPU probably supports both modes but I'm not sure.

Yes, it takes long so don't bother. If you cared to create a modified target+mk gist it should be enough for someone from pandora to take over.

[joerg-krause]

This should be feasible. Adding a target armv7-unknown-linux-gnueabi with softfp ABI and use the default toolchain from pandora.

The problem with the arm-unknown-linux-gnueabi target from current HEAD is, that it is NOT soft-float despite of gnueabi. It really shouldn't use the mfpu=vfp compiler option.

[MagaTailor]

Yeah, I noticed that myself a few days ago thanks to your earlier activity and updated my native bootstrapping recipe (which wasn't supposed to be exclusively about pandora).

You should definitely create a PR fixing this!

EDIT: And BTW the armv7-unknown-linux-gnueabi target you suggested is definitely not the way to go if you're going to use cargo as it's not an official target, that is going to be missing from Cargo.toml of certain crates resulting in build errors.

[joerg-krause]

I was annoyed by rebuilding Rust all the time I changed something. Luckily, I found a nice gist from Kevin Mehall: https://gist.github.com/kevinmehall/16e8b3ea7266b048369d.

I modified the script to my needs and created a repository for it: https://github.com/joerg-krause/rust-cross-libs.

Hopefully, there will be a better way to cross-compile the Rust libraries for custom targets without hacking and cross-bootstrapping Rust!

[MagaTailor]

Very nice! My comment about nonsensical testing procedures was not related though.

[sanxiyn]

Hi, everyone. What is the status of this issue?

[joerg-krause]

Note, that since commit https://github.com/rust-lang/rust/commit/8016dc39aa7fa1b796cf8e3dd7cd93c65c53d7a8 there issupport now for ARMv5TE with a glibc toolchain.

However, as I am not want to run the rust compiler or cargo on my ARMv5 target, I simply need the libraries so that the rust programs can link against them. I build my own tool rust-cross-libs to allow building the libs with a musl based toolchain. However, this should also work with a glibc based toolchain, although I did not tested it recently.

[Mark-Simulacrum]

I'm going to close this; I can't seem to see anything inherently broken here and according to https://forge.rust-lang.org/platform-support.html the ARMv5TE target is builtin to the compiler, if not tested or built per commit.