search

smaeul / portage-overlay

Extra ebuilds for Gentoo systems, mostly packages fixed to work with musl
18 stars 2 forks source link

support for std-thumbv7neon-unknown-linux-musleabihf #19

Closed stefson closed 4 years ago

stefson commented 5 years ago

hey there,

www-client/firefox is going to ask for this additional target when compiling with --enable-neon, from v68.0 on I believe. So, for a native compile, it would be armv7-rustc + armv7-std and in addition to that thumbv7neon-std

ping me if you're interested to bootstrap/test/hack something, ok? :-)

smaeul commented 5 years ago

Does it need both targets, or just the thumbv7neon target? Thankfully, the rust cross-toolchain situation is pretty good, so all it should take is some config.toml tweaking. I'll see what I can cook up.

stefson commented 5 years ago

yes, it needs both targets, but it already has one included. rustc+armv7-std from the ebuild, plus thumbv7neon-std to be added.

for tweaking the ebuild, it would only take this:

add thumbv7neon use flag; then treat it like the wasm useflag: 

+   if use arm && use thumbv7neon; then
+       rust_targets="${rust_targets},\"thumbv7neon-unknown-linux-musleabihf\""
+   fi
+   if use arm && use thumbv7neon; then
+       cat <<- EOF >> "${S}"/config.toml
+           [target.thumbv7neon-unknown-linux-musleabihf]
+           cc = "$(tc-getBUILD_CC)"
+           cxx = "$(tc-getBUILD_CXX)"
+           linker = "$(tc-getCC)"
+           ar = "$(tc-getAR)"
+       EOF
+   fi

But if it's not too much to ask, would you mind to push out a pre built std-lib? arm is really slow with this :/

stefson commented 5 years ago

I think the config.toml should be something like this:

cat config.toml

[llvm]
optimize = true
release-debuginfo = false
assertions = false
targets = "ARM"
experimental-targets = ""
link-shared = true
[build]
build = "armv7-unknown-linux-musleabihf"
host = ["armv7-unknown-linux-musleabihf"]
target = ["armv7-unknown-linux-musleabihf","thumbv7neon-unknown-linux-musleabihf"]
cargo = "/var/tmp/portage/dev-lang/rust-1.36.0/work/rust-stage0/bin/cargo"
rustc = "/var/tmp/portage/dev-lang/rust-1.36.0/work/rust-stage0/bin/rustc"
docs = false
submodules = false
python = "python3.6"
locked-deps = true
vendor = true
extended = true
tools = ["cargo",]
verbose = 2
[install]
prefix = "/usr"
libdir = "lib/rust-1.36.0"
docdir = "share/doc/rust-1.36.0"
mandir = "share/rust-1.36.0/man"
[rust]
optimize = true
debuginfo = false
debug-assertions = false
default-linker = "armv7a-unknown-linux-musleabihf-gcc"
channel = "stable"
rpath = false
lld = false
[target.armv7-unknown-linux-musleabihf]
cc = "armv7a-unknown-linux-musleabihf-gcc"
cxx = "armv7a-unknown-linux-musleabihf-g++"
linker = "armv7a-unknown-linux-musleabihf-gcc"
ar = "armv7a-unknown-linux-musleabihf-ar"
llvm-config = "/usr/lib/llvm/8/bin/llvm-config"
[target.thumbv7neon-unknown-linux-musleabihf]
cc = "armv7a-unknown-linux-musleabihf-gcc"
cxx = "armv7a-unknown-linux-musleabihf-g++"
linker = "armv7a-unknown-linux-musleabihf-gcc"
ar = "armv7a-unknown-linux-musleabihf-ar"
llvm-config = "/usr/lib/llvm/8/bin/llvm-config"

sadly I don't really understand how to inject the additional target into your ebuild, otherwise would've opened a pullrequest already.

stefson commented 5 years ago

I learned how to add foreign rustlibs to your ebuilds, using thumbv7neon-unknown-linux-musleabihf fails, as it hasn't been yet added to the source.

smaeul commented 5 years ago

Sorry, I guess I completely missed that there isn't actually a thumbv7neon-unknown-linux-musleabihf target spec in the upstream source. I thought you had been referring to the ebuild. I've added it in the latest-pushed ebuild version and sent the patch upstream (https://github.com/rust-lang/rust/pull/66103).

stefson commented 5 years ago

right now I get an error: 

error: linking with `armv7a-unknown-linux-musleabihf-gcc` failed: exit code: 1
  |
  = note: "armv7a-unknown-linux-musleabihf-gcc" "-Wl,--as-needed" "-Wl,-z,noexecstack" "-Wl,--eh-frame-hdr" "-L" "/var/tmp/portage/dev-lang/rust-1.38.0/work/rustc-1.38.0-src/build/x86_64-gentoo-linux-musl/stage2/lib/rustlib/thumbv7neon-unknown-linux-musleabihf/lib" "/var/tmp/portage/dev-lang/rust-1.38.0/work/rustc-1.38.0-src/build/x86_64-gentoo-linux-musl/stage2-std/thumbv7neon-unknown-linux-musleabihf/release/deps/std-89d19d7317c0a871.std.cjs1kmgl-cgu.0.rcgu.o" "/var/tmp/portage/dev-lang/rust-1.38.0/work/rustc-1.38.0-src/build/x86_64-gentoo-linux-musl/stage2-std/thumbv7neon-unknown-linux-musleabihf/release/deps/std-89d19d7317c0a871.std.cjs1kmgl-cgu.1.rcgu.o" "/var/tmp/portage/dev-lang/rust-1.38.0/work/rustc-1.38.0-src/build/x86_64-gentoo-linux-musl/stage2-std/thumbv7neon-unknown-linux-musleabihf/release/deps/std-89d19d7317c0a871.std.cjs1kmgl-cgu.10.rcgu.o" "/var/tmp/portage/dev-lang/rust-1.38.0/work/rustc-1.38.0-src/build/x86_64-gentoo-linux-musl/stage2-std/thumbv7neon-unknown-linux-musleabihf/release/deps/std-89d19d7317c0a871.std.cjs1kmgl-cgu.11.rcgu.o" "/var/tmp/portage/dev-lang/rust-1.38.0/work/rustc-1.38.0-src/build/x86_64-gentoo-linux-musl/stage2-std/thumbv7neon-unknown-linux-musleabihf/release/deps/std-89d19d7317c0a871.std.cjs1kmgl-cgu.12.rcgu.o" "/var/tmp/portage/dev-lang/rust-1.38.0/work/rustc-1.38.0-src/build/x86_64-gentoo-linux-musl/stage2-std/thumbv7neon-unknown-linux-musleabihf/release/deps/std-89d19d7317c0a871.std.cjs1kmgl-cgu.13.rcgu.o" "/var/tmp/portage/dev-lang/rust-1.38.0/work/rustc-1.38.0-src/build/x86_64-gentoo-linux-musl/stage2-std/thumbv7neon-unknown-linux-musleabihf/release/deps/std-89d19d7317c0a871.std.cjs1kmgl-cgu.14.rcgu.o" "/var/tmp/portage/dev-lang/rust-1.38.0/work/rustc-1.38.0-src/build/x86_64-gentoo-linux-musl/stage2-std/thumbv7neon-unknown-linux-musleabihf/release/deps/std-89d19d7317c0a871.std.cjs1kmgl-cgu.15.rcgu.o" "/var/tmp/portage/dev-lang/rust-1.38.0/work/rustc-1.38.0-src/build/x86_64-gentoo-linux-musl/stage2-std/thumbv7neon-unknown-linux-musleabihf/release/deps/std-89d19d7317c0a871.std.cjs1kmgl-cgu.2.rcgu.o" "/var/tmp/portage/dev-lang/rust-1.38.0/work/rustc-1.38.0-src/build/x86_64-gentoo-linux-musl/stage2-std/thumbv7neon-unknown-linux-musleabihf/release/deps/std-89d19d7317c0a871.std.cjs1kmgl-cgu.3.rcgu.o" "/var/tmp/portage/dev-lang/rust-1.38.0/work/rustc-1.38.0-src/build/x86_64-gentoo-linux-musl/stage2-std/thumbv7neon-unknown-linux-musleabihf/release/deps/std-89d19d7317c0a871.std.cjs1kmgl-cgu.4.rcgu.o" "/var/tmp/portage/dev-lang/rust-1.38.0/work/rustc-1.38.0-src/build/x86_64-gentoo-linux-musl/stage2-std/thumbv7neon-unknown-linux-musleabihf/release/deps/std-89d19d7317c0a871.std.cjs1kmgl-cgu.5.rcgu.o" "/var/tmp/portage/dev-lang/rust-1.38.0/work/rustc-1.38.0-src/build/x86_64-gentoo-linux-musl/stage2-std/thumbv7neon-unknown-linux-musleabihf/release/deps/std-89d19d7317c0a871.std.cjs1kmgl-cgu.6.rcgu.o" "/var/tmp/portage/dev-lang/rust-1.38.0/work/rustc-1.38.0-src/build/x86_64-gentoo-linux-musl/stage2-std/thumbv7neon-unknown-linux-musleabihf/release/deps/std-89d19d7317c0a871.std.cjs1kmgl-cgu.7.rcgu.o" "/var/tmp/portage/dev-lang/rust-1.38.0/work/rustc-1.38.0-src/build/x86_64-gentoo-linux-musl/stage2-std/thumbv7neon-unknown-linux-musleabihf/release/deps/std-89d19d7317c0a871.std.cjs1kmgl-cgu.8.rcgu.o" "/var/tmp/portage/dev-lang/rust-1.38.0/work/rustc-1.38.0-src/build/x86_64-gentoo-linux-musl/stage2-std/thumbv7neon-unknown-linux-musleabihf/release/deps/std-89d19d7317c0a871.std.cjs1kmgl-cgu.9.rcgu.o" "-o" "/var/tmp/portage/dev-lang/rust-1.38.0/work/rustc-1.38.0-src/build/x86_64-gentoo-linux-musl/stage2-std/thumbv7neon-unknown-linux-musleabihf/release/deps/libstd-89d19d7317c0a871.so" "-Wl,--version-script=/var/tmp/portage/dev-lang/rust-1.38.0/temp/rustcoQcUJR/list" "/var/tmp/portage/dev-lang/rust-1.38.0/work/rustc-1.38.0-src/build/x86_64-gentoo-linux-musl/stage2-std/thumbv7neon-unknown-linux-musleabihf/release/deps/std-89d19d7317c0a871.3j8mozjrmu7kuhwg.rcgu.o" "/var/tmp/portage/dev-lang/rust-1.38.0/work/rustc-1.38.0-src/build/x86_64-gentoo-linux-musl/stage2-std/thumbv7neon-unknown-linux-musleabihf/release/deps/std-89d19d7317c0a871.2xqh42makntdvv5o.rcgu.o" "-Wl,-zrelro" "-Wl,-znow" "-Wl,-O1" "-nodefaultlibs" "-L" "/var/tmp/portage/dev-lang/rust-1.38.0/work/rustc-1.38.0-src/build/x86_64-gentoo-linux-musl/stage2-std/thumbv7neon-unknown-linux-musleabihf/release/deps" "-L" "/var/tmp/portage/dev-lang/rust-1.38.0/work/rustc-1.38.0-src/build/x86_64-gentoo-linux-musl/stage2-std/release/deps" "-L" "/usr/lib/llvm/9/lib" "-L" "/var/tmp/portage/dev-lang/rust-1.38.0/work/rustc-1.38.0-src/build/x86_64-gentoo-linux-musl/stage2-std/thumbv7neon-unknown-linux-musleabihf/release/build/compiler_builtins-9e4aa193cad9df89/out" "-L" "/var/tmp/portage/dev-lang/rust-1.38.0/work/rustc-1.38.0-src/build/x86_64-gentoo-linux-musl/stage2-std/thumbv7neon-unknown-linux-musleabihf/release/build/backtrace-sys-670a23a2a0749855/out" "-L" "/var/tmp/portage/dev-lang/rust-1.38.0/work/rustc-1.38.0-src/build/x86_64-gentoo-linux-musl/stage2/lib/rustlib/thumbv7neon-unknown-linux-musleabihf/lib" "-Wl,-Bstatic" "-Wl,--whole-archive" "/var/tmp/portage/dev-lang/rust-1.38.0/temp/rustcoQcUJR/libpanic_unwind-ee582e60162abd60.rlib" "-Wl,--no-whole-archive" "-Wl,--whole-archive" "/var/tmp/portage/dev-lang/rust-1.38.0/temp/rustcoQcUJR/libbacktrace-cebaa12c33f409a3.rlib" "-Wl,--no-whole-archive" "-Wl,--whole-archive" "/var/tmp/portage/dev-lang/rust-1.38.0/temp/rustcoQcUJR/libbacktrace_sys-1e1f43b69f641dba.rlib" "-Wl,--no-whole-archive" "-Wl,--whole-archive" "/var/tmp/portage/dev-lang/rust-1.38.0/temp/rustcoQcUJR/librustc_demangle-842243603f9d5d4c.rlib" "-Wl,--no-whole-archive" "-Wl,--whole-archive" "/var/tmp/portage/dev-lang/rust-1.38.0/temp/rustcoQcUJR/libhashbrown-857346ec8204be84.rlib" "-Wl,--no-whole-archive" "-Wl,--whole-archive" "/var/tmp/portage/dev-lang/rust-1.38.0/temp/rustcoQcUJR/librustc_std_workspace_alloc-c1a180b7f3ed5f4a.rlib" "-Wl,--no-whole-archive" "-Wl,--whole-archive" "/var/tmp/portage/dev-lang/rust-1.38.0/temp/rustcoQcUJR/libunwind-fdf3372abf9667f3.rlib" "-Wl,--no-whole-archive" "-Wl,--whole-archive" "/var/tmp/portage/dev-lang/rust-1.38.0/temp/rustcoQcUJR/libcfg_if-45c0958490d99c81.rlib" "-Wl,--no-whole-archive" "-Wl,--whole-archive" "/var/tmp/portage/dev-lang/rust-1.38.0/temp/rustcoQcUJR/liblibc-5c30a929cb0a9db0.rlib" "-Wl,--no-whole-archive" "-Wl,--whole-archive" "/var/tmp/portage/dev-lang/rust-1.38.0/temp/rustcoQcUJR/liballoc-fdfc05544a56e037.rlib" "-Wl,--no-whole-archive" "-Wl,--whole-archive" "/var/tmp/portage/dev-lang/rust-1.38.0/temp/rustcoQcUJR/librustc_std_workspace_core-fad22619bcc34b29.rlib" "-Wl,--no-whole-archive" "-Wl,--whole-archive" "/var/tmp/portage/dev-lang/rust-1.38.0/temp/rustcoQcUJR/libcore-7ea93c8a476993c3.rlib" "-Wl,--no-whole-archive" "/var/tmp/portage/dev-lang/rust-1.38.0/temp/rustcoQcUJR/libcompiler_builtins-780e590fa9eb7c23.rlib" "-Wl,-Bdynamic" "-lgcc_s" "-lc" "-shared" "-lssp_nonshared"
  = note: /usr/libexec/gcc/armv7a-unknown-linux-musleabihf/ld: cannot find -lssp_nonshared
          collect2: error: ld returned 1 exit status

error: aborting due to previous error

error: Could not compile `std`.

will now cross emerge libssp_nonshared-0 and try again

stefson commented 5 years ago

with libssp_nonshared-0 available on the cross rootfs, the thumbv7neon-unknown-linux-musleabihf is standing at the ready: 

/usr/lib/rustlib/thumbv7neon-unknown-linux-musleabihf
/usr/lib/rustlib/thumbv7neon-unknown-linux-musleabihf/lib
/usr/lib/rustlib/thumbv7neon-unknown-linux-musleabihf/lib/liballoc-fdfc05544a56e037.rlib
/usr/lib/rustlib/thumbv7neon-unknown-linux-musleabihf/lib/libbacktrace-cebaa12c33f409a3.rlib
/usr/lib/rustlib/thumbv7neon-unknown-linux-musleabihf/lib/libbacktrace_sys-1e1f43b69f641dba.rlib
/usr/lib/rustlib/thumbv7neon-unknown-linux-musleabihf/lib/libcfg_if-45c0958490d99c81.rlib
/usr/lib/rustlib/thumbv7neon-unknown-linux-musleabihf/lib/libcompiler_builtins-780e590fa9eb7c23.rlib
/usr/lib/rustlib/thumbv7neon-unknown-linux-musleabihf/lib/libcore-7ea93c8a476993c3.rlib
/usr/lib/rustlib/thumbv7neon-unknown-linux-musleabihf/lib/libgetopts-fbf3549a6c83d2db.rlib
/usr/lib/rustlib/thumbv7neon-unknown-linux-musleabihf/lib/libhashbrown-857346ec8204be84.rlib
/usr/lib/rustlib/thumbv7neon-unknown-linux-musleabihf/lib/liblibc-5c30a929cb0a9db0.rlib
/usr/lib/rustlib/thumbv7neon-unknown-linux-musleabihf/lib/libpanic_abort-8fe399f8bab68844.rlib
/usr/lib/rustlib/thumbv7neon-unknown-linux-musleabihf/lib/libpanic_unwind-ee582e60162abd60.rlib
/usr/lib/rustlib/thumbv7neon-unknown-linux-musleabihf/lib/libproc_macro-b72a5dc7ba79aa4c.rlib
/usr/lib/rustlib/thumbv7neon-unknown-linux-musleabihf/lib/librustc_demangle-842243603f9d5d4c.rlib
/usr/lib/rustlib/thumbv7neon-unknown-linux-musleabihf/lib/librustc_std_workspace_alloc-c1a180b7f3ed5f4a.rlib
/usr/lib/rustlib/thumbv7neon-unknown-linux-musleabihf/lib/librustc_std_workspace_core-fad22619bcc34b29.rlib
/usr/lib/rustlib/thumbv7neon-unknown-linux-musleabihf/lib/libstd-89d19d7317c0a871.rlib
/usr/lib/rustlib/thumbv7neon-unknown-linux-musleabihf/lib/libstd-89d19d7317c0a871.so
/usr/lib/rustlib/thumbv7neon-unknown-linux-musleabihf/lib/libterm-fd943c890ee53557.rlib
/usr/lib/rustlib/thumbv7neon-unknown-linux-musleabihf/lib/libterm-fd943c890ee53557.so
/usr/lib/rustlib/thumbv7neon-unknown-linux-musleabihf/lib/libtest-9e23016c07f04bb4.rlib
/usr/lib/rustlib/thumbv7neon-unknown-linux-musleabihf/lib/libtest-9e23016c07f04bb4.so
/usr/lib/rustlib/thumbv7neon-unknown-linux-musleabihf/lib/libunicode_width-cef9f1595e7984a6.rlib
/usr/lib/rustlib/thumbv7neon-unknown-linux-musleabihf/lib/libunwind-fdf3372abf9667f3.rlib

not sure how to do a runtime testing, can you please give me directions with that?

smaeul commented 5 years ago

You should be able to do a hello world for each target installed. Here's an example on my box; it should be the same on arm, just with different triples:

$ cargo new --bin hello
     Created binary (application) `hello` package
$ cd hello
$ cargo build --target powerpc-unknown-linux-musl
   Compiling hello v0.1.0 (/tmp/hello)
    Finished dev [unoptimized + debuginfo] target(s) in 2.40s
$ file target/powerpc-unknown-linux-musl/debug/hello
target/powerpc-unknown-linux-musl/debug/hello: ELF 32-bit MSB executable, PowerPC or cisco 4500, version 1 (SYSV), statically linked, with debug_info, not stripped
$ target/powerpc-unknown-linux-musl/debug/hello
Hello, world!
$ cargo build --target powerpc64-unknown-linux-musl
   Compiling hello v0.1.0 (/tmp/hello)
    Finished dev [unoptimized + debuginfo] target(s) in 1.57s
$ file target/powerpc64-unknown-linux-musl/debug/hello
target/powerpc64-unknown-linux-musl/debug/hello: ELF 64-bit MSB executable, 64-bit PowerPC or cisco 7500, version 1 (SYSV), statically linked, with debug_info, not stripped
$ target/powerpc64-unknown-linux-musl/debug/hello
Hello, world!
stefson commented 4 years ago

hmm, my guess is that this works a bit as with firefox, as in: cargo build --target armv7a-unknown-linux-musleabihf, plus a command to force the use of neon useflags.

Do you think you can push out a rust-std-thumbv7neon-unknown-linux-musleabihf for rust-1.38.0? The rpi2 I own hasn't got enough ram for compiling native.

smaeul commented 4 years ago

Yes, I'll add thumbv7neon-unknown-linux-musleabihf (and the other official triples) to the cross-build I do to generate tarballs. Now that 1.39.0 is out, that should be pretty soon.

stefson commented 4 years ago

great, just ping me when they're online.

smaeul commented 4 years ago

https://portage.smaeul.xyz/distfiles/rust-std-1.39.0-thumbv7neon-unknown-linux-musleabihf.tar.xz is available to go along with the rust 1.39.0 ebuild.

stefson commented 4 years ago

I grabbed the fresh armv7 stage, and installed the thumbv7-neon std alongside, and it turns out the hello world is statically linked only: 

cargo new --bin hello & cd hello
cargo build --target thumbv7neon-unknown-linux-musleabihf
cd target/thumbv7neon-unknown-linux-musleabihf/debug/
hello: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), statically linked, with debug_info, not stripped

not sure if this is what you want?

stefson commented 4 years ago

regarding firefox, the detection doesn't seem to work properly: 

 1:22.99 checking for rustc... /usr/bin/rustc
 1:23.00 checking for cargo... /usr/bin/cargo
 1:27.40 checking rustc version... 1.39.0
 1:27.55 checking cargo version... 1.39.0
 1:32.14 checking for rust target triplet... thumbv7neon-unknown-linux-musleabihf
 1:32.69 checking for rust host triplet... thumbv7neon-unknown-linux-musleabihf
 1:32.70 ERROR: The rust compiler host (armv7a-unknown-linux-musleabihf) is not suitable for the configure host (armv7a-unknown-linux-musleabihf/thumbv7neon-unknown-linux-musleabihf).
 1:33.01 *** Fix above errors and then restart with\
 1:33.02                "./mach build"
 1:33.02 gmake: *** [client.mk:115: configure] Error 1

but I believe this is expected, as it's broken with glibc too, in the same way.

smaeul commented 4 years ago

Yes, it's expected that it's statically linked. Changing that behavior is why the -gentoo- targets were created.

Since this appears to be working on the rust side, I'm going to close this issue. The firefox issue will continue to be tracked in #16.