gentoo / musl

[MIRROR] musl development overlay
https://gitweb.gentoo.org/proj/musl.git
99 stars 59 forks source link

dev-lang/rust-bin: initial commit for armv7a #366

Closed stefson closed 1 year ago

stefson commented 3 years ago

this version of rust-bin has been tested with cbindgen-0.15, librsvg-2.48.x, spidermonkey-78 and firefox-esr-78

only problem I found is that firefox needs LD_LIBRARY_PATH pointing to it's own libs, that's due to lazy binding problems during linking: https://bugs.gentoo.org/531846#c36

if CI comes back without issues, then this is good to go

stefson commented 3 years ago

it doesn't compile >=cbindgen-0.17 anymore for me on armv7, so package is outdated as of now

anarchpenguin commented 3 years ago

Steffen, what is the status of this work? Are you still working on it or is it ready to be merged to the overlay?

stefson commented 3 years ago

well, I can't update it beyond 1.48.0 at this point, since smauel hasn't pushed out a stage of 1.51.0 yet.

1.44.1 should not go to the overlay, since its old llvm-9 based. what do you think is best? wait for new 1.51.0 stage, upgrade to llvm-10 based 1.46.0 or go for the llvm-11 based 1.48.0?

xmhd commented 3 years ago

well, I can't update it beyond 1.48.0 at this point, since smauel hasn't pushed out a stage of 1.51.0 yet.

1.44.1 should not go to the overlay, since its old llvm-9 based. what do you think is best? wait for new 1.51.0 stage, upgrade to llvm-10 based 1.46.0 or go for the llvm-11 based 1.48.0?

new rust vers should 'just work' once this lands https://github.com/rust-lang/compiler-team/issues/422 and then smaeul shouldn't need to maintain rust.

stefson commented 2 years ago

I finally got a grip on how this patchset works and backported it.

dev-lang/rust-1.49.0 is available at smaeul, and both 1.50.0 and 1.51.0 will be added in the next days.

it seems not much progress has been made at upstream

(for amd64 and arm64, in tree ebuilds work)

ernsteiswuerfel commented 2 years ago

I finally got a grip on how this patchset works and backported it.

dev-lang/rust-1.49.0 is available at smaeul, and both 1.50.0 and 1.51.0 will be added in the next days.

I appreciate your efforts very much! Though I did not succeed in building your 1.49.0 on ppc. If you like, check out the build.log: rust-1.49.0:20220301-023721.log

I'll certainly try your next releases on ::smaeul!

stefson commented 2 years ago

please provide emerge --info and emerge -pv =rust-1.49.0

ernsteiswuerfel commented 2 years ago

please provide emerge --info and emerge -pv =rust-1.49.0

 #  emerge --info
Portage 3.0.30 (python 3.10.2-final-0, default/linux/ppc/17.0/musl, gcc-11.2.0, musl-1.2.2-r7, 5.16.11-gentoo-P9 ppc)
=================================================================
System uname: Linux-5.16.11-gentoo-P9-ppc-POWER9,_altivec_supported-with-libc
KiB Mem:    65807424 total,  57668316 free
KiB Swap:   16777212 total,  16777212 free
Timestamp of repository gentoo: Tue, 01 Mar 2022 10:00:01 +0000
Head commit of repository gentoo: 83322d2a2edb18c54bd234d8ab1c5e222769164c
Timestamp of repository musl: Sun, 27 Feb 2022 01:34:53 +0000
Head commit of repository musl: b78395fcb2f528ad58b814fcb6ab31e52417ab05

Timestamp of repository smaeul: Sun, 27 Feb 2022 21:36:38 +0000
Head commit of repository smaeul: fdac991a5e53fe77c0801c9045c850ab918011d3

sh bash 5.1_p16
ld GNU ld (Gentoo 2.37_p1 p2) 2.37
distcc 3.4 powerpc-gentoo-linux-musl [disabled]
app-misc/pax-utils:        1.3.3::gentoo
app-shells/bash:           5.1_p16::gentoo
dev-lang/perl:             5.34.0-r6::gentoo
dev-lang/python:           3.9.10-r1::gentoo, 3.10.2-r1::gentoo
dev-lang/rust:             1.48.0::smaeul
dev-util/cmake:            3.22.2::gentoo
dev-util/meson:            0.60.3::gentoo
sys-apps/baselayout:       2.7-r3::gentoo
sys-apps/openrc:           0.44.10::gentoo
sys-apps/sandbox:          2.25::gentoo
sys-devel/autoconf:        2.71-r1::gentoo
sys-devel/automake:        1.16.4::gentoo
sys-devel/binutils:        2.37_p1-r2::gentoo
sys-devel/binutils-config: 5.4::gentoo
sys-devel/gcc:             11.2.0::gentoo
sys-devel/gcc-config:      2.5-r1::gentoo
sys-devel/libtool:         2.4.6-r6::gentoo
sys-devel/llvm:            11.1.0::gentoo, 13.0.1::gentoo
sys-devel/make:            4.3::gentoo
sys-kernel/linux-headers:  5.15-r3::gentoo (virtual/os-headers)
sys-libs/musl:             1.2.2-r7::gentoo
Repositories:

gentoo
    location: /var/db/repos/gentoo
    sync-type: rsync
    sync-uri: rsync://yea/gentoo-portage
    priority: -1000
    sync-rsync-verify-metamanifest: no
    sync-rsync-extra-opts: 
    sync-rsync-verify-jobs: 2
    sync-rsync-verify-max-age: 24

localrepo
    location: /var/db/repos/localrepo
    masters: gentoo

musl
    location: /var/db/repos/musl
    sync-type: git
    sync-uri: https://github.com/gentoo-mirror/musl.git
    masters: gentoo

smaeul
    location: /var/db/repos/smaeul
    sync-type: git
    sync-uri: https://github.com/gentoo-mirror/smaeul.git
    masters: gentoo

ACCEPT_KEYWORDS="ppc"
ACCEPT_LICENSE="*"
CBUILD="powerpc-gentoo-linux-musl"
CFLAGS="-Os -mcpu=7450 -mtune=7450 -pipe"
CHOST="powerpc-gentoo-linux-musl"
CONFIG_PROTECT="/etc /usr/share/gnupg/qualified.txt"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo"
CXXFLAGS="-Os -mcpu=7450 -mtune=7450 -pipe"
DISTDIR="/var/cache/distfiles"
ENV_UNSET="CARGO_HOME DBUS_SESSION_BUS_ADDRESS DISPLAY GOBIN GOPATH PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR"
FCFLAGS="-Os -mcpu=7450 -mtune=7450 -pipe"
FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs buildpkg-live clean-logs config-protect-if-modified distlocks ebuild-locks fixlafiles ipc-sandbox merge-sync network-sandbox news parallel-fetch pid-sandbox preserve-libs protect-owned qa-unresolved-soname-deps sandbox sfperms split-log strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr"
FFLAGS="-Os -mcpu=7450 -mtune=7450 -pipe"
GENTOO_MIRRORS="ftp://ftp.gwdg.de/pub/linux/gentoo ftp://ftp.fi.muni.cz/pub/linux/gentoo"
INSTALL_MASK="charset.alias /usr/share/locale/locale.alias"
LANG="C.UTF8"
LC_ALL="C"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
LINGUAS="de en de_DE"
MAKEOPTS="-j16 -l18"
PKGDIR="/var/cache/binpkgs"
PORTAGE_BZIP2_COMMAND="pbzip2"
PORTAGE_COMPRESS="zstd"
PORTAGE_COMPRESS_FLAGS="-6"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git"
PORTAGE_TMPDIR="/var/tmp"
SHELL="/bin/bash"
USE="X a52 aac acl alsa big-endian bluetooth brotli bzip2 cairo caja cdda cli colord crypt cryptsetup css cups dav1d dbus djvu dri elogind enca exif flac fortran gtk gtk3 gui iconv icu introspection ipv6 jpeg lcms libglvnd libnotify lm-sensors mmap mng mp3 mtp ncurses networkmanager nls nptl nss ogg opengl openmp opus pam pcre pdf png policykit ppc pulseaudio qt5 raw readline seccomp smp spell split-usr ssl svg system-icu system-sqlite threads tiff truetype udev udisks unicode upower vorbis vpx webp widgets xattr xml xmp xxhash zlib zstd" ABI_PPC="32" ADA_TARGET="gnat_2020" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="karbon sheets words" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_PPC="altivec" ELIBC="musl" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock greis isync itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 timing tsip tripmate tnt ublox ubx" GRUB_PLATFORMS="ieee1275" INPUT_DEVICES="libinput" KERNEL="linux" L10N="de en" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LLVM_TARGETS="AMDGPU" LUA_SINGLE_TARGET="lua5-3" LUA_TARGETS="lua5-1" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php7-4 php8-0" POSTGRES_TARGETS="postgres12 postgres13" PYTHON_SINGLE_TARGET="python3_10" PYTHON_TARGETS="python3_10" RUBY_TARGETS="ruby27" SANE_BACKENDS="genesys" USERLAND="GNU" VIDEO_CARDS="fbdev r300 radeon" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq proto steal rawnat logmark ipmark dhcpmac delude chaos account"
Unset:  ADDR2LINE, AR, ARFLAGS, AS, ASFLAGS, CC, CCLD, CONFIG_SHELL, CPP, CPPFLAGS, CTARGET, CXX, CXXFILT, ELFEDIT, EMERGE_DEFAULT_OPTS, EXTRA_ECONF, F77FLAGS, FC, GCOV, GPROF, LD, LEX, LFLAGS, LIBTOOL, MAKE, MAKEFLAGS, NM, OBJCOPY, OBJDUMP, PORTAGE_BINHOST, PORTAGE_BUNZIP2_COMMAND, PORTAGE_RSYNC_EXTRA_OPTS, RANLIB, READELF, RUSTFLAGS, SIZE, STRINGS, STRIP, YACC, YFLAGS

and

 # emerge -p1v rust virtual/rust
!!! It seems /run is not mounted. Process management may malfunction.

These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild     U ~] dev-lang/rust-1.49.0:stable/1.49::smaeul [1.48.0:stable/1.48::smaeul] USE="system-llvm -clippy -debug -doc -libressl -miri -nightly (-parallel-compiler) -rls -rustfmt -system-bootstrap -test -verify-sig% -wasm" LLVM_TARGETS="AMDGPU (PowerPC) -AArch64 -ARM -AVR -BPF -Hexagon -Lanai -MSP430 -Mips -NVPTX -RISCV -Sparc -SystemZ -WebAssembly -X86 -XCore" 0 KiB
[ebuild     U #] virtual/rust-1.49.0::localrepo [1.48.0::localrepo] USE="-rustfmt" 0 KiB
stefson commented 2 years ago

I did my testing of the backported patches in an amd64 chroot, there were no problems whatsoever - I'm actually sitting on the backported patchset up to =rust-1.55.0 (:

You could try to switch the system-llvm flag off, and maybe the system-bootstrap flag on.

ernsteiswuerfel commented 2 years ago

You could try to switch the system-llvm flag off, and maybe the system-bootstrap flag on.

Looks like I get the same error with -system-llvm +system-bootstrap. rust-1.49.0:20220301-155430.log.gz

stefson commented 2 years ago

@ernsteiswuerfel head over to smaeul, I've got 1.50.0 ready in a pullrequest

ernsteiswuerfel commented 2 years ago

@ernsteiswuerfel head over to smaeul, I've got 1.50.0 ready in a pullrequest

I was not able to merge your PR in my local smaeul repository (managed by eselect repository) but I cloned your rust-1.50 in my localrepo.

Emerging 1.50 did not succeed:

>>> Unpacking source...
>>> Unpacking rustc-1.50.0-src.tar.xz to /var/tmp/portage/dev-lang/rust-1.50.0/work
>>> Source unpacked in /var/tmp/portage/dev-lang/rust-1.50.0/work
>>> Preparing source in /var/tmp/portage/dev-lang/rust-1.50.0/work/rustc-1.50.0-src ...
/var/tmp/portage/dev-lang/rust-1.50.0/temp/environment: line 3886: /var/tmp/portage/dev-lang/rust-1.50.0/work/rust-1.49.0-powerpc-gentoo-linux-musl/install.sh: No such file or directory
 * ERROR: dev-lang/rust-1.50.0::localrepo failed (prepare phase):
stefson commented 2 years ago

There are no stages available, you need to bootstrap all the way through with system-bootstrap. I remember that you failed to emerge 1.49.0, so maybe go back to 1.48.0 as a base and start to bootstrap this way?

ernsteiswuerfel commented 2 years ago

I modified the ebuild to make bootstrapping with 1.48 possible via

-local rustc_wanted="$(ver_cut 1).$(($(ver_cut 2) - 1))"
+local rustc_wanted="$(ver_cut 1).$(($(ver_cut 2) - 2))"

But I didn't succeed, this version seems to be to old for 1.50 already. rust-1.50.0:20220314-115624.log

stefson commented 2 years ago

thats not going to work, you always need the prior main version of rust to bootstrap. rust ist a self hosting compiler, this means you can't use gcc or clang to compile it.

that machine is rather fast, try to go back to rust-1.48.0 (or rust-1.47.0 if you have to), and system-bootstrap your way up from there. this seems to be your only choice to me.

ernsteiswuerfel commented 2 years ago

thats not going to work, you always need the prior main version of rust to bootstrap. rust ist a self hosting compiler, this means you can't use gcc or clang to compile it.

Ok, I see. Bootstrapping 1.49 on ppc32 didn't work either... Last version which builds here is 1.48. I'llt try next on ppc64 and see how that turns out.

Nevertheless, thanks for you work on it anyway! 1.49 on my x86 Pentium 4 box works well.

that machine is rather fast, try to go back to rust-1.48.0 (or rust-1.47.0 if you have to), and system-bootstrap your way up from there. this seems to be your only choice to me.

My Talos II got 2 x 4-core POWER9 so thats 8 cores w. 32 threads. Good for distcc'ing my slower boxes or building stuff in a chroot. 😃

ernsteiswuerfel commented 2 years ago

@stefson Meanwhile I built your 1.50.0 on ppc64 which worked perfectly well, bootstrapping the way from 1.48 up. 🌞 So only ppc(32) got a problem here. rust-1.50.0:20220315-232043.log.gz

stefson commented 2 years ago

Thats really unfortunate, you can try to reach out to the musl mailing list and explain your ppc32 problem to ask for suggestions. I was having a similar problem with rust-1.41.0 before time64 was added with musl-1.2.0, so maybe some ppc32 implementation is not correct or complete.

In the meantime, feel free to try the rust-1.51.0 ebuild: rust-1.51.0.tar.gz

I'd be happy to hear from you if you've managed to build a firefox-91esr with that, given that this is an interesting package for you.

ernsteiswuerfel commented 2 years ago

@stefson clang build of firefox-91.7.1 failed but the gcc build succeeded. Thanks! rust-1.52.0 would be great too as you need it for current librsvg to build.

Also had to lower rust requirements in cargo.eclass to make cbindgen build.

stefson commented 2 years ago

unfortunatley thats not true, you need at least rust-1.54.0 to compile the now stable librsvg-1.52.6. I just spend half a day debugging this lol

this one liner gets pulled in through the vendored sources: https://github.com/EmbarkStudios/cfg-expr/commit/3fffc4700ecf8d563261e04be67eba07a9992ce6#diff-b1a35a68f14e696205874893c07fd24fdb88882b47c23cc0e0c80a30c7d53759R1 and now demands this function: https://github.com/rust-lang/rust/pull/83366 which was stabilized in rust-1.54.0

please add the failed build.log from the clang build so I can take a look at it.

ernsteiswuerfel commented 2 years ago

Failed clang firefox-91.7.1 build. Big Endian patches are from: https://github.com/void-ppc/void-packages/tree/master/srcpkgs/firefox/patches

firefox-91.7.1:20220320-141047.log.gz

stefson commented 2 years ago

I've seen a similar compile error long time ago with an x86 build, it seems alpine has hit it as well. They decided to rip out the entire section:

https://github.com/alpinelinux/aports/blob/master/community/firefox-esr/disable-moz-stackwalk.patch

I'd say give it a try, what do you think?

ernsteiswuerfel commented 2 years ago

Turned out i needed following patch for the firefox clang build: https://github.com/void-ppc/void-packages/blob/master/srcpkgs/firefox/patches/fix-i686-ppc-musl.patch

Seems to be a less invasive version of your patch posted.

stefson commented 2 years ago

I think it literally means the same, as you end up at

#  define MOZ_STACKWALK_SUPPORTS_LINUX 0

anyway, which disables the moz_stackwalk stuff. In alpine for all cases, in void-ppc for all cases except GLIBC.

ernsteiswuerfel commented 2 years ago

Btw. firefox-91.7.0 fails to build on musl x86 for another reason. firefox-91.7.0:20220320-161451.log.gz

stefson commented 2 years ago

this is an old bug that should have been fixed a long time ago, you may want to look at 0011-musl-Support-custom-vendor-strings.patch from the old firefox-78esr patchset series; it has all the relevant information for you to talk to the upstream of that crate

you can also try to pass EXTRA_ECONF="--disable-cranelift" via env file in /etc/portage/env/www-client/

ernsteiswuerfel commented 2 years ago

unfortunatley thats not true, you need at least rust-1.54.0 to compile the now stable librsvg-1.52.6. I just spend half a day debugging this lol

Interesting, librsvg-2.52.7.ebuild only requires >=virtual/rust-1.52. librsvg-2.54.0.ebuild is the first one to require >=virtual/rust-1.54.

stefson commented 2 years ago

the 2.52.x-librsvg ebuilds were added just a few weeks ago, no one questioned the minimal rust of 1.52 as everyone had already upgraded to a llvm-13 based rust.

In short: librsvg-2.52.5 makes use of a function called option_insert, which is stable with rust-1.53.0 or later, and librsvg-2.52.6 makes use of a function called extended_key_value_attributes, which is stable with rust-1.54.0 or later. Its very uncommon to raise compiler demands in point releases, especially if its done dead silently like here.

librsvg-2.54.0 needs a rust compiler with edition = 2021, which is stable in rust-1.56.1 or later.

I will push all the ebuilds I have ready to my own ebuild in the evening, smauel seems to need longer to review my pullrequest, or maybe he doesn't feel comfortable with depending on system-bootstrap all the way. I will tag you when its done.