void-linux / void-packages

The Void source packages collection
https://voidlinux.org
Other
2.55k stars 2.13k forks source link

`./xbps-src bootstrap' fails #25604

Closed unspecd closed 3 years ago

unspecd commented 3 years ago

System

Expected behavior

./xbps-src zap && ./xbps-src bootstrap succeeds.

Actual behavior

...
[*] Collecting package files
kernel-libc-headers-4.19.0_2: collecting files...
musl-1.1.24_3: collecting files...
musl-devel-1.1.24_3: collecting files...
xbps-triggers-0.117_1: collecting files...
base-files-0.141_4: collecting files...
zlib-1.2.11_3: collecting files...
libgcc-9.3.0_6: collecting files...
libstdc++-9.3.0_6: collecting files...
binutils-2.34_1: collecting files...
libgcc-devel-9.3.0_6: collecting files...
libstdc++-devel-9.3.0_6: collecting files...
libssp-9.3.0_6: collecting files...
libssp-devel-9.3.0_6: collecting files...
gcc-9.3.0_6: collecting files...
libada-9.3.0_6: collecting files...
libada-devel-9.3.0_6: collecting files...
ERROR: libada-devel-9.3.0_6: file `/usr/lib/gcc/x86_64-linux-musl/9.3/plugin/include/ada/gcc-interface/ada-tree.def' already installed by package gcc-9.3.0_6.
Transaction failed! see above for errors.
=> ERROR: xbps-src: failed to install base-chroot!

Context

etc/conf contains the following line:

XBPS_PKG_OPTIONS_gcc=~ada
ahesford commented 3 years ago

Two things:

  1. Unless you really understand the consequences, you don't want bootstrap. Use binary-bootstrap instead. (When bootstrapping from source, it takes a few iterations to remove host artifacts from the masterdir.)

  2. If you do use bootstrap, try XBPS_PKG_OPTIONS="~ada" instead of the gcc-specific variant. I remember some reports about the per-package option variables not working as expected.

sgn commented 3 years ago
2. If you _do_ use `bootstrap`, try `XBPS_PKG_OPTIONS="~ada"` instead of the gcc-specific variant. I remember some reports about the per-package option variables not working as expected.

~ada still puts some ada related files into gcc installation, which normally split into gcc-ada. I think we would need to either:

unspecd commented 3 years ago

@ahesford

try XBPS_PKG_OPTIONS="~ada"

Thanks. I didn't test yet, I'll try a little later.

it takes a few iterations to remove host artifacts

https://github.com/void-linux/void-packages/blob/458f2a29520e38a1b26c247ad90e7523e9b6ed41/README.md#L160-L164 Wonder if this is still relevant in light of your comment.

https://github.com/void-linux/void-packages/blob/458f2a29520e38a1b26c247ad90e7523e9b6ed41/README.md#L164-L168 I also don't understand what "container" means here. Could you please provide a sequence of commands to achieve this?

sgn commented 3 years ago

Wonder if this is still relevant in light of your comment.

ahesford said:

When bootstrapping from source, it takes a few iterations to remove host artifacts from the masterdir.

Yes, it's still relevant, a few iterations includes stage0, stage1, stage2, and final masterdir

Could you please provide a sequence of commands to achieve this?

$ # make sure we start from nothing
$ rm -rf /path/to/hostdir /path/to/masterdir
$ # build stage0
$ ./xbps-src -N -H /path/to/hostdir -m /path/to/masterdir bootstrap
$ # L165: use the resulting stage 0 container to rebuild all  `bootstrap` packages again
$ rm -rf /path/to/hostdir/binpkgs
$ ./xbps-src -N -H /path/to/hostdir -m /path/to/masterdir pkg base-chroot
$ # L166: then use `binary-bootstrap` (stage 1)
$ rm -rf /path/to/masterdir
$ ./xbps-src -N -H /path/to/hostdir -m /path/to/masterdir binary-bootstrap
$ # L166: and rebuild the `bootstrap` packages once more
$ rm -rf /path/to/hostdir/binpkgs
$ ./xbps-src -N -H /path/to/hostdir -m /path/to/masterdir pkg base-chroot
$ # L167: then use `binary-bootstrap` again
$ rm -rf /path/to/masterdir
$ ./xbps-src -N -H /path/to/hostdir -m /path/to/masterdir binary-bootstrap
travankor commented 3 years ago

@ev-ermakov

It's not really a bug like they stated, but maybe some documentation on bootstrapping could be made clearer in the future.

unspecd commented 3 years ago

@sgn Thanks, now I get it.

I've also found that m4 will not compile if texinfo is not in the hostmakedepends list (stage0 -> stage1).

sgn commented 3 years ago

Anything else to fix? https://github.com/void-linux/void-packages/commit/66bf185310ef656f05f8b9c24fb58470e8f27a8f

RossComputerGuy commented 3 years ago

Is bootstrapping for a different architecture just all of those commands with -a set? Or do I have to bootstrap for the host system's architecture and then run pkg with -a set? If I do have to bootstrap with -a, then that will explain why I'm getting libssp missing errors on aarch64 and not x86-64.

Duncaen commented 3 years ago

Yes -a instructs xbps to do cross compilation, without it its quite surprising that this does not fail earlier, but that is the issue.

Using ./xbps-src bootstrap cross-arch is like creating a masterdir with ./xbps-src binary-bootstrap cross-arch which is useless, you want a build chroot that works on your host architecture and then cross compile from there with a cross toolchain.

RossComputerGuy commented 3 years ago

So binary-bootstrap then -a aarch64 pkg base-system? But then I get the error saying libssp>=4.4.0_1 is missing when it's trying to build libressl-devel, never had this error when building for x86-64.

Duncaen commented 3 years ago

So binary-bootstrap then -a aarch64 pkg base-system?

Yes, works fine here, might be either your masterdir being broken or you local repo containing some outdated packages.

xbps-checkvers -i -R hostdir/binpkgs/ -D ~/void-packages/
XBPS_ARCH=aarch64 xbps-checkvers -i -R hostdir/binpkgs/ -D ~/void-packages/
RossComputerGuy commented 3 years ago

xbps-checkvers -i -R hostdir/binpkgs/ -D ~/void-packages/ does not show anything.

RossComputerGuy commented 3 years ago
=> libressl-3.1.5_2: running pre-pkg hook: 04-generate-runtime-deps ...
   SONAME: libtls.so.20 <-> libtls20-3.1.5_2
   SONAME: libcrypto.so.46 <-> libcrypto46-3.1.5_2
   SONAME: libpthread.so.0 <-> glibc>=2.32_1
   SONAME: libssp.so.0 <-> libssp>=4.4.0_1
   SONAME: libc.so.6 <-> glibc>=2.32_1
   SONAME: libssl.so.48 <-> libssl48-3.1.5_2
=> libressl-3.1.5_2: running pre-pkg hook: 05-prepare-32bit ...

...

=> libevent-2.1.12_1: building [gnu-configure] (dependency of nghttp2) for aarch64...
   [target] libressl-devel-3.1.5_2: found (/host/binpkgs)
   [runtime] libressl-devel-3.1.5_2: found (/host/binpkgs)
   [runtime] libevent-2.1.12_1: not found (self, ignored)
=> libevent-2.1.12_1: installing target dependencies: libressl-devel-3.1.5_2 ...
=> libevent-2.1.12_1: removing autodeps, please wait...
=> libevent-2.1.12_1: removing autocrossdeps, please wait...
=> ERROR: libevent-2.1.12_1: failed to install 'libressl-devel-3.1.5_2' dependency! (error 19)
MISSING: libssp>=4.4.0_1
MISSING: libssp>=4.4.0_1
MISSING: libssp>=4.4.0_1
MISSING: libssp>=4.4.0_1
Transaction aborted due to unresolved dependencies.
=> ERROR: Please see above for the real error, exiting...

My fork is only 7 commits behind and according to Git, the files for libssp/gcc are in sync. https://github.com/ExpidusOS/packages/. Just about 10 hours ago, x86_64 finished compiling so it should be very much up to date.

Duncaen commented 3 years ago

Something wrong with your local repo. libssp exists since 2009 and the shared library version has not changed since 2012, maybe even longer.

RossComputerGuy commented 3 years ago

I uploaded the packages to my server, idk what went wrong. I haven't gotten any other errors.

Sent from ProtonMail mobile

-------- Original Message -------- On Feb 12, 2021, 10:04 AM, Duncan Overbruck wrote:

Something wrong with your local repo. libssp exists since 2009 and the shared library version has not changed since 2012, maybe even longer.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe.

RossComputerGuy commented 3 years ago
python3-setuptools 52.0.0_1 53.0.0_1 python3-setuptools /home/ross/ExpidusOS/packages/hostdir/binpkgs
base-chroot 0.1.0_3 0.66_3 base-chroot /home/ross/ExpidusOS/packages/hostdir/binpkgs
base-system 0.1.0_1 0.114_1 base-system /home/ross/ExpidusOS/packages/hostdir/binpkgs
libcap 2.46_1 2.48_1 libcap /home/ross/ExpidusOS/packages/hostdir/binpkgs
base-files 0.1.0_7 0.141_7 base-files /home/ross/ExpidusOS/packages/hostdir/binpkgs
util-linux-libs 2.36_1 2.36.1_2 util-linux-libs /home/ross/ExpidusOS/packages/hostdir/binpkgs
util-linux 2.36_1 2.36.1_2 util-linux /home/ross/ExpidusOS/packages/hostdir/binpkgs
base-devel 0.1.0_2 20181003_2 base-devel /home/ross/ExpidusOS/packages/hostdir/binpkgs
gd 2.3.0_2 2.3.1_1 gd /home/ross/ExpidusOS/packages/hostdir/binpkgs
postgresql13 13.1_1 13.2_1 postgresql13 /home/ross/ExpidusOS/packages/hostdir/binpkgs

I got the checkvers to show output. The base packages follow a different versioning because my OS is a non-rolling distro.

RossComputerGuy commented 3 years ago

xbps-dgraph -f libressl-devel | dot -Tpng | feh - shows that libssp is not needed so why does it say that libssp is needed?

Duncaen commented 3 years ago

No idea why your package builds with libssp.

RossComputerGuy commented 3 years ago

I noticed that the dummy vpkg only provides libssp for musl targets but the vpkg for aarch64 glibc has libssp, I'm testing if making it so libssp is provided for both in the dummy vpkg will fix the issue.

Duncaen commented 3 years ago

It shouldn't link libssp, but its pretty hard to follow your fork because you don't use git features and void-packages commit style. If you touch just single templates in commits and use normal merges, diffing this with github would be easier.

RossComputerGuy commented 3 years ago

My solution fixed it. And I think one of these files has to do with ssp being enabled, I only messed with the repo keys and shlibs so this problem may occur outside of ExpidusOS (my fork).

common/shlibs:libssp.so.0 libssp-4.4.0_1
common/build-style/void-cross.sh:       --disable-libssp \
common/build-style/void-cross.sh:       --enable-default-ssp \
common/build-style/void-cross.sh:       --enable-libssp \
common/travis/license.lst:Classpath-exception-2.0
common/travis/license.lst:deprecated_GPL-2.0-with-classpath-exception
common/environment/configure/hardening.sh:  # Our compilers use --enable-default-pie and --enable-default-ssp,
Duncaen commented 3 years ago

Yea doesn't sound like a solution tbh, this will make problems in the future. common/build-style/void-cross.sh is the build style for cross compilers. Something is wrong with your repo or void-packages fork, my or voids cross aarch64 builds do not link libssp.