batch: treat bootstrap packages specially

[stapelberg]

Here are some rough notes from when I last bumped all package versions and hence had to bootstrap:

% for pkg in $(echo attr gcc-libs bash bc binutils bison coreutils diffutils ed elfutils file findutils flex gawk gcc glib glibc gmp gperf grep gzip help2man kmod libcap libffi linux m4 make meson mpc mpfr ninja openssl pam patchelf perl pkg-config python3 sed strace tar texinfo util-linux zlib); do echo $pkg; cp -a O.pkg/${pkg}-* pkg/; done

rebuild gcc first:

1. gmp
2. mpfr
3. mpc
4. gcc (so that it finds the new glibc libc.so)
5. gawk

Then:

1. bc, binutils, m4, sed, zlib, tar, diffutils, findutils, ed, texinfo, grep, gzip, kmod, libffi, pam, perl, flex, make, libcap, attr
2. bison (m4), util-linux (pam), openssl (perl), glibc (zlib), coreutils (attr), gperf
3. file (glibc, zlib), python3 (openssl, zlib, libffi), glib (util-linux), elfutils (glibc, zlib), patchelf (glibc), ninja
4. meson (ninja, python3)

We should probably make the batch tool figure out the required order. It can already recognize packages from the bootstrap set.

[the-maldridge]

I'm currently trying to bootstrap from a clean git checkout on a Void Linux host. Following your notes here I'm not having much luck, as there seems to be an implicit dependency on having a built distri bash available.

For reference, what I tried was:

maldridge@theGibson:~$ cd Documents/distri/
maldridge@theGibson:~/Documents/distri (master)$ export DISTRIROOT=$PWD
maldridge@theGibson:~/Documents/distri (master)$ distri build -pkg gmp
2019/12/21 23:35:12 building gmp-amd64-6.1.2-3
2019/12/21 23:35:12 downloading https://gmplib.org/download/gmp/gmp-6.1.2.tar.bz2 to gmp-6.1.2.tar.bz2
2019/12/21 23:35:22 verifying gmp-6.1.2.tar.bz2
build: build: builddeps: package "bash" not found (pattern /home/maldridge/Documents/distri/build/distri/pkg/bash-amd)
maldridge@theGibson:~/Documents/distri (master %)$ distri build -pkg bash
2019/12/21 23:35:37 building bash-amd64-4.4.18-3
2019/12/21 23:35:37 downloading https://ftp.gnu.org/gnu/bash/bash-4.4.18.tar.gz to bash-4.4.18.tar.gz
2019/12/21 23:35:39 verifying bash-4.4.18.tar.gz
build: build: builddeps: package "bash" not found (pattern /home/maldridge/Documents/distri/build/distri/pkg/bash-amd)`

I also checked cmd/distri/bootstrap.go which I thought might help with bootstrapping, but haven't had much luck with that either since it fails to build.

Is there some documentation I missed somewhere on bootstrapping?

[stapelberg]

See the first code block in my comment: you need to copy these packages from an existing distri release, or build them non-hermetically (i.e. with host dependencies, using -hermetic=false) and then re-build them again.

We should probably have a clear term for these two different kinds of bootstrapping.