Have all modules point to either github/debian/gentoo/alternatives archives for tarballs

[tlaurion]

Heads had several build breakages in the past because repositories go either down, have certificate problems etc.

Pointing to github releases or Debian archives (ref: https://github.com/osresearch/linux-builder/issues/1#issuecomment-1189779961) would mitigate this issue for reproducibility in the future. Neither of those should have either cert/DNS issues that go unnoticed.

Those may require checksums update, though under modules, as opposed to bouncing packages from mirror to mirrors like we do from time to time. But building an old Heads commit after that change should be possible as long as the recipes from CircleCI are respected to have the same docker image (OS version) and installed packages.

Edit: Need arose at https://github.com/osresearch/heads/pull/1188#issuecomment-1203239597

[tlaurion]

Becomes relevant again, see #1439 where acpica.org now points to intel.com and breaks all past of 4.19 coreboot builds and forks.

EDIT: but debian archives are not helping in #1439 situation, see https://snapshot.debian.org/package/acpica-unix/ only offering 2023 and 2020 source tarballs and nothing in between. Short term fix for acpica archives in between is to patch under Heads coreboot crossgcc buildsystem to point download urls to https://distfiles.macports.org/acpica/

This is not sustainable.

coreboot 4.19+ points to github releases. But Github also has history of breaking archives, also covered in coreboot upstream issue https://review.coreboot.org/c/coreboot/+/76399

[JonathonHall-Purism]

Maybe we could add locations as secondary mirrors instead of switching everything over to another source?

If we have another place (or more than one) to store archives, we could upload whatever version we are using and have the build system try those sources as secondary mirrors. Mirrors could be down at any given time (builds only break if there are no reachable sources for a given archive). Digests don't have to change right away so it has a lower initial impact.

Even if we're not perfect about uploading everything we use, we usually have a copy of the relevant archive when the primary source goes down (and build system checks the digest), we could upload it reactively.

Tricky part there is finding place(s) to put the archives I think. As a secondary mirror, I might be able to do something on cloud.puri.sm, or maybe source.puri.sm could handle it (does not sound significantly different from anything already done for PureOS).

[tlaurion]

Hey @githubisnonfree (Leah from libreboot), we discussed over libreboot channel a while back the possibility of hosting, and mirroring, tarballs used by Heads. Does the offer still stands?

The easiest way to do this as of today is to simply build Heads locally:

• one legacy board (x230-legacy) OR server variant (kgpe-d16) (no fbwhiptail, whiptail equivalent)
• one hotp variant (x230-hotp-maximized) : fbwhiptail and dependencies

On debian-11 per .circleci example:

eg: sudo apt install -y build-essential zlib1g-dev uuid-dev libdigest-sha-perl libelf-dev bc bzip2 bison flex git gnupg gawk iasl m4 nasm patch python python2 python3 wget gnat cpio ccache pkg-config cmake libusb-1.0-0-dev autoconf texinfo ncurses-dev doxygen graphviz udev libudev1 libudev-dev automake libtool rsync innoextract sudo libssl-dev device-tree-compiler u-boot-tools sharutils e2fsprogs parted curl unzip imagemagick libncurses5-dev zip make BOARD=x230-legacy and make BOARD=x230-hotp-maximized

(once #1570 being merged, no additional step will be needed to download blobs for any of the boards).

Then the packages would be under: packages/x86/ to be uploaded wherever.

And doing so, cairo/pixman/whiptail/fbwhiptail and all other dependencies would be downloaded locally with their checksums validated.

The result could then be uploaded to what is then mirrored to/from others. And then Heads could find a way either statically/better, dynamically, to point/parse to libreboot mirror list to get new mirror list in case prior sources are failing to download.

Thoughts?

---

Sidenote, does libreboot plan to rebridge to Matrix? Otherwise, how can we foster better collaboration? Matrix supports OFTC, Soonet, W3C, gitter.im until librechat<->Matrix resolve their withstanding issues. What would be the way to bridge the gap between our communities?