Closed RebornRider closed 3 years ago
I added the loop to work around download unreliability using Tor. What is your mirror configuration?
404 doesn't look like a tor reliability issue.
I added the loop to work around download unreliability using Tor. What is your mirror configuration?
My mirror list in prepare-chroot-base is:
DEFAULT_ARCHLINUX_MIRROR="\
http://mirror.rackspace.com\
,http://mirrors.evowise.com\
,https://mirror.rackspace.com\
"
template-archlinux.log shows:
--> Archlinux 01_install_core.sh
--> Archlinux prepare-chroot-base
--> Binding INSTALLDIR '/home/user/qubes-builder/qubes-src/linux-template-builder/mnt' to bootstrap environment...
--> Setting pacman mirrorlist as 'http://mirror.rackspace.com http://mirrors.evowise.com https://mirror.rackspace.com'...
--> Initializing pacman keychain...
I'm using sys-firewall as my NetVM, so I'm not running over tor.
The versions of zstd and gnupg that rackspace / evowise list are newer than what is requested when building the template:
They currently list:
But the template tries to get these earlier versions:
But the template tries to get these earlier versions:
- zstd-1.4.5-1-x86_64.pkg.tar.zst
- gnupg-2.2.26-1-x86_64.pkg.tar.zst
That’s odd. I wonder if the mirror from which Pacman is getting its database is behind the mirror from which it is getting the packages. Can you try using mirrors.kernel.org
as the sole mirror? It’s Tier 1, meaning it is updated directly from Arch.
404 doesn't look like a tor reliability issue.
Indeed it is not.
I repeated al the steps with the addition of setting http://mirrors.kernel.org/ as the sole mirror. (after running setup.sh) That mirror seems to have way slower download speeds for me, but the packages now get downloaded and installed! 👍🏾
But after that, make template
now gets stuck on:
qubes/qubes-vm-xen 4.8.5-29
Xen is a virtual machine monitor
--> Installing mandatory qubes packages...
resolving dependencies...
warning: cannot resolve "pulseaudio<14.1", a dependency of "qubes-vm-pulseaudio"
warning: cannot resolve "qubes-vm-pulseaudio", a dependency of "qubes-vm-dependencies"
:: The following package cannot be upgraded due to unresolvable dependencies:
qubes-vm-dependencies
:: Do you want to skip the above package for this upgrade? [y/N] error: failed to prepare transaction (could not satisfy dependencies)
:: unable to satisfy dependency 'pulseaudio<14.1' required by qubes-vm-pulseaudio
:: unable to satisfy dependency 'qubes-vm-pulseaudio' required by qubes-vm-dependencies
resolving dependencies...
warning: cannot resolve "pulseaudio<14.1", a dependency of "qubes-vm-pulseaudio"
warning: cannot resolve "qubes-vm-pulseaudio", a dependency of "qubes-vm-dependencies"
:: The following package cannot be upgraded due to unresolvable dependencies:
qubes-vm-dependencies
:: Do you want to skip the above package for this upgrade? [y/N] error: failed to prepare transaction (could not satisfy dependencies)
...
I still had to apply the fix for pulsaudio, to get make qubes-vm
to succeed before running make template
.
https://github.com/Qubes-Community/Contents/blob/master/docs/building/building-archlinux-template.md#missing-pulsecore-error-when-building-the-gui-agent-linux
from 4.1 , only clearnet. qubes-builder need reproducible builds something like CircleCi
Building template: archlinux-minimal
-> Preparing instalation of archlinux template...
--> Archlinux 00_prepare.sh
--> Downloading Archlinux bootstrap tarball (v)...
--2021-01-22 20:53:20-- https://mirrors.edge.kernel.org/archlinux/iso/latest/archlinux-bootstrap-2021.01.01-x86_64.tar.gz
Resolving mirrors.edge.kernel.org (mirrors.edge.kernel.org)... 147.75.101.1, 2604:1380:2001:3900::1
Connecting to mirrors.edge.kernel.org (mirrors.edge.kernel.org)|147.75.101.1|:443... connected.
HTTP request sent, awaiting response... 304 Not Modified
File 'cache_archlinux/archlinux-bootstrap-2021.01.01-x86_64.tar.gz' not modified on server. Omitting download.
--2021-01-22 20:53:20-- https://mirrors.edge.kernel.org/archlinux/iso/latest/archlinux-bootstrap-2021.01.01-x86_64.tar.gz.sig
Resolving mirrors.edge.kernel.org (mirrors.edge.kernel.org)... 147.75.101.1, 2604:1380:2001:3900::1
Connecting to mirrors.edge.kernel.org (mirrors.edge.kernel.org)|147.75.101.1|:443... connected.
HTTP request sent, awaiting response... 304 Not Modified
File 'cache_archlinux/archlinux-bootstrap-2021.01.01-x86_64.tar.gz.sig' not modified on server. Omitting download.
--> Preparing GnuPG to verify tarball...
gpg: key 3348882F6AC6A4C2: "Pierre Schmitz (Arch Linux Master Key) <pierre@master-key.archlinux.org>" not changed
gpg: key 5184252D824B18E8: 1 signature not checked due to a missing key
gpg: key 5184252D824B18E8: "Thomas B�chler (Arch Linux Master Key) <thomas@master-key.archlinux.org>" not changed
gpg: key 7EFD567D4C7EA887: 1 signature not checked due to a missing key
gpg: key 7EFD567D4C7EA887: "Ionut Biru (Arch Linux Master Key) <ionut@master-key.archlinux.org>" not changed
gpg: key BA1DFB64FFF979E7: 4 signatures not checked due to missing keys
gpg: key BA1DFB64FFF979E7: "Allan McRae (Arch Linux Master Key) <allan@master-key.archlinux.org>" not changed
gpg: key A04F9397CDFD6BB0: 2 signatures not checked due to missing keys
gpg: key A04F9397CDFD6BB0: "Dan McGee (Arch Linux Master Key) <dan@master-key.archlinux.org>" not changed
gpg: key 7F2D434B9741E8AC: 9 signatures not checked due to missing keys
gpg: key 7F2D434B9741E8AC: "Pierre Schmitz <pierre@archlinux.de>" not changed
gpg: Total number processed: 6
gpg: unchanged: 6
--> Verifying tarball...
gpg: Signature made Fri Jan 1 12:23:57 2021 MSK
gpg: using RSA key 4AA4767BBC9C4B1D18AE28B77F2D434B9741E8AC
gpg: Good signature from "Pierre Schmitz <pierre@archlinux.de>" [unknown]
gpg: WARNING: This key is not certified with a trusted signature!
gpg: There is no indication that the signature belongs to the owner.
Primary key fingerprint: 4AA4 767B BC9C 4B1D 18AE 28B7 7F2D 434B 9741 E8AC
--> NB: Bootstrap tarball not newer than bootstrap directory, will use existing!
-> Image file already exists, assuming *update*...
--> Archlinux 01_install_core.sh
--> Archlinux prepare-chroot-base
--> NB: INSTALLDIR '/home/user/qubes-builder/qubes-src/linux-template-builder/mnt' already appears to have an environment; will leave as-is!
--> Unbinding INSTALLDIR...
umount: cache_archlinux/bootstrap/mnt: not mounted.
-> Installing package groups...
--> Archlinux 02_install_groups.sh
--> Synchronize resolv.conf...
--> Updating installed packages...
:: Synchronizing package databases...
core is up to date
extra is up to date
community is up to date
:: Starting full system upgrade...
there is nothing to do
--> Installing archlinux package groups...
--> Selected packages: xorg xterm ethtool net-tools sudo wget diffutils
:: There are 48 members in group xorg:
:: Repository extra
1) xf86-video-vesa 2) xorg-bdftopcf 3) xorg-docs 4) xorg-font-util 5) xorg-fonts-100dpi 6) xorg-fonts-75dpi 7) xorg-fonts-encodings 8) xorg-iceauth 9) xorg-mkfontscale 10) xorg-server 11) xorg-server-common 12) xorg-server-devel 13) xorg-server-xephyr 14) xorg-server-xnest 15) xorg-server-xvfb 16) xorg-sessreg 17) xorg-setxkbmap 18) xorg-smproxy 19) xorg-x11perf 20) xorg-xauth 21) xorg-xbacklight 22) xorg-xcmsdb 23) xorg-xcursorgen 24) xorg-xdpyinfo 25) xorg-xdriinfo 26) xorg-xev 27) xorg-xgamma 28) xorg-xhost 29) xorg-xinput 30) xorg-xkbcomp 31) xorg-xkbevd 32) xorg-xkbutils 33) xorg-xkill 34) xorg-xlsatoms 35) xorg-xlsclients 36) xorg-xmodmap 37) xorg-xpr 38) xorg-xprop 39) xorg-xrandr 40) xorg-xrdb 41) xorg-xrefresh 42) xorg-xset 43) xorg-xsetroot 44) xorg-xvinfo 45) xorg-xwayland 46) xorg-xwd 47) xorg-xwininfo 48) xorg-xwud
Enter a selection (default=all):
resolving dependencies...
looking for conflicting packages...
warning: dependency cycle detected:
warning: harfbuzz will be installed before its freetype2 dependency
warning: dependency cycle detected:
warning: mesa will be installed before its libglvnd dependency
Packages (125) db-5.3.28-5 fontconfig-2:2.13.91+48+gfcb0420-2 freetype2-2.10.4-1 gdbm-1.19-1 graphite-1:1.3.14-1 groff-1.22.4-3 harfbuzz-2.7.4-1 libdrm-2.4.104-1 libedit-20191231_3.1-3 libepoxy-1.5.5-1 libevdev-1.10.1-1 libfontenc-1.1.4-3 libglvnd-1.3.2-1 libgudev-234-1 libice-1.0.10-3 libinput-1.16.4-1 libomxil-bellagio-0.9.3-3 libpciaccess-0.16-2 libpipeline-1.5.3-1 libpng-1.6.37-3 libsm-1.2.3-2 libunwind-1.3.1-2 libutempter-1.2.1-1 libwacom-1.7-1 libx11-1.7.0-3 libxau-1.0.9-3 libxaw-1.0.13-3 libxcb-1.14-1 libxcomposite-0.4.5-3 libxcursor-1.2.0-2 libxdamage-1.1.5-3 libxdmcp-1.1.3-3 libxext-1.3.4-3 libxfixes-5.0.3-4 libxfont2-2.0.4-3 libxft-2.3.3-2 libxi-1.7.10-3 libxinerama-1.1.4-3 libxkbfile-1.1.0-2 libxmu-1.1.3-2 libxpm-3.5.13-2 libxrandr-1.5.2-3 libxrender-0.9.10-4 libxshmfence-1.3-2 libxt-1.2.0-2 libxtst-1.2.3-4 libxv-1.0.11-4 libxxf86vm-1.1.4-4 llvm-libs-11.0.1-1 lm_sensors-3.6.0-2 luit-20201003-1 man-db-2.9.3-1 mesa-20.3.3-1 mtdev-1.1.6-1 perl-5.32.0-3 pixman-0.40.0-1 vulkan-icd-loader-1.2.166-1 wayland-1.18.0-2 xbitmaps-1.1.2-2 xcb-proto-1.14.1-3 xcb-util-0.4.0-3 xcb-util-image-0.4.0-3 xcb-util-keysyms-0.4.0-3 xcb-util-renderutil-0.3.9-3 xcb-util-wm-0.4.1-3 xf86-input-libinput-0.30.0-1 xkeyboard-config-2.31-1 xorg-fonts-alias-100dpi-1.0.4-1 xorg-fonts-alias-75dpi-1.0.4-1 xorg-util-macros-1.19.2-2 xorgproto-2020.1-1 diffutils-3.7-3 ethtool-1:5.10-1 net-tools-1.60.20181103git-2 sudo-1.9.5.p1-1 wget-1.21.1-1 xf86-video-vesa-2.5.0-1 xorg-bdftopcf-1.1-2 xorg-docs-1.7.1-3 xorg-font-util-1.3.2-2 xorg-fonts-100dpi-1.0.3-7 xorg-fonts-75dpi-1.0.3-7 xorg-fonts-encodings-1.0.5-2 xorg-iceauth-1.0.8-2 xorg-mkfontscale-1.2.1-2 xorg-server-1.20.10-3 xorg-server-common-1.20.10-3 xorg-server-devel-1.20.10-3 xorg-server-xephyr-1.20.10-3 xorg-server-xnest-1.20.10-3 xorg-server-xvfb-1.20.10-3 xorg-sessreg-1.1.2-2 xorg-setxkbmap-1.3.2-2 xorg-smproxy-1.0.6-3 xorg-x11perf-1.6.1-2 xorg-xauth-1.1-2 xorg-xbacklight-1.2.3-2 xorg-xcmsdb-1.0.5-3 xorg-xcursorgen-1.0.7-2 xorg-xdpyinfo-1.3.2-4 xorg-xdriinfo-1.0.6-2 xorg-xev-1.2.4-1 xorg-xgamma-1.0.6-3 xorg-xhost-1.0.8-2 xorg-xinput-1.6.3-2 xorg-xkbcomp-1.4.4-1 xorg-xkbevd-1.1.4-3 xorg-xkbutils-1.0.4-4 xorg-xkill-1.0.5-2 xorg-xlsatoms-1.1.3-2 xorg-xlsclients-1.1.4-2 xorg-xmodmap-1.0.10-2 xorg-xpr-1.0.5-2 xorg-xprop-1.2.5-1 xorg-xrandr-1.5.1-2 xorg-xrdb-1.2.0-2 xorg-xrefresh-1.0.6-2 xorg-xset-1.2.4-2 xorg-xsetroot-1.1.2-2 xorg-xvinfo-1.1.4-2 xorg-xwayland-1.20.0.r800.ge4a9f0bb4-1 xorg-xwd-1.0.7-2 xorg-xwininfo-1.1.5-2 xorg-xwud-1.0.5-2 xterm-363-1
Total Download Size: 0.13 MiB
Total Installed Size: 355.74 MiB
:: Proceed with installation? [Y/n]
:: Retrieving packages...
error: failed retrieving file 'net-tools-1.60.20181103git-2-x86_64.pkg.tar.xz' from mirrors.evowise.com : The requested URL returned error: 404
error: failed retrieving file 'net-tools-1.60.20181103git-2-x86_64.pkg.tar.xz' from mirror.rackspace.com : The requested URL returned error: 404
error: failed retrieving file 'net-tools-1.60.20181103git-2-x86_64.pkg.tar.xz' from mirror.rackspace.com : The requested URL returned error: 404
warning: failed to retrieve some files
error: failed to commit transaction (failed to retrieve some files)
Errors occurred, no packages were upgraded.
make[1]: *** [Makefile:65: rootimg-build] Error 1
@IDBEHOLDS See: #816.
Okay, I've got the archlinux template building and working.
I had to do 4 things:
depends=('alsa-lib' 'alsa-utils' 'pulseaudio-alsa' 'pulseaudio<14.1')
to pulseaudio<14.3 in archlinux/PKGBUILD (line 74)
see: https://github.com/QubesOS/qubes-gui-agent-linux/commit/291f697eb1a92f83954d75ee9d180906bda4ceedqvm-run -u root --pass-io archlinux 'leafpad /etc/sudoers.d/qubes'
and replacing %qubes ALL=(ALL) ROLE=unconfined_r TYPE=unconfined_t NOPASSWD: ALL
with %qubes ALL=(ALL) NOPASSWD: ALL
Otherwise trying to access sudo would result in the error:
/etc/sudoers.d/qubes:2:22: syntax error
%qubes ALL=(ALL) ROLE=unconfined_r TYPE=unconfined_t NOPASSWD: ALL
Since the 404'ing packages are not found in any of the default remote mirrors, the local package DBs must be out of date. So I don't think it has to do with using any specific mirror.
My guess is that this could be fixed by always calling pacman with the -y or -yy flags to force update the DBs, or, maybe more robustly, downloading all the needed packages to a local cache "atomically".
Just for reference, I was also able to get the template to build by exclusively using an archlinux package cache. (which also keeps 3 old versions of all packages)
--- a/prepare-chroot-base
+++ b/prepare-chroot-base
@@ -57,6 +57,7 @@ do
ARCHLINUX_MIRRORLIST=$(sed "s|#Server = ${MIRROR_ENTRY}/|Server = ${MIRROR_ENTRY}/|" <<< $ARCHLINUX_MIRRORLIST)
done
echo "$ARCHLINUX_MIRRORLIST" > "${CACHEDIR}/bootstrap/etc/pacman.d/mirrorlist"
+echo 'Server = http://10.99.99.99/archlinux/$repo/os/$arch' > "${CACHEDIR}/bootstrap/etc/pacman.d/mirrorlist"
cp /etc/resolv.conf "${BOOTSTRAP_DIR}/etc/"
Currently, the only blocker is in vmm-xen - otherwise the template builds correctly. This can be closed.
When I have seen this sort of error, it invariably relates to transitory errors in mirroring - that's why there is a 404 , because (as suggested) the server provides only an early or later version from that in the package list.
Qubes OS version
Qubes release 4.0 (R.4.0)
Affected component(s) or functionality
builder-archlinux
Brief summary
When trying to build the archlinux template, "make template" gets stuck in an endless loop and never finishes.
The terminal gets stuck on:
template-archlinux.log constantly continously repeats the following errors:
How Reproducible
I've tried to follow the the archlinux documentation multiple times with the same result. This happens every time. https://github.com/Qubes-Community/Contents/blob/master/docs/building/building-archlinux-template.md
To Reproduce
Steps to reproduce the behavior:
Expected behavior
make template finishes sucessfully
Actual behavior
make template never finishes
Screenshots
Additional context
I'm using the fedora-32 template as the basis for my build-archlinux2 AppVM
Solutions you've tried
I've tried to add https://archive.archlinux.org/repos/2021/01/15/ to the mirrorlist in prepare-chroot-base, as it contains the failing packages at the missing versions.
Relevant documentation you've consulted
Related, non-duplicate issues
logs template-archlinux.log