search

QubesOS / qubes-issues

The Qubes OS Project issue tracker
https://www.qubes-os.org/doc/issue-tracking/
536 stars 48 forks source link

Building the archlinux template gets stuck in endless loop, as it fails retrieve the gnupg-2.2.26-1 and std-1.4.5-1 packages from the pacman mirrors #6357

Closed RebornRider closed 3 years ago

RebornRider commented 3 years ago

Qubes OS version

Qubes release 4.0 (R.4.0)

Affected component(s) or functionality

builder-archlinux

Brief summary

When trying to build the archlinux template, "make template" gets stuck in an endless loop and never finishes.

The terminal gets stuck on:

-> Building template archlinux (logfile: build-logs/template-archlinux.log)...

template-archlinux.log constantly continously repeats the following errors:

:: Retrieving packages... error: failed retrieving file 'zstd-1.4.5-1-x86_64.pkg.tar.zst' from mirrors.evowise.com : The requested URL returned error: 404 error: failed retrieving file 'zstd-1.4.5-1-x86_64.pkg.tar.zst' from mirror.rackspace.com : The requested URL returned error: 404 error: failed retrieving file 'zstd-1.4.5-1-x86_64.pkg.tar.zst' from mirror.rackspace.com : The requested URL returned error: 404 warning: failed to retrieve some files error: failed retrieving file 'gnupg-2.2.26-1-x86_64.pkg.tar.zst' from mirrors.evowise.com : The requested URL returned error: 404 error: failed retrieving file 'gnupg-2.2.26-1-x86_64.pkg.tar.zst' from mirror.rackspace.com : The requested URL returned error: 404 error: failed retrieving file 'gnupg-2.2.26-1-x86_64.pkg.tar.zst' from mirror.rackspace.com : The requested URL returned error: 404 warning: failed to retrieve some files error: failed to commit transaction (failed to retrieve some files) Errors occurred, no packages were upgraded. ==> ERROR: Failed to install packages to new root

How Reproducible

I've tried to follow the the archlinux documentation multiple times with the same result. This happens every time. https://github.com/Qubes-Community/Contents/blob/master/docs/building/building-archlinux-template.md

To Reproduce

Steps to reproduce the behavior:

  1. follow https://github.com/Qubes-Community/Contents/blob/master/docs/building/building-archlinux-template.md till step 7
  2. apply the pulsaudio fix from https://github.com/Qubes-Community/Contents/blob/master/docs/building/building-archlinux-template.md#missing-pulsecore-error-when-building-the-gui-agent-linux
  3. follow the documentation till step 8 (make template)

Expected behavior

make template finishes sucessfully

Actual behavior

make template never finishes

Screenshots

Additional context

I'm using the fedora-32 template as the basis for my build-archlinux2 AppVM

Solutions you've tried

I've tried to add https://archive.archlinux.org/repos/2021/01/15/ to the mirrorlist in prepare-chroot-base, as it contains the failing packages at the missing versions.

Relevant documentation you've consulted

Related, non-duplicate issues

logs template-archlinux.log

DemiMarie commented 3 years ago

I added the loop to work around download unreliability using Tor. What is your mirror configuration?

marmarek commented 3 years ago

404 doesn't look like a tor reliability issue.

RebornRider commented 3 years ago

I added the loop to work around download unreliability using Tor. What is your mirror configuration?

My mirror list in prepare-chroot-base is:

DEFAULT_ARCHLINUX_MIRROR="\
http://mirror.rackspace.com\
,http://mirrors.evowise.com\
,https://mirror.rackspace.com\
"


  

template-archlinux.log shows:

--> Archlinux 01_install_core.sh
--> Archlinux prepare-chroot-base
  --> Binding INSTALLDIR '/home/user/qubes-builder/qubes-src/linux-template-builder/mnt' to bootstrap environment...
  --> Setting pacman mirrorlist as 'http://mirror.rackspace.com http://mirrors.evowise.com https://mirror.rackspace.com'...
  --> Initializing pacman keychain...


  

I'm using sys-firewall as my NetVM, so I'm not running over tor.
  

The versions of zstd and gnupg that rackspace / evowise list are newer than what is requested when building the template:

They currently list:

But the template tries to get these earlier versions:

DemiMarie commented 3 years ago

But the template tries to get these earlier versions:

  • zstd-1.4.5-1-x86_64.pkg.tar.zst
  • gnupg-2.2.26-1-x86_64.pkg.tar.zst

That’s odd. I wonder if the mirror from which Pacman is getting its database is behind the mirror from which it is getting the packages. Can you try using mirrors.kernel.org as the sole mirror? It’s Tier 1, meaning it is updated directly from Arch.

DemiMarie commented 3 years ago

404 doesn't look like a tor reliability issue.

Indeed it is not.

RebornRider commented 3 years ago

I repeated al the steps with the addition of setting http://mirrors.kernel.org/ as the sole mirror. (after running setup.sh) That mirror seems to have way slower download speeds for me, but the packages now get downloaded and installed! 👍🏾

But after that, make template now gets stuck on:

qubes/qubes-vm-xen 4.8.5-29
    Xen is a virtual machine monitor
  --> Installing mandatory qubes packages...
resolving dependencies...
warning: cannot resolve "pulseaudio<14.1", a dependency of "qubes-vm-pulseaudio"
warning: cannot resolve "qubes-vm-pulseaudio", a dependency of "qubes-vm-dependencies"
:: The following package cannot be upgraded due to unresolvable dependencies:
      qubes-vm-dependencies

:: Do you want to skip the above package for this upgrade? [y/N] error: failed to prepare transaction (could not satisfy dependencies)

:: unable to satisfy dependency 'pulseaudio<14.1' required by qubes-vm-pulseaudio
:: unable to satisfy dependency 'qubes-vm-pulseaudio' required by qubes-vm-dependencies
resolving dependencies...
warning: cannot resolve "pulseaudio<14.1", a dependency of "qubes-vm-pulseaudio"
warning: cannot resolve "qubes-vm-pulseaudio", a dependency of "qubes-vm-dependencies"
:: The following package cannot be upgraded due to unresolvable dependencies:
      qubes-vm-dependencies

:: Do you want to skip the above package for this upgrade? [y/N] error: failed to prepare transaction (could not satisfy dependencies)
...


I still had to apply the fix for pulsaudio, to get make qubes-vm to succeed before running make template. https://github.com/Qubes-Community/Contents/blob/master/docs/building/building-archlinux-template.md#missing-pulsecore-error-when-building-the-gui-agent-linux

IDBEHOLDS commented 3 years ago

from 4.1 , only clearnet. qubes-builder need reproducible builds something like CircleCi

Building template: archlinux-minimal
-> Preparing instalation of archlinux template...
--> Archlinux 00_prepare.sh
  --> Downloading Archlinux bootstrap tarball (v)...
--2021-01-22 20:53:20--  https://mirrors.edge.kernel.org/archlinux/iso/latest/archlinux-bootstrap-2021.01.01-x86_64.tar.gz
Resolving mirrors.edge.kernel.org (mirrors.edge.kernel.org)... 147.75.101.1, 2604:1380:2001:3900::1
Connecting to mirrors.edge.kernel.org (mirrors.edge.kernel.org)|147.75.101.1|:443... connected.
HTTP request sent, awaiting response... 304 Not Modified
File 'cache_archlinux/archlinux-bootstrap-2021.01.01-x86_64.tar.gz' not modified on server. Omitting download.

--2021-01-22 20:53:20--  https://mirrors.edge.kernel.org/archlinux/iso/latest/archlinux-bootstrap-2021.01.01-x86_64.tar.gz.sig
Resolving mirrors.edge.kernel.org (mirrors.edge.kernel.org)... 147.75.101.1, 2604:1380:2001:3900::1
Connecting to mirrors.edge.kernel.org (mirrors.edge.kernel.org)|147.75.101.1|:443... connected.
HTTP request sent, awaiting response... 304 Not Modified
File 'cache_archlinux/archlinux-bootstrap-2021.01.01-x86_64.tar.gz.sig' not modified on server. Omitting download.

  --> Preparing GnuPG to verify tarball...
gpg: key 3348882F6AC6A4C2: "Pierre Schmitz (Arch Linux Master Key) <pierre@master-key.archlinux.org>" not changed
gpg: key 5184252D824B18E8: 1 signature not checked due to a missing key
gpg: key 5184252D824B18E8: "Thomas B�chler (Arch Linux Master Key) <thomas@master-key.archlinux.org>" not changed
gpg: key 7EFD567D4C7EA887: 1 signature not checked due to a missing key
gpg: key 7EFD567D4C7EA887: "Ionut Biru (Arch Linux Master Key) <ionut@master-key.archlinux.org>" not changed
gpg: key BA1DFB64FFF979E7: 4 signatures not checked due to missing keys
gpg: key BA1DFB64FFF979E7: "Allan McRae (Arch Linux Master Key) <allan@master-key.archlinux.org>" not changed
gpg: key A04F9397CDFD6BB0: 2 signatures not checked due to missing keys
gpg: key A04F9397CDFD6BB0: "Dan McGee (Arch Linux Master Key) <dan@master-key.archlinux.org>" not changed
gpg: key 7F2D434B9741E8AC: 9 signatures not checked due to missing keys
gpg: key 7F2D434B9741E8AC: "Pierre Schmitz <pierre@archlinux.de>" not changed
gpg: Total number processed: 6
gpg:              unchanged: 6
  --> Verifying tarball...
gpg: Signature made Fri Jan  1 12:23:57 2021 MSK
gpg:                using RSA key 4AA4767BBC9C4B1D18AE28B77F2D434B9741E8AC
gpg: Good signature from "Pierre Schmitz <pierre@archlinux.de>" [unknown]
gpg: WARNING: This key is not certified with a trusted signature!
gpg:          There is no indication that the signature belongs to the owner.
Primary key fingerprint: 4AA4 767B BC9C 4B1D 18AE  28B7 7F2D 434B 9741 E8AC
  --> NB: Bootstrap tarball not newer than bootstrap directory, will use existing!
-> Image file already exists, assuming *update*...
--> Archlinux 01_install_core.sh
--> Archlinux prepare-chroot-base
  --> NB: INSTALLDIR '/home/user/qubes-builder/qubes-src/linux-template-builder/mnt' already appears to have an environment; will leave as-is!
  --> Unbinding INSTALLDIR...
umount: cache_archlinux/bootstrap/mnt: not mounted.
-> Installing package groups...
--> Archlinux 02_install_groups.sh
  --> Synchronize resolv.conf...
  --> Updating installed packages...
:: Synchronizing package databases...
 core is up to date
 extra is up to date
 community is up to date
:: Starting full system upgrade...
 there is nothing to do
  --> Installing archlinux package groups...
    --> Selected packages: xorg xterm ethtool net-tools sudo wget diffutils
:: There are 48 members in group xorg:
:: Repository extra
   1) xf86-video-vesa  2) xorg-bdftopcf  3) xorg-docs  4) xorg-font-util  5) xorg-fonts-100dpi  6) xorg-fonts-75dpi  7) xorg-fonts-encodings  8) xorg-iceauth  9) xorg-mkfontscale  10) xorg-server  11) xorg-server-common  12) xorg-server-devel  13) xorg-server-xephyr  14) xorg-server-xnest  15) xorg-server-xvfb  16) xorg-sessreg  17) xorg-setxkbmap  18) xorg-smproxy  19) xorg-x11perf  20) xorg-xauth  21) xorg-xbacklight  22) xorg-xcmsdb  23) xorg-xcursorgen  24) xorg-xdpyinfo  25) xorg-xdriinfo  26) xorg-xev  27) xorg-xgamma  28) xorg-xhost  29) xorg-xinput  30) xorg-xkbcomp  31) xorg-xkbevd  32) xorg-xkbutils  33) xorg-xkill  34) xorg-xlsatoms  35) xorg-xlsclients  36) xorg-xmodmap  37) xorg-xpr  38) xorg-xprop  39) xorg-xrandr  40) xorg-xrdb  41) xorg-xrefresh  42) xorg-xset  43) xorg-xsetroot  44) xorg-xvinfo  45) xorg-xwayland  46) xorg-xwd  47) xorg-xwininfo  48) xorg-xwud

Enter a selection (default=all): 
resolving dependencies...
looking for conflicting packages...
warning: dependency cycle detected:
warning: harfbuzz will be installed before its freetype2 dependency
warning: dependency cycle detected:
warning: mesa will be installed before its libglvnd dependency

Packages (125) db-5.3.28-5  fontconfig-2:2.13.91+48+gfcb0420-2  freetype2-2.10.4-1  gdbm-1.19-1  graphite-1:1.3.14-1  groff-1.22.4-3  harfbuzz-2.7.4-1  libdrm-2.4.104-1  libedit-20191231_3.1-3  libepoxy-1.5.5-1  libevdev-1.10.1-1  libfontenc-1.1.4-3  libglvnd-1.3.2-1  libgudev-234-1  libice-1.0.10-3  libinput-1.16.4-1  libomxil-bellagio-0.9.3-3  libpciaccess-0.16-2  libpipeline-1.5.3-1  libpng-1.6.37-3  libsm-1.2.3-2  libunwind-1.3.1-2  libutempter-1.2.1-1  libwacom-1.7-1  libx11-1.7.0-3  libxau-1.0.9-3  libxaw-1.0.13-3  libxcb-1.14-1  libxcomposite-0.4.5-3  libxcursor-1.2.0-2  libxdamage-1.1.5-3  libxdmcp-1.1.3-3  libxext-1.3.4-3  libxfixes-5.0.3-4  libxfont2-2.0.4-3  libxft-2.3.3-2  libxi-1.7.10-3  libxinerama-1.1.4-3  libxkbfile-1.1.0-2  libxmu-1.1.3-2  libxpm-3.5.13-2  libxrandr-1.5.2-3  libxrender-0.9.10-4  libxshmfence-1.3-2  libxt-1.2.0-2  libxtst-1.2.3-4  libxv-1.0.11-4  libxxf86vm-1.1.4-4  llvm-libs-11.0.1-1  lm_sensors-3.6.0-2  luit-20201003-1  man-db-2.9.3-1  mesa-20.3.3-1  mtdev-1.1.6-1  perl-5.32.0-3  pixman-0.40.0-1  vulkan-icd-loader-1.2.166-1  wayland-1.18.0-2  xbitmaps-1.1.2-2  xcb-proto-1.14.1-3  xcb-util-0.4.0-3  xcb-util-image-0.4.0-3  xcb-util-keysyms-0.4.0-3  xcb-util-renderutil-0.3.9-3  xcb-util-wm-0.4.1-3  xf86-input-libinput-0.30.0-1  xkeyboard-config-2.31-1  xorg-fonts-alias-100dpi-1.0.4-1  xorg-fonts-alias-75dpi-1.0.4-1  xorg-util-macros-1.19.2-2  xorgproto-2020.1-1  diffutils-3.7-3  ethtool-1:5.10-1  net-tools-1.60.20181103git-2  sudo-1.9.5.p1-1  wget-1.21.1-1  xf86-video-vesa-2.5.0-1  xorg-bdftopcf-1.1-2  xorg-docs-1.7.1-3  xorg-font-util-1.3.2-2  xorg-fonts-100dpi-1.0.3-7  xorg-fonts-75dpi-1.0.3-7  xorg-fonts-encodings-1.0.5-2  xorg-iceauth-1.0.8-2  xorg-mkfontscale-1.2.1-2  xorg-server-1.20.10-3  xorg-server-common-1.20.10-3  xorg-server-devel-1.20.10-3  xorg-server-xephyr-1.20.10-3  xorg-server-xnest-1.20.10-3  xorg-server-xvfb-1.20.10-3  xorg-sessreg-1.1.2-2  xorg-setxkbmap-1.3.2-2  xorg-smproxy-1.0.6-3  xorg-x11perf-1.6.1-2  xorg-xauth-1.1-2  xorg-xbacklight-1.2.3-2  xorg-xcmsdb-1.0.5-3  xorg-xcursorgen-1.0.7-2  xorg-xdpyinfo-1.3.2-4  xorg-xdriinfo-1.0.6-2  xorg-xev-1.2.4-1  xorg-xgamma-1.0.6-3  xorg-xhost-1.0.8-2  xorg-xinput-1.6.3-2  xorg-xkbcomp-1.4.4-1  xorg-xkbevd-1.1.4-3  xorg-xkbutils-1.0.4-4  xorg-xkill-1.0.5-2  xorg-xlsatoms-1.1.3-2  xorg-xlsclients-1.1.4-2  xorg-xmodmap-1.0.10-2  xorg-xpr-1.0.5-2  xorg-xprop-1.2.5-1  xorg-xrandr-1.5.1-2  xorg-xrdb-1.2.0-2  xorg-xrefresh-1.0.6-2  xorg-xset-1.2.4-2  xorg-xsetroot-1.1.2-2  xorg-xvinfo-1.1.4-2  xorg-xwayland-1.20.0.r800.ge4a9f0bb4-1  xorg-xwd-1.0.7-2  xorg-xwininfo-1.1.5-2  xorg-xwud-1.0.5-2  xterm-363-1

Total Download Size:     0.13 MiB
Total Installed Size:  355.74 MiB

:: Proceed with installation? [Y/n] 
:: Retrieving packages...
error: failed retrieving file 'net-tools-1.60.20181103git-2-x86_64.pkg.tar.xz' from mirrors.evowise.com : The requested URL returned error: 404
error: failed retrieving file 'net-tools-1.60.20181103git-2-x86_64.pkg.tar.xz' from mirror.rackspace.com : The requested URL returned error: 404
error: failed retrieving file 'net-tools-1.60.20181103git-2-x86_64.pkg.tar.xz' from mirror.rackspace.com : The requested URL returned error: 404
warning: failed to retrieve some files
error: failed to commit transaction (failed to retrieve some files)
Errors occurred, no packages were upgraded.
make[1]: *** [Makefile:65: rootimg-build] Error 1
iamahuman commented 3 years ago

@IDBEHOLDS See: #816.

RebornRider commented 3 years ago

Okay, I've got the archlinux template building and working.

I had to do 4 things:

  1. specify http://mirrors.kernel.org/ as the sole mirror in prepare-chroot-base
  2. add the pulseaudio v14.2 dependency as per https://github.com/Qubes-Community/Contents/blob/master/docs/building/building-archlinux-template.md#missing-pulsecore-error-when-building-the-gui-agent-linux
  3. change depends=('alsa-lib' 'alsa-utils' 'pulseaudio-alsa' 'pulseaudio<14.1') to pulseaudio<14.3 in archlinux/PKGBUILD (line 74) see: https://github.com/QubesOS/qubes-gui-agent-linux/commit/291f697eb1a92f83954d75ee9d180906bda4ceed
  4. fix broken Passwordless Root Access by running qvm-run -u root --pass-io archlinux 'leafpad /etc/sudoers.d/qubes' and replacing %qubes ALL=(ALL) ROLE=unconfined_r TYPE=unconfined_t NOPASSWD: ALL with %qubes ALL=(ALL) NOPASSWD: ALL Otherwise trying to access sudo would result in the error: 
    /etc/sudoers.d/qubes:2:22: syntax error
%qubes ALL=(ALL) ROLE=unconfined_r TYPE=unconfined_t NOPASSWD: ALL
kevinjohna6 commented 3 years ago

Since the 404'ing packages are not found in any of the default remote mirrors, the local package DBs must be out of date. So I don't think it has to do with using any specific mirror.

My guess is that this could be fixed by always calling pacman with the -y or -yy flags to force update the DBs, or, maybe more robustly, downloading all the needed packages to a local cache "atomically".

Just for reference, I was also able to get the template to build by exclusively using an archlinux package cache. (which also keeps 3 old versions of all packages)

--- a/prepare-chroot-base
+++ b/prepare-chroot-base
@@ -57,6 +57,7 @@ do
   ARCHLINUX_MIRRORLIST=$(sed "s|#Server = ${MIRROR_ENTRY}/|Server = ${MIRROR_ENTRY}/|" <<< $ARCHLINUX_MIRRORLIST)
 done
 echo "$ARCHLINUX_MIRRORLIST" > "${CACHEDIR}/bootstrap/etc/pacman.d/mirrorlist"
+echo 'Server = http://10.99.99.99/archlinux/$repo/os/$arch' > "${CACHEDIR}/bootstrap/etc/pacman.d/mirrorlist"
 cp /etc/resolv.conf "${BOOTSTRAP_DIR}/etc/"
unman commented 3 years ago

Currently, the only blocker is in vmm-xen - otherwise the template builds correctly. This can be closed.

When I have seen this sort of error, it invariably relates to transitory errors in mirroring - that's why there is a 404 , because (as suggested) the server provides only an early or later version from that in the package list.