Qubes Arch repositories are not enabled in Arch templates

[unman]

Qubes OS version

4.0

Affected component(s) or functionality

Builder

Brief summary

The current build of Arch templates does not include Qubes repositories although official packages are provided at https://archlinux.qubes-os.org. The Qubes signing key is not included in the template.

How Reproducible

100%

To Reproduce

Steps to reproduce the behavior:

1. Build template
2. Check Repository lists

Expected behavior

Official repositories will be available and usable out of the box

Actual behavior

No Qubes repository enabled, and signing key not enabled in the template

Solutions you've tried

Adding repository and enabling signing key - PR on way

Relevant documentation you've consulted

Related, non-duplicate issues

[DemiMarie]

As far as I can tell, the official Qubes Arch repositories haven’t been building new packages in a while. @marmarek @fepitre is that intentional?

[lubellier]

The ArchLinux template now includes the qubes repositories. The CI restarts to publish the ArchLinux packages see the update status with archlinux tag and the updated repository with 2022 versions of qubes-db-vm, qubes-vm-core, qubes-vm-gui, ...

[andrewdavidwong]

The ArchLinux template now includes the qubes repositories. The CI restarts to publish the ArchLinux packages see the update status with archlinux tag and the updated repository with 2022 versions of qubes-db-vm, qubes-vm-core, qubes-vm-gui, ...

Does this mean that this issue should now be closed, or is there still remaining work to be done? (If the latter, what remains to be done?)

[lubellier]

Does this mean that this issue should now be closed, or is there still remaining work to be done? (If the latter, what remains to be done?)

We should wait the point of view of the issue creator (@unman ) or a feedback from the core team.

[marmarek]

There are still some build issues, not all packages are there yet: https://ftp.qubes-os.org/~qubesos2-bot/status.html

[unman]

Until the repository is complete (and signed) I think it is too early to close.

[lubellier]

I checked the last build-issue from the CI Build failed: vmm-xen for vm (4.1 archlinux) #2734, the error in the log is about adding the already present packages in qubes.db.tar.gz ($(ls -v pkgs/*.pkg.tar.*); do repo-add pkgs/qubes.db.tar.gz "$pkg"; done;'):

2022-01-25 12:30:44.387155 +0000 build-archlinux: + systemd-nspawn --directory=/home/user/builder-r4.1/chroot-vm-archlinux --keep-unit --register=no --bind=/home/user/builder-r4.1/qubes-packages-mirror-repo/vm-archlinux:/tmp/qubes-packages-mirror-repo --chdir=/tmp/qubes-packages-mirror-repo --user=user /bin/sh -c 'for pkg in `ls -v pkgs/*.pkg.tar.*`; do repo-add pkgs/qubes.db.tar.gz "$pkg"; done;'
2022-01-25 12:30:44.395243 +0000 build-archlinux: Spawning container chroot-vm-archlinux on /home/user/builder-r4.1/chroot-vm-archlinux.
2022-01-25 12:30:44.395333 +0000 build-archlinux: Press ^] three times within 1s to kill container.
2022-01-25 12:30:44.677228 +0000 build-archlinux: .[1m.[32m==>.[0m.[1m Extracting qubes.db.tar.gz to a temporary location....[0m.
2022-01-25 12:30:44.688524 +0000 build-archlinux: .[1m.[32m==>.[0m.[1m Extracting qubes.files.tar.gz to a temporary location....[0m.
2022-01-25 12:30:44.695509 +0000 build-archlinux: .[1m.[32m==>.[0m.[1m Adding package 'pkgs/pacman-5.2.2-1-x86_64.pkg.tar.zst'.[0m.
2022-01-25 12:30:44.741674 +0000 build-archlinux: /usr/sbin/repo-add: line 204: /dev/fd/63: No such file or directory.
2022-01-25 12:30:44.746788 +0000 build-archlinux: .[1m.[31m==> ERROR:.[0m.[1m Invalid package file 'pkgs/pacman-5.2.2-1-x86_64.pkg.tar.zst'..[0m.
2022-01-25 12:30:44.748353 +0000 build-archlinux: .[1m.[32m==>.[0m.[1m No packages modified, nothing to do..[0m.

Where come from these packages? Is it possible to test without these already presents packages?

The package build on my qubes-builder works, I'm using the building-archlinux-template guide:

user@builder qubes-builder]$ make vmm-xen-vm 
Currently installed dependencies:
createrepo_c-0.17.5-1.fc34.x86_64
createrepo_c-0.17.5-1.fc34.x86_64
debootstrap-1.0.126-1.nmu1.fc34.noarch
dialog-1.3-30.20210117.fc34.x86_64
dpkg-dev-1.21.1-1.fc34.noarch
git-2.34.1-1.fc34.x86_64
perl-Digest-MD5-2.58-2.fc34.x86_64
perl-Digest-SHA-6.02-459.fc34.x86_64
python3-sh-1.14.1-2.fc34.noarch
rpm-build-4.16.1.3-1.fc34.x86_64
rpmdevtools-9.5-1.fc34.noarch
--> Archlinux dist-prepare-chroot (makefile):
-> Building vmm-xen (archlinux) for archlinux vm (logfile: build-logs/vmm-xen-vm-archlinux.log)
--> Done:
      qubes-src/vmm-xen/pkgs/vm-archlinux/qubes-vm-xen-4.14.3-8-x86_64.pkg.tar.zst
[user@builder qubes-builder]$ 

I try to understand why I don't get the same issue with my qubes-builder build (vs the CI build)? Some advises ? I want to reproduce the CI build issue in my environment.

[techge]

I have manually added the repo some month ago and it is working fine so far. We probably could use some more eyes on testing and build issues, sure, but as it is a community template anyway, that should be fine.

It seems to be worse to have the repo not enabled automatically than missing some packages, right? Thus, I suggest enabling it as OP asked for. If the only reason this haven't done yet is a missing PR I am willing to provide it.

Is there something else blocking that I could try to look into? Or shall I create a PR for enabling the repo?

[DemiMarie]

Is there something else blocking that I could try to look into?

Nope!

Or shall I create a PR for enabling the repo?

Do it :)

[unman]

I understood that Marek did not want to do this while there were missing packages - that hasn't changed. The PR we need is to build those missing packages.

[ejose19]

I've compared a recent make qubes-vm build output packages with https://ftp.qubes-os.org/repo/archlinux/r4.1/current/vm/archlinux/pkgs/ and noticed only qubes-vm-gui-common is missing from qubes repo (although there's an outdated version in current-testing repo), is the current state still a blocker to include qubes repo and its key into built templates? If not then I'd like to work on a PR.