jail install from pkgbase fails if 'freebsd-git-devtools' is in the host's package repository

llfw commented 6 months ago

Prerequisites

[X] Have you checked for an existing issue describing your problem?
[3.4.1] Are you running the latest version?
[N/A] Is your ports tree recent?
[15.0-CURRENT] Is your FreeBSD Host on a supported release?

Describe the bug

installing a jail from pkgbase fails if the 'freebsd-git-devtools' package is available:

# poudriere jail -c -j main -v 15 -a amd64 -m pkgbase=https://pkg.eden.le-fay.org/base
[00:00:00] Creating main fs at /poudriere/jails/main... done
[00:00:00] Installing 15 amd64 from https://pkg.eden.le-fay.org/base ...Updating pkgbase repository catalogue...
Fetching meta.conf: 100%    163 B   0.2kB/s    00:01
Fetching packagesite.pkg: 100%   36 KiB  37.3kB/s    00:01
Processing entries: 100%
pkgbase repository update completed. 514 packages processed.
All repositories are up to date.
Updating pkgbase repository catalogue...
pkgbase repository is up to date.
All repositories are up to date.
pkg: No packages available to install matching 'freebsd-git-devtools-20230925_1' have been found in the repositories
[00:00:01] Recording filesystem state for clean... done
[00:00:01] Jail main 15 amd64 is ready to be used
#

this is because jail.sh tries to use a package name regexp to match base packages. a more reliable method is to use the package origin, which is base for pkgbase packages.

quick and dirty fix:

--- jail.sh.orig    2024-03-23 18:34:56.012333000 +0000
+++ jail.sh 2024-03-23 18:49:43.894047000 +0000
@@ -859,7 +859,7 @@

    pkg -o REPOS_DIR="${JAILMNT}/etc/pkg" -r ${JAILMNT}/ update
    # Omit the man/debug/kernel and tests packages, uneeded for us.
-   pkg search -qx '^FreeBSD-.*' | grep -vE -- '-man|-dbg|-kernel-|-tests' | xargs pkg -o REPOS_DIR="${JAILMNT}/etc/pkg" -r ${JAILMNT}/ install -y
+   pkg -o REPOS_DIR="${JAILMNT}/etc/pkg" -r ${JAILMNT}/ search -q -Lname -Sorigin '^base' | grep -vE -- '-man|-dbg|-kernel-|-tests' | xargs pkg -o REPOS_DIR="${JAILMNT}/etc/pkg" -r ${JAILMNT}/ install -y
    if [ -n "${KERNEL}" ]; then
        pkg -o REPOS_DIR="${JAILMNT}/etc/pkg" -r ${JAILMNT}/ install -y FreeBSD-kernel-"${KERNEL}" || \
            err 1 "Failed to install FreeBSD-kernel-${KERNEL}"

How to reproduce

i don't have a reproducer command because i don't see any way at all to install from pkgbase with a publicly accessible repository; poudriere expects the repository to be at '/latest' but the pkg.freebsd.org repository is at '/base_latest'.

Expected behavior

the jail is installed.

Screenshots

N/A

Environment

Host OS [e.g. 12.2 amd64]: 15.0 amd64
Jail OS [e.g. 12.0 powerpc]: 15.0 amd64
Browser: [e.g. chrome, safari]: N/A
Poudriere Version [e.g. 3.3.1 or git hash or port version]: 3.4.1
Ports branch and revision [e.g. 2020Q3 r550754]: N/A (no ports tree involved)

igalic commented 5 months ago

i didn't realize that pkg-search -x is case insensitive by default for a less intrusive fix, you can add -C, but I think looking at origin is probably better

evadot commented 5 months ago

It's probably fixed already in git as we -o REPOS_DIR and -r ${JAILMNT} for the pkg search, can you test please ?

llfw commented 5 months ago

i tested with master (a656392b661fd309f71d9ce0884853663f1baaf1) and got a different error that i don't understand:

# ~lexi/tmp/poudriere/bin/poudriere jail -c -j test -v 15 -a amd64 -m pkgbase=https://pkg.eden.le-fay.org/base
[00:00:00] Creating test fs at /poudriere/jails/test... done
[00:00:00] Installing 15 amd64 from  ...pkg: invalid scheme /FreeBSD:15:amd64/https
pkg: Cannot parse configuration file!
pkg: invalid scheme /FreeBSD:15:amd64/https
pkg: Cannot parse configuration file!
pkg: invalid scheme /FreeBSD:15:amd64/https
pkg: Cannot parse configuration file!
[00:00:00] Recording filesystem state for clean... done
[00:00:00] Jail test 15 amd64 is ready to be used
Exiting with status 0

it seems like the pkg repo has been created wrongly:

# cat /poudriere/jails/test/etc/pkg/pkgbase.conf
pkgbase: {
  url: "/FreeBSD:15:amd64/https://pkg.eden.le-fay.org/base"
  enabled: yes
}

has the command syntax for creating pkgbase jails changed?

llfw commented 5 months ago

seems like this problem is from jail.sh:

install_from_pkgbase() {
        msg_n "Installing ${VERSION} ${ARCH} from ${SOURCES_URL} ..."
        mkdir -p "${JAILMNT}/etc/pkg"
        cat <<EOF > "${JAILMNT}/etc/pkg/pkgbase.conf"
pkgbase: {
  url: "${SOURCES_URL}/FreeBSD:${VERSION}:${ARCH}/${PKGBASEREPO}"
  enabled: yes
}
EOF

$SOURCES_URL is blank, while $PKGBASEREPO contains the pkgbase URL.

llfw commented 5 months ago

okay, i got it to work with this command:

# poudriere jail -c -j test -v 15 -a amd64 -m pkgbase=latest -U https://pkg.eden.le-fay.org/base

and this seems to install base packages properly.

evadot commented 5 months ago

okay, i got it to work with this command:
# poudriere jail -c -j test -v 15 -a amd64 -m pkgbase=latest -U https://pkg.eden.le-fay.org/base
and this seems to install base packages properly.

That's the intended syntax yes. Shall we close this bug then ?

llfw commented 5 months ago

this one can be closed (i wasn't sure if you prefer to wait for a release before closing bugs) but should i open one for the error when -U isn't specified? that could be a bit more clear about the right syntax.

bdrewery commented 5 months ago

Looks like this was fixed in 07f9a730f9d9b7e45c9ccfc4b78119bc62bd38ab. I added some validations for -U and updated the docs too.

grahamperrin commented 5 months ago

# poudriere jail -c -j test -v 15 -a amd64 -m pkgbase=latest -U https://pkg.eden.le-fay.org/base

Thanks!

I became blind to the 15 above after repeatedly keying 15.0-CURRENT based on the manual page example,

… version should in the form of “12.0-RELEASE”. …

I'm good, now:

poudriere jail -c -v 15 -j main -m pkgbase=base_latest -U https://pkg.freebsd.org/

freebsd / poudriere