search

freebsd / poudriere

Port/Package build and test system
https://github.com/freebsd/poudriere/wiki
BSD 2-Clause "Simplified" License
379 stars 161 forks source link

Avoid unnecessary package building with `-b` by removing duplicate options and dependencies #1064

Closed patmaddox closed 8 months ago

patmaddox commented 11 months ago

Duplicate options e.g. "+FOO +FOO" was causing the wanted and remote options comparison to fail, causing poudriere to build packages unnecessarily.

Duplicate dependencies e.g. "foo/bar foo/bar" was causing the wanted and remote deps comparison to fail, causing poudriere to skip packages.

Building this list of 59 ports with -n:

before:

[131amd64-default] [2023-07-27_00h07m28s] [load_priorities:] Queued: 721 Built: 0 Failed: 0 Skipped: 0 Ignored: 0 Fetched: 263 Tobuild: 458 Time: 00:00:41

after:

[131amd64-default] [2023-07-27_00h21m24s] [load_priorities:] Queued: 716 Built: 0 Failed: 0 Skipped: 0 Ignored: 0 Fetched: 701 Tobuild: 15 Time: 00:03:40

ftp/curl is a good example:

[00:00:12] [Dry Run] Package fetch: Skipping libssh2-1.11.0,3: options wanted: -GCRYPT -TRACE +ZLIB [00:00:12] [Dry Run] Package fetch: Skipping libssh2-1.11.0,3: options remote: -GCRYPT -GCRYPT -TRACE -TRACE +ZLIB +ZLIB ... [00:00:13] [Dry Run] Package fetch: Skipping meson-1.1.1: deps wanted: ninja-1.11.1,2 python39-3.9.17 [00:00:13] [Dry Run] Package fetch: Skipping meson-1.1.1: deps remote: ninja-1.11.1,2 ninja-1.11.1,2 python39-3.9.17 python39-3.9.17

patmaddox commented 11 months ago

This issue is not due to an error in the script, but because pkg-rquery is returning duplicates:

$ pkg rquery '%n %Ok %Ov' libssh2 # src/share/poudriere/common.sh:3834
libssh2 GCRYPT off
libssh2 TRACE off
libssh2 ZLIB on
libssh2 GCRYPT off
libssh2 TRACE off
libssh2 ZLIB on

$ pkg rquery '%n %dn-%dv' meson # src/share/poudriere/common.sh:3838
meson python39-3.9.17
meson ninja-1.11.1,2
meson python39-3.9.17
meson ninja-1.11.1,2
igalic commented 11 months ago

see https://github.com/freebsd/pkg/issues/2164