newly added disabled option is assumed missing

igalic commented 12 months ago

Prerequisites

[ ] Have you checked for an existing issue describing your problem?
[x] Are you running the latest version?
[x] Is your ports tree recent?
[x] Is your FreeBSD Host on a supported release?

Describe the bug

I would like to never build rust or llvm or any other thing I don't need to build other than the port I ask for.

i set the following in /usr/local/etc/poudriere.conf:

PACKAGE_FETCH_WHITELIST="gcc* rust* llvm* py*-cryptography *-rust"

yet, when trying to build net/cloud-init-devel (a python project), poudriere "recognizes" that rust has changed (see https://github.com/freebsd/freebsd-ports/commit/098de5bc2195a064523eb32e7db6071b2f8ed93a )

poudriere testport output

``` [00:00:16] Calculating ports order and dependencies [00:00:16] Trimming IGNORED and blacklisted ports [00:00:16] Package fetch: Looking for missing packages to fetch from pkg+https://pkg.FreeBSD.org/${ABI}/latest [00:00:16] Package fetch: Skipping py39-oauthlib-3.2.2: not in whitelist [00:00:16] Package fetch: Skipping py39-hatch-fancy-pypi-readme-22.8.0_2: not in whitelist [00:00:16] Package fetch: Skipping py39-hatch-vcs-0.3.0_2: not in whitelist [00:00:16] Package fetch: Skipping py39-hatchling-1.18.0: not in whitelist [00:00:16] Package fetch: Skipping py39-pyjwt-2.8.0: not in whitelist [00:00:16] Package fetch: Skipping meson-1.2.2: not in whitelist [00:00:16] Package fetch: Skipping public_suffix_list-20231001: not in whitelist [00:00:16] Package fetch: Skipping py39-maturin-1.3.0: not in whitelist [00:00:16] Package fetch: Skipping py39-trove-classifiers-2023.9.19: not in whitelist Updating FreeBSD repository catalogue... pkg-static: Repository FreeBSD has a wrong packagesite, need to re-create database [14-amd64-ports-cloudinit] Fetching meta.conf: 100% 163 B 0.2kB/s 00:01 [14-amd64-ports-cloudinit] Fetching packagesite.pkg: 100% 7 MiB 1.4MB/s 00:05 Processing entries: 100% FreeBSD repository update completed. 34000 packages processed. All repositories are up to date. [00:00:32] Package fetch: Skipping rust-1.72.0: options wanted: +DOCS -GDB -LTO -PORT_LLVM +SOURCES +WASM [00:00:32] Package fetch: Skipping rust-1.72.0: options remote: +DOCS -GDB -LTO +SOURCES +WASM [00:00:32] Package fetch: Skipping py39-cryptography-41.0.4,1: deps wanted: py39-cffi-1.16.0 python39-3.9.18 [00:00:32] Package fetch: Skipping py39-cryptography-41.0.4,1: deps remote: py39-cffi-1.15.1 python39-3.9.18 [00:00:32] Package fetch: Will fetch py39-setuptools-rust-1.7.0 [00:00:32] Package fetch: Will fetch 1 packages from remote or local pkg cache No packages are required to be fetched. Integrity check was successful. [00:00:32] Package fetch: Using cached copy of py39-setuptools-rust-1.7.0 [00:00:32] Sanity checking the repository [00:00:32] Checking for stale cache files [00:00:32] Checking packages for incremental rebuild needs [00:00:32] Checking packages for missing dependencies [00:00:32] Deleting py39-setuptools-rust-1.7.0.pkg: missing dependency: rust-1.72.0 [00:00:32] Deleting stale symlinks... done [00:00:32] Deleting empty directories... done [00:00:32] Package fetch: Generating logs for fetched packages [00:00:32] Unqueueing existing packages [00:00:32] Unqueueing orphaned build dependencies [00:00:32] Sanity checking build queue [00:00:33] Processing PRIORITY_BOOST [00:00:33] Balancing pool [00:00:33] Recording filesystem state for prepkg... done [00:00:33] Building 5 packages using up to 2 builders [00:00:33] Hit CTRL+t at any time to see build progress and stats [00:00:33] [01] [00:00:00] Builder starting [00:00:43] [01] [00:00:10] Builder started [00:00:43] [01] [00:00:00] Building lang/rust | rust-1.72.0 [00:00:43] [01] [00:00:00] Status lang/rust | rust-1.72.0: check-sanity [00:00:43] [01] [00:00:00] Status lang/rust | rust-1.72.0: pkg-depends ^C[00:00:45] Error: Signal SIGINT caught, cleaning up and exiting ```

the most important part from that:

[00:00:32] Package fetch: Skipping rust-1.72.0: options wanted: +DOCS -GDB -LTO -PORT_LLVM +SOURCES +WASM
[00:00:32] Package fetch: Skipping rust-1.72.0: options remote: +DOCS -GDB -LTO +SOURCES +WASM

Note that this issue is not fixed by #1064.

How to reproduce

This is tricky to reproduce, either you need to time it correctly, or you need to introduce new disabled options to a port that other ports (build) depend on.

Expected behavior

I never ever want to build a compiler that takes hours to build on my tiny VMs. Never.

Environment

Host OS: 13.2-RELEASE-p2
Jail OS: 14.0-BETA5 amd64
Poudriere Version: tip of master, including #1064, #1076, #1065 .
Ports branch and revision: tip of main, + a patch to upgrade cloud-init.

Additional context

bdrewery commented 5 months ago

I get the frustration here but I don't see a safe assumption. If we want PORT_LLVM to be disabled and the remote does not list the option, I assume you assume that means it is disabled. What if the option was removed and it was enabled by default? It could go either way. We need exact option matching to be safe.

If however you have no PORT_DBDIR dir / (selected options) for this port I could see not caring and just fetching. But why would an option be in remote/local and have the same version? Seems wrong to begin with.

I think what you're really asking for is a NEVER_BUILD_ALWAYS_FETCH list or FETCH_REGARDLESS_OF_OPTIONS. Even this won't capture the case where a remote version/deps differ. #822 will help in this case are dep versions won't matter as much. Point being at some case Poudriere will want to build Rust. So maybe you really want a NEVER_BUILD list. That sounds frustrating to use itself.

bdrewery commented 5 months ago

I think #1129 is what is being asked for here. If so please close this, or let me know you agree.

freebsd / poudriere