options command generating errors about missing ports tree FLAVOR feature

madpilot78 commented 8 months ago

Prerequisites

[x] Have you checked for an existing issue describing your problem?
[x] Are you running the latest version?
[x] Is your ports tree recent?
[x] Is your FreeBSD Host on a supported release?

Describe the bug

With latest ports-mgmt/poudriere-devel port running the poudriere options command passinga port with a flavor (f.e. net/unison@nox11) causes errors like:

[00:00:00] Error: Trying to build FLAVOR-specific net/unison@nox11 but ports tree has no FLAVORS support.

And the port is skipped.

How to reproduce

see above

Expected behavior

No errors being output. No skipping

Environment

Host OS [e.g. 12.2 amd64]: head
Jail OS [e.g. 12.0 powerpc]: any
Poudriere Version [e.g. 3.3.1 or git hash or port version]: 3.4.99.20231030
Ports branch and revision [e.g. 2020Q3 r550754]: head

Additional context

After further investigation, when running the options command, the P_PORTS_FEATURES variable is not being populated via the options command.

If I get more insight to share I'll do.

vanvoljg commented 8 months ago

I'm having the same trouble, different port with a flavor.

[00:00:00] Error: Trying to build FLAVOR-specific graphics/ImageMagick7@nox11 but ports tree has no FLAVORS support.

Host OS: 13.2-RELEASE-p4 amd64
Jail OS: Same
Poudriere version 3.4.99.20231030
Ports branch and revision: head

This checkout of the ports tree does support flavors, though:

# make -f /usr/local/poudriere/ports/head/Mk/bsd.port.mk -V PORTS_FEATURES
FLAVORS

I do note that I'm not seeing any references to SUBPACKAGE/S (or SUBPKG/S) in my ports tree other than ports-mgmt/portfmt, and the feature is not available for this ports tree, so.... I'm wondering if the issue is really more about poudriere incorrectly handling a missing feature in the ports tree it's using. (This is entirely a gut feeling/guess; I'm so unfamiliar with the codebase that it's not even funny, so please forgive my ignorance.)

There's supposedly a test for empty $P_PORTS_FEATURES (https://github.com/freebsd/poudriere/blob/master/test/common.bulk.sh#L696), but maybe there's something not correct in the test runner, because the fetch_global_port_vars function as used in Poudriere is not populating the variable correctly, but the tests seem to pass.

Again, I'm sorry if I missed something that requires context to understand, this is the first time I've really looked into the repo.

madpilot78 commented 8 months ago

The SUBPACKAGES feature is still being worked on, support needs to be added in poudriere before it can be unleashed on the ports tree.

But we should be getting it at some point.

Looks like the bug was introduced when adding support for SUBPACKAGES and not updating the options.sh script, but looking at it I found nothing obvious about how to fix this.

vanvoljg commented 8 months ago

I saw it was still in progress, yeah. :slightly_smiling_face:. Been over a month and a half without movement, so I just assumed it was going to be a "some day" kind of feature.

It definitely feels like a miss when adding support for SUBPACKAGES -- I don't think added support was intended to be breaking.

Seems like two things need to be done:

Handle missing ports features gracefully. Seems to me that a missing expected feature should either be reported and ignored, but shouldn't make poudriere skip the port like this. It should fall back to the same behaviour it had before feature support was added.
Correctly test population of $P_PORTS_FEATURES variable.

Like you, I can't really tell what the correct way is to go about fixing the way poudriere handles a missing feature. And I'm all the way lost in this shell script testing framework.

I'm fine with simple scripts, but as soon as we get to multiple directories and sources, I'm lost in the weeds. Poudriere is a marvel of complex shell scripts that my tiny brain can't hold enough of to make any sense. Anything more than a couple screens worth is too much for me.

madpilot78 commented 8 months ago

Handle missing ports features gracefully.

This is in theory already done. Obviously if you ask for something that depends on a feature (like a specific flavor) there is not much to be done, skipping it is correct. Reverting on the unflavored port looks arbitrary.

Anyway the only issue at hand is the options subcommand not properly populating $P_PORTS_FEATURES.

I tried to figure it out but could not find a solution. I plan to find some spare time in the near future to try harder.

madpilot78 commented 8 months ago

I have created a pull request with a patch that works fine for me.

You can test it and report back.

Find it here: #1087

Thanks!

ahktenzero commented 8 months ago

I updated poudriere-devel to 3.4.99.20231113 which includes the changes in #1087 and the options command now fails with an error saying the port does not exist for any port I try. It's looking for the port in /usr/ports (which is empty on my system) and not the ports tree I use for building packages at /usr/local/poudriere/ports/default.

After adding some debug output, the problem is that jail_start sets PORTSDIR to /usr/ports, which is correct for commands running inside the jail as the ports tree gets mounted there, but in options.sh the commands to check and update the options are run outside the jail so it ends up looking at the wrong ports tree.

I've uploaded a PR with a fix, #1089.

madpilot78 commented 8 months ago

@ahktenzero Thanks for noticing my mistake, I clearly missed this detail of jail_start.

bapt commented 7 months ago

https://bapt.nours.eu/patch-fix-options-for-flavors.diff this should fix this without having to spawn a jail by default

madpilot78 commented 7 months ago

https://bapt.nours.eu/patch-fix-options-for-flavors.diff this should fix this without having to spawn a jail by default

@bapt I've tested it with my common usage patterns and it looks fine, does what is expected.

madpilot78 commented 6 months ago

Fixed by 9143ab494773f7a1ba69efc31177d4bbfe51bf7f

freebsd / poudriere