michael-o
commented
1 year ago @bdrewery As promised, here is the issue to improve -b.
Open michael-o opened 1 year ago
michael-o
commented
1 year ago @bdrewery As promised, here is the issue to improve -b.
michael-o
commented
10 months ago @bdrewery gave it some thoughts today since I don't want to live with a patch forever. Here is my idea how we could move this forward: As far as I understand only a single repo is supported since packagesite is used throughout, would be much more work to rewrite to multiple -b options or supplying an external file. Compromise: If a URL is passed, don't treat is as official, disable FreeBSD repo and add a repo custom using that supplied URL.
WDYT? I'd be willing to work on a PR if this is acceptable for you...
Lengthly edit: I wanted to figure out whether poudriere would support a multi-repo setup. Here is what it looks like now and what I would like to have ideally in the future:
graph
subgraph Wish
direction BT
FreeBSD_w[FreeBSD] --> ldadw-base_w[ldadw-base] & ldadocgen_w[ldadocgen] & general_w[general]
ldadw-base_w --> ldadocgen_w & general_w
end
subgraph Possible
direction BT
subgraph Currently Used
direction BT
FreeBSD_cu[FreeBSD] --> ldadw-base_cu[ldadw-base]
ldadw-base_cu[ldadw-base] --> ldadocgen_cu[ldadocgen]
end
subgraph ldadw-base
direction BT
FreeBSD_p_lb[FreeBSD] --> ldadw-base_p_lb[ldadw-base]
end
subgraph ldadocgen
direction BT
FreeBSD_p_ldg[FreeBSD] --> ldadocgen_p_ldg[ldadocgen]
end
subgraph general
direction BT
FreeBSD_p_g[FreeBSD] --> general_p_g[general]
end
endI hacked common.sh and added additionally to the default official repo for -b a custom repo in a a single conf file. Immediately https://github.com/freebsd/poudriere/blob/abc58e8177fe669b200d0d0f6bd12c70c9086a6c/src/share/poudriere/common.sh#L3831 failed with https://github.com/freebsd/pkg/issues/2201.
My "Wish" is not possible at all without making poudriere supporting such a multi-repo/multi-inheritance setup. My current setup is flawed as well because is is very limited. As far as I can see, I can only go with the middle setup for now, wasting cycles by rebuilding packages with custom options :-(
I think that issue could be closed, but I would like to keep it open for reference for others at least until #798 has been merged and tested by people.
yannk
commented
1 week ago I found this issue on my way to open my own.
I came up with the same type of fix:
FreeBSD: {
url: ${packagesite};
mirror_type: NONE;
}I use -b https://myownbinaryrepo
Seems like an easy fix would be to detect that -b is set to an URL and disable SRV in that case
michael-o
commented
1 week ago I found this issue on my way to open my own.
I came up with the same type of fix:
FreeBSD: { url: ${packagesite}; mirror_type: NONE; }I use
-b https://myownbinaryrepoSeems like an easy fix would be to detect that
-bis set to an URL and disableSRVin that case
That was my observation as well, but does not solve my wish situation where still only one repo is supported.
Prerequisites
What is your proposal?
Remove the terms "official only" from this option and if any of the symbolic names,
latest,quarterly,release_Xkeep repo nameFreeBSDand enforce SRV RR, signatures, but when a custom URL is provided, setFreeBSDtono, add a new repo, e.g.,customwhere priority is higher (10?) and hardcodemirror_type,pubkey,signature_typetoNONE. A more sophisticated way is to accept a file path where the admin can supply custom.confwhich contains arbitrary repos copied toLOCALBASEinto the jail. In this case it needs to be clarified how to provide public keys, if required. (This will provide nice layering of repos)What is the existing behavior, if any?
When providing the custom, selfhosted repo with
-b <URL>the fetch fails because my packages aren't signed which I don't need nor is this an official repo.What is the motivation / use case for the change?
I build a base set of ports with custom options because the default ones do not work for our environment. Those options aren't only for direct, but also for automatic ones. On top there are several other use cases (sets) which use those base ports partially as dependency which I don't want to build over and over again. The option for
bulk/testport-b <URL>expects only official repos which quite limits this nice feature. I want to use it with any selfhosted repo which will satisfy the fetch requirements.Did you consider any alternatives?
Yes, patching
common.sh, but this is an ugly hack. More specificly, in https://github.com/freebsd/poudriere/blob/5f69efba7802c412193cecf2d00a34fd77bb56fd/src/share/poudriere/common.sh#L3805-L3809 I had to switch of signature check and DNS SRV RRs to use my custom repo. Hack:Is this really a ports feature request?
Yes, it is a poudriere issue.
Additional context
This was found in https://github.com/freebsd/poudriere/issues/883.
Real world example
port set
ldadw_base:with the following in
/usr/local/etc/poudriere.d/:123-release-amd64-default-ldadw_base-ports:Available internally at
https://deblndw011x.ad001.siemens.net/FreeBSD/packages/${ABI}/ldadw-baseNow for a specific purpose bastille jail I have the following:
with the following in
/usr/local/etc/poudriere.d/:123-release-amd64-default-ldadocgen-ports:123-release-amd64-default-ldadocgen-options/contains merged options fromldadw_baseandldadocgento satisfy-b's requirements. Many of the ports inldadocgenwill use a lot fromldadw_base, e.g., apache24, tomcat85, tomcat-native will use apr which inbasehas been pulled bysubversionalready along with many other build dependencies , e.g., ninja, cmake, GNU autotools, etc.