search

juhp / fbrnch

Tool to update fedora packages branches
https://hackage.haskell.org/package/fbrnch
GNU General Public License v2.0
23 stars 3 forks source link

Bootstrap bcond management #5

Closed QuLogic closed 3 years ago

QuLogic commented 4 years ago

Related to #4, we do have a lot of packages with bootstrap conditionals. I would like to see if these are effective, so it'd be nice if fbrnch could calculate that for me.

So something like fbrnch sort --with bootstrap could process the spec files with the bootstrap bcond on, and thus with limited dependencies. Then I could see what is missing a bootstrap.

Relatedly, it'd be nice if building with bootstrap could be automated. The nice thing about bootstrap is it will automatically add ~bootstrap to the version-release (which sorts older), so there's no need to mess with releases or changelog. fbrnch parallel could build once with %bcond_without bootstrap, then flip the line to %bcond_with bootstrap and build again after the first build set is done. Maybe bonus points if it noticed some things already had boostrap on and didn't mess with them.

juhp commented 4 years ago

Thanks for this good suggestion: in fact I remember someone making a similar request when I first announced rpmbuild-order to devel list (and did some tests with all-specs, etc).

I have implemented this first for rpmbuild-order-0.4.2.

The nice thing about bootstrap is it will automatically add ~bootstrap to the version-release (which sorts older), so there's no need to mess with releases or changelog.

Is that R Packaging specific or a general thing?

fbrnch parallel could build once with %bcond_without bootstrap, then flip the line to %bcond_with bootstrap and build again after the first build set is done. Maybe bonus points if it noticed some things already had boostrap on and didn't mess with them.

Aha interesting. Should it be specific to R packages? Or perhaps it needs some option to trigger, probably not all SIGs or languages would want such behavior by default?

juhp commented 4 years ago

I pushed this to fbrnch-0.2-6 which is building now in Copr.

With that, you should be able to do fbrnch sort -w bootstrap -b master * for example. It does break your main cycle into lots of smaller cycles at least. :)

QuLogic commented 4 years ago

Bootstrapping is general: https://docs.fedoraproject.org/en-US/packaging-guidelines/#bootstrapping

I guess maybe an option to start? I don't really use a full re-bootstrapped build too much; I guess it's more useful for arch-spinup or when a dependency breaks soname or similar (so that no package in the cycle could be installed).

QuLogic commented 4 years ago

Relatedly, bootstrap is not understood properly for wait-repo in fbrnch parallel. With some other new dependencies, I built R-isoband and R-magick. These have loops with other new things, so they have boostrap on.

The first time around, it got stuck waiting for R-isoband-0.2.2-1.fc33 and R-magick-2.4.0-1.fc33 to appear in f33-build, so I cancelled it. Then running again, it said:

R-isoband-0.2.2-1.fc33 is already latest (modulo disttag)
** R-isoband job failed ** (1 jobs left)
R-magick-2.4.0-1.fc33 is already latest (modulo disttag)
** R-magick job failed ** (0 jobs left)
fbrnch: Build failures: R-magick R-isoband

That's because those jobs have bootstrap enabled, and so the disttag has ~bootstrap appended.

I'm not sure if this is a bug in fbrnch, or because fedpkg verrel doesn't return the ~bootstrap on F31 (dunno if it's correct on F32+.)

juhp commented 4 years ago

I suspect it is likely a fbrnch bug, thanks - I need to handle "sub-dist" parts of release tags correctly.

juhp commented 4 years ago

First I thought this was only because fbrnch is being a bit sloppy with comparing release tags...

But I see that bootstrap is actually embedded inside %dist itself (in macros.dist). :-( I think it's not uncommon to override %dist (and fbrnch also does this) so I feel it is not a good way to do it. And that may well be why fedpkg verrel also doesn't output the bootstrap bcond: though fbrnch doesn't use fedpkg verrel either because it is slow. IMO bootstrap should be handled (appended) by rpm itself, or at least not be defined inside %dist: it has actually nothing to do with %dist at all, so it is just a hack.

juhp commented 4 years ago

Well despite my rant, I changed the way I read the package NVR now. Instead of overriding dist with --define, I now read the system dist and replace that string with the desired target dist. This is not perfect, though the chances of dist appearing twice in a srpm nvr should admittedly be rather small.

I have also made a fix to be more precise about comparing NVR release.

juhp commented 4 years ago

These fixes should be in fbrnch-0.4-1 building in copr now. Thanks for reporting these, and please report if you still see problems.

juhp commented 3 years ago

Relatedly, it'd be nice if building with bootstrap could be automated. The nice thing about bootstrap is it will automatically add ~bootstrap to the version-release (which sorts older), so there's no need to mess with releases or changelog. fbrnch parallel could build once with %bcond_without bootstrap, then flip the line to %bcond_with bootstrap and build again after the first build set is done. Maybe bonus points if it noticed some things already had boostrap on and didn't mess with them.

It is probably better to open a separate issue for this RFE: though I can't really promise a timeline.

juhp commented 3 years ago

Relatedly, it'd be nice if building with bootstrap could be automated. The nice thing about bootstrap is it will automatically add ~bootstrap to the version-release (which sorts older), so there's no need to mess with releases or changelog. fbrnch parallel could build once with %bcond_without bootstrap, then flip the line to %bcond_with bootstrap and build again after the first build set is done. Maybe bonus points if it noticed some things already had boostrap on and didn't mess with them.

I think this would need to be under flag at least, though I feel it is an edge case. Maybe something like fbrnch commit --bootstrap?

juhp commented 3 years ago

I will close this out for now since bootstrap is only used by quite few Fedora packagers I feel. But open another ticket to suggest more.

juhp commented 3 years ago

Just noting here that older versions of rpmspec (eg RHEL8/current Debian stable) do not support --with/--without.