Issue regarding RPM/RHEL noarch packages syncing

Cloud-Kid commented 3 weeks ago

Hi again,

For the last few weeks, I try to implement repomanager as a replacement in our infrastructure for DEB/RHEL repositories in order to dismantle our current clunky repo syncing solution.

For now I had 0 issues regarding the debian syncing side part even if it was not yet tested on large set of clients. But I had some issues with the RHEL side of thing. Currently it seems not possible to correctly sync a RHEL repository fully and I try to understand why.

It may be RedHat specific, but I'm pretty sure it applies to CentOS too (I can't try it in my environment). It also maybe linked to this issue : #176 but I don't know enough about repos & packages to confirm it.

PS : The RedHat suscribtion SSL certs have been configured and are confimed to work.

Goal :

Trying to sync a RedHat Repo (BaseOs for instance) using this repo configuration URL : https://cdn.redhat.com/content/dist/rhel$releasever/$releasever/$basearch/baseos/os

Issue :

Trying to sync x86_64 and noarch packages does not include noarch packages or fails

If only x86_64 is specified the correct URL is specified in the logs (https://cdn.redhat.com/content/dist/rhel8/8/x86_64/baseos/os) so all of packages are retrieved excepts noarch packages that are hosted under the same path > That's the issue

The problem is that noarch packages hosted under the x86_64 URL are no retrieved even if they should Making the repo missing dependencies and causing other issues.

If only noarch is specified then the sync fails because the path doesn't exists, so thats corect (https://cdn.redhat.com/content/dist/rhel8/8/noarch/baseos/os) :

Possible reasons :

There are some RedHat specific magic that complexify the process
The syncing try to get all packages containing "x86_64" in the name under this path https://cdn.redhat.com/content/dist/rhel8/8/x86_64/baseos/os
Some other kind of filtering is taking part during your package retrieval process

I think you may be able to find out what's up with these informations, or maybe if it's a problem in the app or on our environement.

lbr38 commented 3 weeks ago

You're right there is an issue, but with RHEL repos only (I confirm I can sync both x86_64 and noarch packages with CentOS repos).

I may have missed something with RHEL repos, I should have made more sync tests. So I don't think there is a problem with your environment, I will find a workaround.

lbr38 commented 3 weeks ago

Now working :) I saw some noarch packages in the 14717 packages syncing.

If you need noarch packages (even from a x86_64 path), be sure to have selected noarch when creating a mirror (as you can see in the screenshot).

Cloud-Kid commented 3 weeks ago

Sure thing I will try it ASAP (Monday morning) ! I tried to understand what you did in your commits but I could not find any thing that I could understand with my very small knowledge in PHP. What was the problem ? I saw some memory management edits and error handling improving !

Thanks again that was fast ! I will close the issue as soon I can test it and confirm that it works.

lbr38 commented 3 weeks ago

Sure thing I will try it ASAP (Monday morning) ! I tried to understand what you did in your commits but I could not find any thing that I could understand with my very small knowledge in PHP. What was the problem ? I saw some memory management edits and error handling improving !

Thanks again that was fast ! I will close the issue as soon I can test it and confirm that it works.

I have simply modified the logic for searching candidate URLs to explore.

I will wait for you to confirm that everything is OK, thanks for your help.

Cloud-Kid commented 2 weeks ago

Hi Ludovic, Confirmed working with RHEL8 basos repository & Ansible repository, noarch packages appears and the package list is now complete ! Tanks a lot, issue closed

lbr38 / repomanager