search

AlmaLinux / mirrors

The AlmaLinux project mirrors configuration and tools.
GNU General Public License v3.0
86 stars 410 forks source link

Option to fill in variables in mirrorlist #581

Open andreasdijkman opened 2 years ago

andreasdijkman commented 2 years ago

Is there an option to fill in the variables that the mirrorlist returns? We have Spacewalk running and the mirrorlist returns a list of URL's that have the variable $basearch in them. Spacewalk can't resolve or change that variable.

It would be nice to have the option to generate that URL without those variables so it returns static URL's.

Now

The URL https://mirrors.almalinux.org/mirrorlist/8/baseos returns the following:

http://mirror.nl.altushost.com/almalinux/8/BaseOS/$basearch/os/
http://mirror.nforce.com/pub/linux/almalinux/8/BaseOS/$basearch/os/
http://mirror.previder.nl/almalinux/8/BaseOS/$basearch/os/
http://mirrors.evoluso.com/almalinux/8/BaseOS/$basearch/os/
http://mirror.wd6.net/almalinux/8/BaseOS/$basearch/os/
http://almalinux.mirror.wearetriple.com/8/BaseOS/$basearch/os/
http://mirror.mijn.host/almalinux/8/BaseOS/$basearch/os/
http://mirrors.mvps.net/almalinux/8/BaseOS/$basearch/os/
http://almalinux.mirror.liteserver.nl/8/BaseOS/$basearch/os/
http://mirror.almalinux.ams01.readydedis.com/almalinux/8/BaseOS/$basearch/os/

Suggestion

Adding a parameter to the URL will fill in those variables, simething like <url>?basearch=x86_64 The following URL would return the below URL's: https://mirrors.almalinux.org/mirrorlist/8/baseos?basearch=x86_64

http://mirror.nl.altushost.com/almalinux/8/BaseOS/x86_64/os/
http://mirror.nforce.com/pub/linux/almalinux/8/BaseOS/x86_64/os/
http://mirror.previder.nl/almalinux/8/BaseOS/x86_64/os/
http://mirrors.evoluso.com/almalinux/8/BaseOS/x86_64/os/
http://mirror.wd6.net/almalinux/8/BaseOS/x86_64/os/
http://almalinux.mirror.wearetriple.com/8/BaseOS/x86_64/os/
http://mirror.mijn.host/almalinux/8/BaseOS/x86_64/os/
http://mirrors.mvps.net/almalinux/8/BaseOS/x86_64/os/
http://almalinux.mirror.liteserver.nl/8/BaseOS/x86_64/os/
http://mirror.almalinux.ams01.readydedis.com/almalinux/8/BaseOS/x86_64/os/
jonathanspw commented 2 years ago

Are you trying to use spacewalk with our mirrorlist instead of locally-hosted mirrors with direct URLs?

andreasdijkman commented 2 years ago

The Spacewalk-server will be the locally hosted mirror for AlmaLinux. Clients will connect to Spacewalk, which in turn fetches the content from the mirrorlist.

jonathanspw commented 2 years ago

So spacewalk is a proxy/cache of sorts?

I've not used spacewalk before so I'm trying to understand what the actual problem is here that you're trying to overcome.

If spacewalk is a mirror anyway why not just rsync it like any other mirror?

andreasdijkman commented 2 years ago

Spacewalk is a local repository management tool. It is deprecated now but it is/was the upstream version of Red Hat Satellite 5 but still works OK. Our local servers don't download the RPM's from upstream but from the Spacewalk-server. Spacewalk is pointed to upstream servers with a yum-config but (unfortunately) without all the $basearch and other variables, because the server is OS and arch independent. It mirrors the upstream repositories for internal use. We can have multiple versions and multiple archs in Spacewalk so we need to declare the sources by OS-version and by arch.

jonathanspw commented 2 years ago

So spacewalk only caches what's needed, ie what's requested by clients, basically?

andreasdijkman commented 2 years ago

Spacewalk downloads the entire repository or only the latest packages of each repository you tell it to sync. It serves as an offline and controllable repository server for clients on internal networks, that can't connect to the internet by firewall/policy/CSO/whatever reason. But it needs to specifically download each repository that is needed by clients, it's not a caching proxy server of some sorts. It's an offline copy of upstream repositories so client's don't need to connect to internet-based repositories to use or download packages. (Saves a lot of bandwidth too...)