mirrormanager to handle changes to updates repos file hierarchy output by bodhi

[dustymabe]

We are making some changes to bodhi that will allow bodhi to use pungi to create repos. As apart of this the file hierarchy for repos (i.e. where the repos are located) will change. We'll need mirrormanager to handle this once we make the change.

Additionally there will be content in the file hierarchy (like qcows, ISOs, etc) that we actually don't want to mirror. I believe that can be handled by mirrormanager as well.

This spawned from a fedora infra ticket and a releng meeting.

[adrianreber]

Could you provide an example what will change? I tried to understand the infrastructure ticket but I have no real idea what will change. So an example what is currently used and why it does not work and how it should be changed would be helpful.

[dustymabe]

basically here is what it looks like today: https://kojipkgs.fedoraproject.org/mash//updates/f26-updates-170804.1348/f26-updates/

and here is what it would look like in the future: https://koji.stg.fedoraproject.org/compose/mash/f26-updates-170803.0245/f26-updates/

[adrianreber]

For the repositories (a repository is something with a checksum and thus can have a metalink (repomd.xml)) in MirrorManager this means that instead of x86_64/repodata/ it will be compose/Everything/x86_64/os/repodata/ right? Or are other directories important.

That is for the updates repository. Are there path changes for the release repository also?

Will everything be now prefixed with compose/Everything/ ?

When will this start to hit mirrors?

[dustymabe]

For the repositories (a repository is something with a checksum and thus can have a metalink (repomd.xml)) in MirrorManager this means that instead of x86_64/repodata/ it will be compose/Everything/x86_64/os/repodata/ right? Or are other directories important.

Right. A few clarifications. There will also be other arches so compose/Everything/{x86_64,aarch64,etc}/os/. Also, just to make sure we don't only consider the repodata directory because the actual packages are in another directory.

That is for the updates repository. Are there path changes for the release repository also?

The changes we are proposing only affect bodhi so these changes should only be for the updates and updates-testing repos. Release day stuff will not change.

Will everything be now prefixed with compose/Everything/ ?

like URLs? I think that is something we can abstract away if we want to. It just happens to be how pungi lays it out in the directory structure. We really only care about files under compose/Everything/ for updates repo purposes.

When will this start to hit mirrors?

Not currently planned but we will coordinate with you. We'd like to get that to happen sometime within the next month. How do you feel about that?

[adrianreber]

So I still do not totally understand how it will be structured on the master mirror, but a change like this should be pretty easy to implement. The paths on the master mirror are changing every few releases and need to be adapted in MirrorManager. Thanks to this ticket it will not be a surprise like the last time(s).

From MirrorManagers point of view the repodata directory is the only important directory to create a metalink and a repository. And when I speak of a repository I mean a repository object from MirrorManager's point of view. So when we talk about repositories we probably mean something different. Something like 'repo=updates-released-f26&arch=x86_64'

Earliest I can look into this is beginning of September and I would guess it requires only a few 'simple' changes.

[dustymabe]

So I still do not totally understand how it will be structured on the master mirror, but a change like this should be pretty easy to implement. The paths on the master mirror are changing every few releases and need to be adapted in MirrorManager. Thanks to this ticket it will not be a surprise like the last time(s).

let's find some time to talk so that we understand everything :)

From MirrorManagers point of view the repodata directory is the only important directory to create a metalink and a repository. And when I speak of a repository I mean a repository object from MirrorManager's point of view. So when we talk about repositories we probably mean something different. Something like 'repo=updates-released-f26&arch=x86_64'

I'm not sure exactly what all of that means, but I'm sure if we spend a few minutes chatting in IRC or on video then we can understand each other.

Earliest I can look into this is beginning of September and I would guess it requires only a few 'simple' changes.

ok, thanks.

[puiterwijk]

We do not need to change anything at mirrormanager for the pungi generated repos.