search

ros-infrastructure / cookbook-ros-buildfarm

Apache License 2.0
2 stars 2 forks source link

Enable caching of some of the RPM repository metadata #88

Closed cottsay closed 5 months ago

cottsay commented 3 years ago

The names of files which match the caching rule contain the content hashes, so the content at that URL should never change. These files disappear as the repository is updated, and this leads to a race between the loading of the repomd.xml and accessing the files listed within (and here in this config).

This isn't a complete solution for a variety of reasons, among them that it depends on a client accessing the content so that it can be cached to begin with. I'm hoping it will mitigate the problem, however.

nuclearsandwich commented 3 years ago

it depends on a client accessing the content so that it can be cached to begin with. I'm hoping it will mitigate the problem, however.

If this proves an incomplete solution we could consider warming the caches as a finishing step for jobs that update repository data. I'm not sure how much time that would add to import_package jobs though.

nuclearsandwich commented 3 years ago

CI is definitely in a troubled state at the moment but the errors on the repo CI are nginx related and I expect are due to issues with the generated config file.

cottsay commented 3 years ago

the errors on the repo CI are nginx related

Good catch - I made a late change to the cache directory that broke things. It is resolved now.

cottsay commented 5 months ago

We don't use Pulp anymore here - closing this out.