search

etix / mirrorbits

Mirrorbits is a geographical download redirector written in Go for distributing files efficiently across a set of mirrors.
MIT License
503 stars 91 forks source link

Feature Request: Trigger scan via changing/monitoring a special file #170

Open lazka opened 6 months ago

lazka commented 6 months ago

RepositoryScanInterval in the config is using a fixed interval. To avoid things getting outdated for too long it gets run quite often, while the repo likely doesn't change that much. The default is currently every 5 minutes.

It would be nice if one could specify a single file in the config (similar to TraceFileLocation) which gets scanned every few seconds, and if the file changes a full scan is triggered.

For extra context:

Mirrors for pacman repos can (and some do) speed up their rsync interval by using the following script provided by Arch: https://gitlab.archlinux.org/archlinux/infrastructure/-/blob/master/roles/syncrepo/files/syncrepo-template.sh It requires a /lastupdate file which the pacman repo needs to update if any of the files in the repo change, so mirrors can detect changes via HTTP and only then start a full rsync. Example: https://repo.msys2.org/

This means in pacman repos there already exists a file which gets updated on repo changes and could be used in the config for this feature as is.

As a random data point, for the MSYS2 repo a scan takes ~20 seconds.