search

pkgcore / pkgcheck

pkgcore-based QA utility for ebuild repos
https://pkgcore.github.io/pkgcheck
BSD 3-Clause "New" or "Revised" License
35 stars 29 forks source link

Provide network checks for thirdpartymirrors #155

Open mgorny opened 4 years ago

mgorny commented 4 years ago

We won't be able to check them perfectly (or can we?) but it would be nice to flag definitely-dead mirrors and possible HTTPS upgrades.

radhermit commented 4 years ago

Not sure what you mean by perfectly, but in theory running a tree-wide FetchablesUrlCheck gives us all this information, just not in mirror specific form.

mgorny commented 4 years ago

Not really because sometimes the distfile is present on only some of the listed mirrors, and trying to filter false positives is really hard. Checking top-level directory would be helpful as a quick way to immediately flag dead mirrors.

radhermit commented 4 years ago

It's not as simple as that though since there are bouncer mirrors, e.g. checking the top-level for pypi isn't valid:

DeadMirror: thirdpartymirror: pypi: 404 Client Error: Not Found for url: https://files.pythonhosted.org/packages/source: https://files.pythonhosted.org/packages/source
mgorny commented 4 years ago

Yeah, that sucks. maybe I'll remove that mirror ;-).

radhermit commented 4 years ago

Anyway, besides that I have code working. One possible solution would be to recheck against a single valid pkg mirror URL to verify a dead mirror to catch more false positives.