search

nroi / flexo

a central pacman cache
MIT License
172 stars 10 forks source link

Different mirrors serving the same package with different size #99

Closed patlefort closed 1 year ago

patlefort commented 1 year ago

This is more of a question than a bug. How does Flexo handle this case? I've had this problem using my own cache using nginx. If I cache a package from mirror X , then a client with a package database downloaded from mirror Y wants to download the package, but the package from mirror X has a different size from Y, it might fail to download with a maximum size exceeded error from pacman. The package from both mirrors is valid and contain the same thing, so I assume it must be compressed differently. The package cache has no way to know what package database the client has and what is the maximum size.

nroi commented 1 year ago

I've had this problem using my own cache using nginx.

That's surprising, how frequently did this happen? Did this happen with an official Arch Linux mirror, or did you use something special (e.g. a different distro like Manjaro, or Arch Linux ARM, or an unofficial user repo)?

At the moment, there's nothing in flexo to handle this case, the code is based on the assumption that the file name (which includes the version string) uniquely identifies a file, and that this file will always be the same among all official mirrors. If this weren't the case, then you could also run into issues if you don't use flexo and simply switch the mirror in /etc/pacman.d/mirrorlist.

There's also reflector, which seems to be quite popular, and it includes a systemd timer which changes your mirrorlist weekly. I never heard anyone report any issues related to changing file sizes with reflector.

Let me know if you know anything more detailed about this. I hope that if it does happen, it's a very rare edge case.

patlefort commented 1 year ago

I don't know how frequent it can happen but it did. My mirrors come from reflector. So the problem is: it can happen and there is no way to know who's right or wrong since there isn't a central authority on the package database. It think it might be a pacman issue, capping a download to the size defined in the package database is too draconian and it doesn't matter as long as the content is the same. It also raise another issue: how to know if a mirror is altering some packages? I might have to ask the pacman devs.

patlefort commented 1 year ago

Maybe only downloading the database from tier 1 mirrors would minimize the problem?

nroi commented 1 year ago

it can happen and there is no way to know who's right or wrong since there isn't a central authority on the package database.

The central authority is archlinux.org, where all tier 1 mirrors sync their packages and database files from.

how to know if a mirror is altering some packages?

What legitimate reason is there to alter a package in any way? If that happens, then the mirror is at fault, not Flexo. The packages are meant to be synchronized from a trusted source and served by the mirror without any fiddling in between. A mirror which alters packages in any way would soon raise suspicion because users would notice that the signatures don't match, since pacman has been using signed packages as a default for a while now.

At the moment, I don't see any tangible problem. I've never had a bug report in Flexo due to different package sizes, and I've never heard before that this issue comes up when switching your mirrors regularly, as is the case when you use reflector, for example. If I have something more concrete, I can look into it, but as of now, I don't have any reason to assume that this is a real issue.