search

openSUSE / zypper

World's most powerful command line package manager
http://en.opensuse.org/Portal:Zypper
Other
403 stars 110 forks source link

zypper repo freshness is too short by default #470

Open Firstyear opened 1 year ago

Firstyear commented 1 year ago

By default zypper expires metadata every 15 minutes. This is often too short on most clients, where a zypper refresh can be quite slow and causes zypper to appear to lag. Imagine you did "zypper ref" walked away for a moment, and did "zypper install ...". the second zypper install can often taken more than a minute to respond before starting while it refreshes metadata in the background.

The 15 minute refresh is REALLY aggressive, and seems to be needlessly so. I think this should be extended by default. A reasonable amount would be 8 - 18 hours, for "once a day" refreshes.

mlandres commented 1 year ago

No, zypper does not expire the metadata. If autorefresh is turned on for a repo, we just check whether the repos repodata/repomd.xml has changed. But we're aware that in some regions (incl. Australia) the slow connection to download.opensuse.org matters more then the few Kb we're downloading. We're working on it together with the team maintaining our mirror infrastructure.

libzypp-17.31.4 (on TW, soon on Leap15 too) already contains a few patches which may mitigate the issues. It uses a geoip cache, so we can directly connect to the mirrorsites without a roundtrip to d.o.o. Unfortunately there are a few files (repomd.xml and it's signature are amongst them) which have to be taken from d.o.o directly. So there is still room for further enhancements.

Anyhow, /etc/zypp/zypp.conf defines the value for repo.refresh.delay. The default is 10 minutes. This is the time that has to pass before we do another autorefresh check at all. You can increase this time or turn off autorefresh for the repos and call zypper ref explicitly to check and get new data.

Firstyear commented 1 year ago

I'm extremely aware of the issues in Australia - I am one of the people pushing for improvements due to the (generally horrible) experience I have with zypper. I am part of that "mirror team". So I'm very well aware of what's going on.

Even with those changes, geoip and others, this issue exists. Network latency is still a problem and the timeout is too aggressive. Those autorefresh checks make zypper lag extremely badly on any command innvocation.

It needs to be increased to something sensible, because we are not publishing new metadata every 10 minutes for tumbleweed. Unless I'm mistaken, snapshots are daily.

mlandres commented 1 year ago

We don't have distribution agnostic defaults. Ideally the repos would suggest a reasonable refresh default in their metadata, or maybe we can introduce a setting per repo. So update repos can be checked more often than core repos. Until that we have repo.refresh.delay.

Firstyear commented 1 year ago

Yes, I know we don't have that capability in our metadata, but we also don't need it. The majority of cases are users who want to install or update their software once a day. We only release tumbleweed snapshots/updates once a day. So why do we need a check every 15 minutes? The only thing that needs that would be developers testing things on smaller OBS repos, and I'm sure they can work out how to run "zypper refresh".

Why are we optimising our refresh time outs for an edge case, while making our default user experience poor for the overwhelming majority of people?

Setting repo.refresh.delay to 8 hours would already be enough to ensure updates are checked for a few times a day, without trashing user experience.

mlandres commented 1 year ago

The code does not run on TW only and SLE releases far more often. You can increase the config value if it suits better to you. And we can talk about a supporting a distro specific config, if TW want's to maintain it's own.

Firstyear commented 1 year ago

The problem is that this default is too much for all users not just me. This needs a resolution that isn't just "go away, edit your own config".

luc14n0 commented 1 year ago

In my humble opinion, having a distro specific config would be interesting. And what would be even more interesting would be to have a /etc/zypp/zypp.conf.d/ config drop-in directory.

For example, we have now Thorsten's os-update that updates the system, once a day, in the background. We already had YaST's capability to do the same, before. Those kind of software/scripts could ship a drop-in config file with careful tweaks to better suit the use of their tools.

So, for typical scenarios, even for those who don't automate their update/upgrade process, I can't see why Zypper would need to do so many repo metadata checks/refreshes throughout the day, since many people won't even install/update something every single day. If someone could shed a light on this subject, I'd appreciate.

RokeJulianLockhart commented 1 year ago

If autorefresh is turned on for a repo, we just check whether the repos repodata/repomd.xml has changed.

I think that that is perfect. If the repo data is different, refresh. If not, don't. Why have arbitrary timeouts?