search

openSUSE / zypper

World's most powerful command line package manager
http://en.opensuse.org/Portal:Zypper
Other
391 stars 107 forks source link

Zypper issues invalid range requests #458

Open Firstyear opened 1 year ago

Firstyear commented 1 year ago

When zypper is connecting to a single mirror, it can often issue invalid range requests. This is seen where zypper requests multiple's GETs of the same file, with succesive ranges. However, zypper is not testing if accept ranges is supported on the server, and is ignoring the status codes on response.

What should occur

Zypper should NOT issue range requests to a single server OR when issuing range requests, an initial single request is issued. The result of 206 allows further requests to be issued, a 200 is respected and the entire body consumed.

Current behaviour

Zypper blindly issues multiple range requests, does not wait for their status codes, and ignores the 200 status.

https://developer.mozilla.org/en-US/docs/Web/HTTP/Range_requests#partial_request_responses

bzeller commented 1 year ago

Zypper does not ignore the 206 status code. The code would cancel the request to that server and start a new one on a different mirror...

Also when we only have one mirror we fall back to downloading the file without ranges.

Firstyear commented 1 year ago

That behaviour seems to be different to what we are seeing in production though. So there is a problem here. For example, the 200 doesn't prompt zypper to only issues non-range reqs to a mirror, and it continually sends range requests even after a 200.

bzeller commented 1 year ago

Do you know how many mirrors where returned for the request?

Firstyear commented 1 year ago

1 mirror was returned.

Firstyear commented 1 year ago

If you set download.download.max_concurrent_connections = 0 or download.max_concurrent_connections = 1 then you get this behaviour.

And the difference is 765.4 KiB/s with zypper doing ranges, vs 10MiB/s with ZYPP_MULTICURL=0

max_1.pcap.zip

Firstyear commented 1 year ago

Given the recent testing here https://github.com/Firstyear/mirrorsorcerer#issues-mirrorsorcerer-can-not-prevent it's hard to see how ranges are actually improving download performance, and I think much more serious examination of them is needed and their place in zypper at all. They may have been valid in the past, but I think that's not true today.