search

anatol / pacoloco

Caching proxy server for Arch Linux pacman
MIT License
216 stars 30 forks source link

Download code refactoring (#7)(#30)(#58) #72

Closed Focshole closed 1 year ago

Focshole commented 1 year ago

Support for partially downloaded files with redirects if unavailable, ranges are now supported (#7) No more file downloaded at the same time with unexpected behaviour (#30) Should reduce download latency with granular error handling and HEAD requests before downloads (#58) Updated readme formatting

I have rewritten some code parts in handleRequest and fixed some race conditions which could happen because of how mutexes were being used. Now if pacoloco shouldn't handle the request (because the file is partial and it is being downloaded), it replies with a 302 to the upstream mirror, so that it won't break the download to other clients (and then it would allow partial downloads for cached packages). Also, this PR should reduce timeout waiting by trying a HEAD request before trying to download a file which is not available upstream. It includes also some formatting fixes in the readme, as well as with some improved examples.

Focshole commented 1 year ago

Finished merging the code!

Focshole commented 1 year ago

This should be fine, I have tested it again with the new version. I'll update you if I get into any issues! If you can, also @Jabbermuggel @HarHarLinks give it a try!

Focshole commented 1 year ago

Of course @anatol too if you can! I'd expect quite a lot of reviews, I had made a lot of code that can be simplified (while writing it I had tried to catch all options and exceptions, for example I'd expect that a lot of else clauses can be dropped! )

I kept it as such because it was easier to me to figure out if I had been missing some corner cases!

Focshole commented 1 year ago

I'll update this code in a few weeks hopefully

Focshole commented 1 year ago

Ok, the code works as expected, I had tested it and simplified a little thanks to recent PRs works. There is a new test for partial downloads too.

Focshole commented 1 year ago

I'll check what had been changed and update this code accordingly!

I had found only one issue with this implementation. If you request a repository page (e.g. host/pkg/archlinux/) without specifying a package, you get some problems with incomplete reads. I'll have to investigate it further.

Focshole commented 1 year ago

I had found only one issue with this implementation.

I had fixed it and added a test case covering it. Updated the PR with the latest commits for the master

anatol commented 1 year ago

I have reworked the download code and it merged as 326216c61f84fb9123f7edfc96c7a16479908b24 It should solve the mentioned problems. Please give it a try.

I am closing this PR as obsolete.