search

hartwork / fetchcommandwrapper

:cow: Wrapper integrating aria2 (https://aria2.github.io/) into portage's FETCHCOMMAND for faster downloads (Python)
https://pypi.org/project/fetchcommandwrapper/
9 stars 4 forks source link

enhancement: randomly choose mirrors #3

Closed Alessandro-Barbieri closed 7 years ago

Alessandro-Barbieri commented 7 years ago

I have an idea: instead of passing the full mirrors list to aria2c, why not pass it n randomly chosen (http or ftp) mirrors? Should dramatically increase the speed when the files are not present in any mirror.

For example if aria2c download from 5 mirrors simultaneously, n can be 6

hartwork commented 7 years ago

Hey!

Should dramatically increase the speed when the files are not present in any mirror.

Since we have --max-file-not-found=2 in by now: is it still slow, for real? How would random mirror selection speed not-found detection up?

I have an idea: instead of passing the full mirrors list to aria2c, why not pass it n randomly chosen (http or ftp) mirrors? [..]

For example if aria2c download from 5 mirrors simultaneously, n can be 6

There is use of randomization. We could make current MAX_STREAMS = 5 (which is your n) configurable, but I don't see a strong reason to bother, yet. Convince me 😃

Alessandro-Barbieri commented 7 years ago

Ok, for now behaves well (after the upgrade)