zlatinb / muwire

MuWire file sharing client for I2P
GNU General Public License v3.0
191 stars 27 forks source link

Improved resilience or a Retry button for Browse List downloads? #91

Open Searinox opened 2 years ago

Searinox commented 2 years ago

I have by now found user file lists totaling into the hundreds of thousands. When you start reaching numbers like 500000 shared files, the file list takes over 10 minutes to finish downloading. I know there is some manner of resilience built into the list because it can stop updating for 1-2 minutes at a time and then resume, but still I find it failing more often the bigger it gets. As a result it usually takes me 2-3 attempts to download a list that big.

The problem is that if a failure occurs you have to start over. There is no manual or automated Retry logic. Can this be made more resilient?

Could the retry mechanism be made more persistent? Or maybe a manual Retry button could be added?

Alternatively, could this be due to the sharer having just added new files? Is there a version ID shared along with the file list so it doesn't get corrupt or fail if files are added/removed during the list get? Would/could the downloading client know if it has to start over in mid-transfer?

zlatinb commented 2 years ago

Right now the download list logic is very simple - grab a copy of the current shared files list and send it over the network. Any updates to the file list after that are not going to be reflected in what the downloader sees.

As far as making it more resilient, I'd rather work on making all I2P connections more resilient as every aspect of MuWire and many other apps will benefit. Having said that, there is one simple way of making the browse list "resumable". It relies on creating some kind of ordering of the shared files; the easiest ordering I can think of is a monotonically increasing sequence number. Then the downloader can request only files with a sequence number higher than the last one received.

So short answer is yes it's possible, and I'll think about it some more, maybe do it for the release after next.

Searinox commented 2 years ago

Sounds like it will come naturally as part of a bigger reliability update. Guess we'll come back around to this whenever that happens then.