search

gpodder / gpodder-sailfish

gPodder 4 for Sailfish OS Packaging
https://openrepos.net/content/keeperofthekeys/gpodder
49 stars 29 forks source link

refreshing multiple podcasts in parallel #150

Closed thigg closed 3 years ago

thigg commented 3 years ago

This change means a huge QOL improvement for me. Refreshing is now so much faster. I think its parallelizing so well, because this is mainly bound by server responsetimes and not by bandwidth.

I'm using from concurrent.futures import ThreadPoolExecutor for parallelization here. We could think about providing max_workers. My guess is, we could even set it to something like 30, because network IO is so slow... (Documentation)

This isn't based on the most up-to-date master, as I cant deploy it in my emulator.

thigg commented 3 years ago

Maybe we can identify more network operations that could be in paralell. I just saw in the logs, that cover art downloading is quite slow as well...

Keeper-of-the-Keys commented 3 years ago

Are you limiting the amount of threads to run in parallel? I don't think we should exceed the cores available a gPodder update should not lock up the phone (I'll test it on my phone - currently subscribed to 25 podcasts)

thigg commented 3 years ago

38 podcasts and no problem. Its limited by the default settings as described in the documentation 

Changed in version 3.8: Default value of max_workers is changed to min(32, os.cpu_count() + 4)
Changed in version 3.5: If max_workers is None or not given, it will default to the number of processors on the machine, multiplied by 5

From my observations I would assume on the Xperia X it is something like 8 right now. (which seems to be 3.8)

Keeper-of-the-Keys commented 3 years ago

Looks very nice, thanks!