search

Komet / MediaElch

Media Manager for Kodi
https://mediaelch.github.io/mediaelch-doc/about.html
GNU Lesser General Public License v3.0
829 stars 93 forks source link

[Scrapers] Scraping should be done in a separate thread #932

Open bugwelle opened 4 years ago

bugwelle commented 4 years ago

Scraping currently is done on the main thread. The only reason that MediaElch does not freeze is because of Qt's signal/slot mechanism and the fact that network requests are not blocking.

However, for example the IMDb scraper freezes MediaElch for about 1 to 2 seconds because the data extraction uses regexes which are quite slow. This becomes more obvious after #931 which introduces a progress bar right away when scraping has started which freezes when we run into the "parseAndAssignInfos" method (debugged and confirmed).

This should be part of #892

mhendu commented 3 years ago

Yes! Also if "Reload All Movies" could be in a separate thread. One thing I like about TMM is that as new movies are found they show in the UI and you can add metadata while other movies are being loaded. With MediaElch you have to wait for the full scan to be completed before adding metadata.

Also, saving metadata when you're doing it across thousands of NFO files can freeze the program. Maybe that should be its own thread, too? For instance, if I want to update IMDB ratings, or trailers, in bulk.