tmplt
commented
5 years ago The easiest way of doing this is to call the respective plugins' mirror resolving functions in the downloader. Defenitely not the best place for that code, but I'm alright with it for a quicker v0.6.0 release. Some things need to be done for this:
- [x] Tag items with which plugin it was generated from.
- [x] Implement some mirror resolve function for
libgen.py, returning some request struct will all required into for curl (e.g. HTTP referrer, etc.). - [x] Resolve mirrors in some async way in
downloader, storing progress so thatdownloader::progress_callbackcan have decent status messages.
What I want is:
Downloading item...
N/A resolving mirrors... (try 1/4)which after resolve turns into:
100% ▕██████████████████████████▏ 0.00/0.00MB @ 15.63kB/s ETA: 0sOr something akin to it.
Course, we'll step-wise resolve mirrors. If the first mirror resolves and downloads correctly, there will be no need to resolve the other mirrors.
As of e9f8a9aecb694c9b44bd0341889090c971148233, direct-download URIs are not resolved from scraped mirrors as this should only be done when an item has been marked for download. The same plugin that supplied the item will be queried for a resolve. An URI that can be used directly to download the item should be returned.
It is possible that some headers must be set for the download to pass (e.g., referrer). See #31.