search

mansuf / mangadex-downloader

A command-line tool to download manga from MangaDex, written in Python.
https://mangadex-dl.mansuf.link/
MIT License
511 stars 43 forks source link

Feature Request: Throttle #24

Closed garbled1 closed 2 years ago

garbled1 commented 2 years ago

Would it be possible to add a throttle option of some kind? Like a sleep between pages? I worry that if we hit it too hard eventually they will cloudflare the thing....

mansuf commented 2 years ago

The app has rate limit handler, so if you being rate limited by MangaDex, the app will put into sleep until given time.

See this example output below

$ mangadex-dl "https://mangadex.org/title/..." --cover none
[INFO] Checking url = https://mangadex.org/title/...
[INFO] Using English language
[INFO] Downloading cover manga ...
[INFO] Fetching all chapters...
[INFO] Using pdf format
[INFO] Getting images from chapter 1
[INFO] We being rate limited, sleeping for 59.27 (attempt: 1) # <--- Delay execution for 59.27 seconds and then continue if done sleeping
...

Also, MangaDex is using DDoS-Guard not Cloudflare.

garbled1 commented 2 years ago

I'm just saying' Id' like to add an additional voluntary rate limit, to avoid any possible future issues.

mansuf commented 2 years ago

Done, added in https://github.com/mansuf/mangadex-downloader/commit/c026bf453b387f4788cf842f617d8e12285b7ef4. Thanks for the idea !

You can start using it from option --delay-requests