oae / kaizoku

Self-hosted manga downloader
https://kaizoku.app
MIT License
492 stars 31 forks source link

Unable to download Mangas from MangaDex #67

Open Ponkhy opened 1 year ago

Ponkhy commented 1 year ago

Describe the bug

Hello,

I just installed kaizoku on my server, unfortunately I'm encountering the issue of being unable to download Mangas. When I add a new one to the library it downloads 1-4 chapters and then stops. After looking more into it I can see this error in the logs:

kaizoku  | X /config/.config/mangal/sources/MangaDex.lua:130: attempt to index a non-table object(nil) with key 'body'
kaizoku  | stack traceback:
kaizoku  |      /config/.config/mangal/sources/MangaDex.lua:130: in main chunk
kaizoku  |      [G]: ?

I've also noticed that I'm now unable to open the website beceuase of the error ERR_HTTP2_PROTOCOL_ERROR. Maybe they've implemented some kind of protection in order to avoid mass downloads? Because after a few hours it's possible to access the website again. Would it be possible to build in a custom delay in order to avoid the issue, or could it be something else?

Many thanks!

Reproduction steps

No response

Expected behavior

No response

Additional context

No response

oae commented 1 year ago

Yes, they have rate limits. See https://api.mangadex.org/docs/rate-limits/#rate-limits

To add a custom delay, I need to rework the queue system.

Ponkhy commented 1 year ago

Ahh, I see, thank you! Yea a custom delay would be pretty good, otherwise MangaDex at least is not usable at the moment. Thanks for the quick reply!

timbroder commented 1 year ago

@Ponkhy You can retry them, that's what I did when I added a bunch of new manga. Doing this once a day or so for a week had everything catch up

Click into the fail queue CleanShot 2023-02-02 at 19 55 30@2x

Click failed, and then retry all

CleanShot 2023-02-02 at 19 55 58@2x

codevski commented 1 year ago

Yeah not sure whats with the rate limit, Tried out tachidesk and used mangadex extension downloads chapters without a issue, could it be if you attempt to download multiple chapters at one time it puts a limit on you? Its a shame mangadex cant really be used for the time being....

Also Clean all results to Internal Server Error

eswiig commented 1 year ago

Yeah I ran into issues with mangadex too, my biggest issue is that mangadex will ban me at the network layer temporarily, so I can't even browse mangadex anymore until the temporary ban lifts (usually not more than an hour or two)

A real shame, mangadex used to be my favorite source, although it has one weakness, often it releases several group releases of the same chapter so you get "dupe" content.