Constant possible rate limiting

namsie commented 3 months ago

Brief summary

While reading manga from any source, I can only read for a few pages before I'm hit with a loading panel and have to exit the manga and re-enter it until it loads the panels. If I wait for the panels to load, they return a network error but the second I press retry the panels load. This loading lasts for a minute or so.

Steps to reproduce

No response

Kotatsu version

7.2.1

Android version

14

Device

Samsung Galaxy zFlip 3

Acknowledgements

[X] This is not a duplicate of an existing issue. Please look through the list of open issues before creating a new one.
[X] This is not an issue with a specific manga source. Otherwise, you have to open an issue in the parsers repository.

galpt commented 3 months ago

@namsie

I'm reading AsuraScans and other sources, using Kotatsu v7.2.1 and this issue doesn't happen to me, not once, or at least I didn't notice it.

I keep telling people in Kotatsu Discord server that everyone should use the Image Optimization Proxy feature because this is an almost-guaranteed way to prevent you from getting rate limited.

Everyone is free to choose either wsrv.nl or 0ms.dev, but I prefer 0ms.dev since I'm one of the devs who helped building the proxy, so I know that this proxy won't rate limit anyone, at least not the normal readers.

Any images from any manga sources you're reading, after they're cached by the proxy, you're sending requests to the proxy. So the proxy only gets images that haven't been cached yet or the cache already expired. This means you can technically request 1 image for 10 times, 100 times, 1000 times per second to the 0ms.dev proxy if the image's already cached, and your requests won't even reach the original manga server, so you won't be rate limited by the original manga website.

About the 1000 requests per second, that's not a joke, see the proxy page here. That basically means 1000 (cached) images per second.

Unlike caching proxies, random manga websites with who-knows-what-kind-of-server-they-are-using usually won't be able to handle that much requests per second for every user, so they just rate limit anyone and you're one of those who got rate limited.

namsie commented 3 months ago

@galpt hey, thanks for the reply!! I switched to the image optimization proxy you recommended (0ms.dev) but the problem is still persisting the same as it was before on any source. do you think its rate limiting or is it a different issue? im not very well versed, so i appreciate your time :)

namsie commented 3 months ago

im so sorry, i keep accidentally pressing buttons, ignore those omfg 😭

galpt commented 3 months ago

@namsie

Without a screenshot, I can only guess. Try AsuraScans, if they load just fine with the image proxy, then the proxy works fine. It could be a problem with your internet connection or your ISP blocking some manga sources. Without any screenshot, I can only guess, but like mentioned, this issue didn't happen to me with different sources, not just AsuraScans.

I will recommend you to keep using the proxy to prevent getting rate limited, maybe also use the Kotatsu DNS feature too. Have you tried using a VPN? if the issue disappears after using a VPN, then this is your ISP problem and might not be a rate limit issue.

If this is about your ISP blocking some manga sources, the DNS feature might work. But if it's a more complicated blocking issue, you need a VPN. Same with bad ISP routing quality, you need a VPN.

tl;dr - try Kotatsu with a VPN first, if it works then it's your ISP.

Raja-Debnath commented 2 months ago

Actually I faced this issue few time past 2 days , note I haven't tried with Asura scans yet ,

I faced this issue on mangafire , some images don't load while the lower one does , I don't what's this issue, I had to quit the manga to fix it

galpt commented 2 months ago

I think rate limiting is more of a parser issue, thus this issue should be closed IMHO since it's opened on the app repo, not the parser repo.

Also if it's another issue about Mangafire, it's already discussed here https://github.com/KotatsuApp/kotatsu-parsers/issues/875 Members also discussed about it on Kotatsu Discord server.

From the discussion, try using the img optimization proxy feature (the 0ms.dev proxy if possible, since it was tested using this proxy). Downloading & reading from mangafire when using that proxy didn't trigger a slowdown/rate limit.

KotatsuApp / Kotatsu