search

RedSquirrel87 / manga-downloader

A very simple application written in JAVA to download any chapters of any manga from the supported websites
31 stars 1 forks source link

Reaperscans.com "Error while getting pages list!" #795

Closed Steven-OConnell closed 5 months ago

Steven-OConnell commented 1 year ago

It seems like something has changed (yet again) on Reaperscans.com's backend.

Webtoon in question: https://reaperscans.com/comics/4073-overgeared

I've included the log however it isn't capturing the same error as the GUI, so here is what I'm getting on that end.

23:34:09:  Getting pages list for chapter Chapter 181 - Enemy of the 2nd place...
23:34:09:  Error while getting pages list!

jvm_log.txt

RedSquirrel87 commented 1 year ago

From what I can see in the log it's simply a http 429 error (too many requests, here more info: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/429 ). I will check this as soon as I will be at home but it should not be framework change or a bug, you should try to add more pause between requests (you can set this param in program's settings).

RedSquirrel87 commented 1 year ago

Oh I just remembered about issue #768, there are probably so many chapters in that manga that the random wait between 1 and 2 seconds isn't even enough anymore. Or probably if you are unlucky you end up with too low waits all the time and so the website blocks them (in which case just try again and again until you are lucky enough to always catch the longest waits from the random values). I think that the only way to solve this problem without relying on random values anymore is to insert a new option in the program where the user himself can set the seconds of waits between requests only for this website, so that he can self-regulate them according to the manga and to how many chapters they have. I'll insert it in the next version 👍

RedSquirrel87 commented 1 year ago

Ok I just tried and I can confirm that it's an issue related to the random waits values, after 3 failed attempts on the fourth it worked and loaded the list correctly. While waiting for the next release you should be able to fix this simply by trying and trying until it works

RedSquirrel87 commented 1 year ago

I can confirm now that I have added the new option to the development version 👍 I have also added another extra option to set to load only chapters from the first page (useful when you want to get only the latest chapters released in chronological order without stressing the web site by loading unnecessarily the entire chapter list every time): image image image

This will be already available in tonight's nightly build and of course it will be included in the next public release too (ETA: next week if nothing goes wrong).

Steven-OConnell commented 1 year ago

Wow awesome, thanks so much. :)

RedSquirrel87 commented 1 year ago

I just discovered that they introduced a new anti-bot challenge in the online viewer: image Does it still work for you? Does it appear just for me? The bad fact is that does not seems to use cookies since it appear everytime, if it's that the case there should not be any way to bypass it with the program at the moment

Steven-OConnell commented 1 year ago

Oh wow, I was unaware they were doing that. It seems to be a global feature as I’m also seeing the same thing, I tried making an account to see if that would change anything but to no avail.

It seems like they REALLY don’t want people scraping their content.

RedSquirrel87 commented 5 months ago

v62 just released, it should fix this issue