search

dteviot / WebToEpub

A simple Chrome (and Firefox) Extension that converts Web Novels (and other web pages) into an EPUB.
Other
740 stars 140 forks source link

Triggers bot #663

Closed RealNasty closed 2 years ago

RealNasty commented 2 years ago

Describe the bug Ranobes.net have just added a antibot system and so it brought some issues with it. When 'webtoepub' is in the proccess of 'searching for urls' it triggers that system and therefor hinders it from fetching the complete book in order to convert it.

I encountered this when fetching big books (1000+ chapters), smaller ones seem to be ok.

What i'm hoping for is a way to i guess reduce the amount of traffic being sent to the site so it does not trigger the antibot system.

dteviot commented 2 years ago

@RealNasty Modified so Chapter Fetch throttling will also apply to getting ToC pages for Ranobes.

image

Test versions for Firefox and Chrome have been uploaded to https://drive.google.com/drive/folders/1B_X2WcsaI_eg9yA-5bHJb8VeTZGKExl8?usp=sharing. Pick the one suitable for you, follow the "How to install from Source (for people who are not developers)" instructions at https://github.com/dteviot/WebToEpub/tree/ExperimentalTabMode#user-content-how-to-install-from-source-for-people-who-are-not-developers and let me know how it goes.

I suggest open WebToEpub on story with few chapters, go to advanced settings to set rate limiting, then go to story with lots of chapters.

For my notes 30 minutes work.

RealNasty commented 2 years ago

@dteviot Much appreciated. It works just fine

dteviot commented 2 years ago

@RealNasty Updated version (0.0.0.145) has been submitted to Firefox and Chrome stores. Firefox version is available now. Chrome might be available in 1 to 3 weeks.