A solution to network error 403, cloudflare

[zeoint]

Is your feature request related to a problem? Please describe.

• Everytime I try to pack an epub from Lightnovelworld that have 1000+ chapters, cloudflare of the site activates and webtoepub throws 'failed with network error 403' and saves the epub with the chapters that were downloaded before cloudflare is thrown.

Describe the solution you'd like

• When a cloudflare is thrown, pause the download, after solving the captcha by the user, resume the download.
• The problem will be solved.

Describe alternatives you've considered

• Comment

Additional context Add any other context or screenshots about the feature request here.

[gamebeaker]

@zeoint You could try to fetch the chapter slower. Link to wiki Manual Throttle|Delay per chapter.

[zeoint]

I had already tried the above methods. I limited max web pages to fetch to 1 and manual throttle to 3seconds/chapter.

It downloaded way less chapters with limit than without any limit.

[zeoint]

Maybe this would solve it,

1. Auto pause download on cloudflare or 403 error. (Option or by default).

2. A button to check the connection status.

   • This button will be clickable only when the download is paused. (Check connection status).

3. After user manually solves the cloudflare and after the user clicks the button, if connection is okay.

4. Then resume downloading of epub from the last successful downloaded chapter.

   • Now that button will be frozen (not be clickable) when download is ongoing (successfully resumed).

[dteviot]

@zeoint

I've made a quick hack to the code so it should now ask if you want to retry when a 403 error is returned by the server. Warning: This is completely untested. I'm not set up to easily test 403 errors.

Test versions for Firefox and Chrome have been uploaded to https://drive.google.com/drive/folders/1B_X2WcsaI_eg9yA-5bHJb8VeTZGKExl8?usp=sharing.

Note, the file(s) you want are the ones labelled "403.experimental". NOT the WebToEpub ones.

Pick the one suitable for you, follow the "How to install from Source (for people who are not developers)" instructions at https://github.com/dteviot/WebToEpub/tree/ExperimentalTabMode#user-content-how-to-install-from-source-for-people-who-are-not-developers and let me know how it goes.

For my notes: 67 minutes work

[zeoint]

Thank you. It is working as intended.

Also, it was a cool idea to add pop-up.

[dteviot]

@zeoint

Updated version (0.0.0.155) has been submitted to Firefox and Chrome stores. Firefox version is available now. Chrome might be available in 3 to 21 days.

[zeoint]

Thank you.