search

dipu-bd / lightnovel-crawler

Generate and download e-books from online sources.
https://pypi.org/project/lightnovel-crawler/
GNU General Public License v3.0
1.49k stars 293 forks source link

Fix this source allnovelfull.com, CloudflareChallengeError. #1697

Open tanreeve opened 2 years ago

tanreeve commented 2 years ago

Novel URL: https://allnovelfull.com/reincarnated-as-the-demon-kings-son.html App Location: EXE App Version: v3.0.0

Describe this issue

Cannot download some chapters (CloudflareChallengeError.) and blank pages in some chapters.

Screenshots

Blank pages image

Error message image

Error Log

ConnectionError: HTTPSConnectionPool(host='allnovelfull.com', port=443): Pool is closed.
CloudflareChallengeError: Detected a Cloudflare version 2 Captcha challenge, This feature is not available in the opensource (free) version.
CloudflareChallengeError: Detected a Cloudflare version 2 Captcha challenge, This feature is not available in the opensource (free) version.
CloudflareChallengeError: Detected a Cloudflare version 2 Captcha challenge, This feature is not available in the opensource (free) version.
CloudflareChallengeError: Detected a Cloudflare version 2 Captcha challenge, This feature is not available in the opensource (free) version.
CloudflareChallengeError: Detected a Cloudflare version 2 Captcha challenge, This feature is not available in the opensource (free) version.
CloudflareChallengeError: Detected a Cloudflare version 2 Captcha challenge, This feature is not available in the opensource (free) version.
CloudflareChallengeError: Detected a Cloudflare version 2 Captcha challenge, This feature is not available in the opensource (free) version.
CloudflareChallengeError: Detected a Cloudflare version 2 Captcha challenge, This feature is not available in the opensource (free) version.
CloudflareChallengeError: Detected a Cloudflare version 2 Captcha challenge, This feature is not available in the opensource (free) version.
dipu-bd commented 2 years ago

I checked but did not see such issues here. Maybe your IP is blocked by cloudflare? Try visiting the link in your browser and see what happens.

You can use VPN or proxy. Another easy way is to use Google Colab.

tanreeve commented 2 years ago

@dipu-bd I'm sure my ip is not blocked by cloudflare because i can access https://allnovelfull.com without VPN or Proxy via browser.

Screenshot : image

Edit :

I tried using Google Colab too but still got those error.

Screenshot : image

dipu-bd commented 2 years ago

Reincarnated As The Demon King S Son (epub).zip

I could download from my VPN.

This site is also causing problem in discord bot too. I will change this crawler to support the browser.