Anorov / cloudflare-scrape

A Python module to bypass Cloudflare's anti-bot page.
MIT License
3.34k stars 456 forks source link

Not working on AWS EC2 #304

Open Pechi77 opened 4 years ago

Pechi77 commented 4 years ago

Before creating an issue, first upgrade cfscrape with pip install -U cfscrape and see if you're still experiencing the problem. Please also confirm your Node version (node --version or nodejs --version) is version 10 or higher.

Make sure the website you're having issues with is actually using anti-bot protection by Cloudflare and not a competitor like Imperva Incapsula or Sucuri. And if you're using an anonymizing proxy, a VPN, or Tor, Cloudflare often flags those IPs and may block you or present you with a captcha as a result.

Please confirm the following statements and check the boxes before creating an issue:

Python version number 3.6.8

Run python --version and paste the output below:

cfscrape version number

Run pip show cfscrape and paste the output below:

Name: cfscrape
Version: 2.0.8
Summary: A simple Python module to bypass Cloudflare's anti-bot page. See https://github.com/Anorov/cloudflare-scrape for more information.
Home-page: https://github.com/Anorov/cloudflare-scrape
Author: Anorov
Author-email: anorov.vorona@gmail.com
License: UNKNOWN
Location: /home/ubuntu/.local/lib/python3.6/site-packages
Requires: requests

Code snippet involved with the issue

  File "<stdin>", line 1, in <module>
  File "/home/ubuntu/.local/lib/python3.6/site-packages/requests/sessions.py", line 546, in get
    return self.request('GET', url, **kwargs)
  File "/home/ubuntu/.local/lib/python3.6/site-packages/cfscrape/__init__.py", line 124, in request
    self.handle_captcha_challenge(resp, url)
  File "/home/ubuntu/.local/lib/python3.6/site-packages/cfscrape/__init__.py", line 147, in handle_captcha_challenge
    raise CloudflareCaptchaError(error, response=resp)
cfscrape.CloudflareCaptchaError: Cloudflare captcha challenge presented for www.owler.com (cfscrape cannot solve captchas)

Complete exception and traceback

(If the problem doesn't involve an exception being raised, leave this blank)

URL of the Cloudflare-protected page

[https://www.owler.com/iaApp/1335456/formtech-enterprises-company-profile]

URL of Pastebin/Gist with HTML source of protected page

[]

kenchangh commented 4 years ago

Yeah it does seem like CloudFlare completely blocked the whole AWS IP ranges with captcha. :( Are there any workarounds that you have learned before? @Anorov