AttributeError: 'NoneType' object has no attribute 'groups'

andreighinea1 commented 4 years ago

Before creating an issue, first upgrade cfscrape with pip install -U cfscrape and see if you're still experiencing the problem. Please also confirm your Node version (node --version or nodejs --version) is version 10 or higher.

Make sure the website you're having issues with is actually using anti-bot protection by Cloudflare and not a competitor like Imperva Incapsula or Sucuri. And if you're using an anonymizing proxy, a VPN, or Tor, Cloudflare often flags those IPs and may block you or present you with a captcha as a result.

Please confirm the following statements and check the boxes before creating an issue:

[X] I've upgraded cfscrape with pip install -U cfscrape
[X] I'm using Node version 10 or higher
[X] The site protection I'm having issues with is from Cloudflare
[X] I'm not using Tor, a VPN, or an anonymizing proxy

Python version number

Run python --version and paste the output below:

Python 3.8.4

cfscrape version number

Run pip show cfscrape and paste the output below:

Name: cfscrape
Version: 2.1.1
Summary: A simple Python module to bypass Cloudflare's anti-bot page. See https://github.com/Anorov/cloudflare-scrape for more information.
Home-page: https://github.com/Anorov/cloudflare-scrape
Author: Anorov
Author-email: anorov.vorona@gmail.com
License: UNKNOWN
Location: c:\python38\lib\site-packages
Requires: requests
Required-by:

Code snippet involved with the issue

import io, cfscrape
from bs4 import BeautifulSoup

url = 'https://www1.animeultima.to/a/rezero-kara-hajimeru-isekai-seikatsu-2nd-season_517659'
scraper = cfscrape.create_scraper()
content = scraper.get(url).content

Complete exception and traceback

Traceback (most recent call last):
  File "C:\Program Files (x86)\Python38-32\lib\site-packages\cfscrape\__init__.py", line 251, in solve_challenge
    challenge, ms = re.search(
AttributeError: 'NoneType' object has no attribute 'groups'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "D:\Misc\Diverse\Scripts\AnimeUltima anime downloader\animeultima_downloader.py", line 6, in <module>
    content = scraper.get(url).content
  File "C:\Program Files (x86)\Python38-32\lib\site-packages\requests\sessions.py", line 543, in get
    return self.request('GET', url, **kwargs)
  File "C:\Program Files (x86)\Python38-32\lib\site-packages\cfscrape\__init__.py", line 129, in request
    resp = self.solve_cf_challenge(resp, **kwargs)
  File "C:\Program Files (x86)\Python38-32\lib\site-packages\cfscrape\__init__.py", line 204, in solve_cf_challenge
    answer, delay = self.solve_challenge(body, domain)
  File "C:\Program Files (x86)\Python38-32\lib\site-packages\cfscrape\__init__.py", line 290, in solve_challenge
    raise ValueError(
ValueError: Unable to identify Cloudflare IUAM Javascript on website. Cloudflare may have changed their technique, or there may be a bug in the script.

Please read https://github.com/Anorov/cloudflare-scrape#updates, then file a bug report at https://github.com/Anorov/cloudflare-scrape/issues."
[Finished in 0.5s with exit code 1]
[shell_cmd: python -u "D:\Misc\Diverse\Scripts\AnimeUltima anime downloader\animeultima_downloader.py"]
[dir: D:\Misc\Diverse\Scripts\AnimeUltima anime downloader]
[path: C:\Program Files (x86)\Common Files\Oracle\Java\javapath;C:\Program Files (x86)\Python38-32\Scripts\;C:\Program Files (x86)\Python38-32\;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Windows\System32\OpenSSH\;C:\Program Files (x86)\NVIDIA Corporation\PhysX\Common;C:\Program Files\NVIDIA Corporation\NVIDIA NvDLISR;C:\Program Files\Git\cmd;C:\Program Files\Tesseract-OCR;C:\Program Files\Tesseract-OCR\tesseract.exe;C:\Program Files (x86)\Python38-32\Executables\geckodriver.exe;C:\Program Files (x86)\Python38-32\Executables\MicrosoftWebDriver.exe;C:\Program Files (x86)\Python38-32\Executables;C:\Program Files\MiKTeX\miktex\bin\x64\;C:\Users\Andrei\AppData\Local\Microsoft\WindowsApps;C:\Program Files\FFmpeg\bin;]

URL of the Cloudflare-protected page

AnimeUltima

URL of Pastebin/Gist with HTML source of protected page

HTML source

dessalines commented 4 years ago

I'm getting this too.

nicolasegp commented 4 years ago

I have the same problem I can see it is because cloudflare changed its challenge code, but in the other reports they have not managed to find the correct challenge :(

URL: https://animeflv.net

Code

import cfscrape
cf = cfscrape.create_scraper()
print( cf.get('https://animeflv.net').content )

Traceback

Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/cfscrape/__init__.py", line 251, in solve_challenge
    challenge, ms = re.search(
AttributeError: 'NoneType' object has no attribute 'groups'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "./anime2.py", line 8, in <module>
    print( cf.get('https://animeflv.net').content )
  File "/usr/lib/python3.8/site-packages/requests/sessions.py", line 543, in get
    return self.request('GET', url, **kwargs)
  File "/usr/lib/python3.8/site-packages/cfscrape/__init__.py", line 129, in request
    resp = self.solve_cf_challenge(resp, **kwargs)
  File "/usr/lib/python3.8/site-packages/cfscrape/__init__.py", line 204, in solve_cf_challenge
    answer, delay = self.solve_challenge(body, domain)
  File "/usr/lib/python3.8/site-packages/cfscrape/__init__.py", line 290, in solve_challenge
    raise ValueError(
ValueError: Unable to identify Cloudflare IUAM Javascript on website. Cloudflare may have changed their technique, or there may be a bug in the script.

Please read https://github.com/Anorov/cloudflare-scrape#updates, then file a bug report at https://github.com/Anorov/cloudflare-scrape/issues."

pei-guo-2510 commented 3 years ago

I have the same issue.

isobarbaric commented 2 years ago

I have the same issue.

Anorov / cloudflare-scrape