Failed to solve Cloudflare challenge

[yujinlin0224]

>>> import CloudflareScraper
>>> scraper = CloudflareScraper.create_scraper()
>>> print(scraper.get("http://www.javlibrary.com/tw/vl_newrelease.php?&mode=2&page=20").content)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\User\Miniconda3\lib\site-packages\requests\sessions.py", line 546, in get
    return self.request('GET', url, **kwargs)
  File "C:\Users\User\CloudflareScraper\CloudflareScraper\__init__.py", line 68, in request
    return self.solve_cf_challenge(resp, **kwargs)
  File "C:\Users\User\CloudflareScraper\CloudflareScraper\__init__.py", line 185, in solve_cf_challenge
    redirect = self.request(method, submit_url, **cloudflare_kwargs)
  File "C:\Users\User\CloudflareScraper\CloudflareScraper\__init__.py", line 68, in request
    return self.solve_cf_challenge(resp, **kwargs)
  File "C:\Users\User\CloudflareScraper\CloudflareScraper\__init__.py", line 185, in solve_cf_challenge
    redirect = self.request(method, submit_url, **cloudflare_kwargs)
  File "C:\Users\User\CloudflareScraper\CloudflareScraper\__init__.py", line 68, in request
    return self.solve_cf_challenge(resp, **kwargs)
  File "C:\Users\User\CloudflareScraper\CloudflareScraper\__init__.py", line 185, in solve_cf_challenge
    redirect = self.request(method, submit_url, **cloudflare_kwargs)
  File "C:\Users\User\CloudflareScraper\CloudflareScraper\__init__.py", line 61, in request
    if self.ifCloudflare(resp):
  File "C:\Users\User\CloudflareScraper\CloudflareScraper\__init__.py", line 78, in ifCloudflare
    raise Exception('Failed to solve Cloudflare challenge!')
Exception: Failed to solve Cloudflare challenge!

> conda list                                                                                                            # packages in environment at C:\Users\User\Miniconda3:
#
# Name                    Version                   Build  Channel
asn1crypto                1.2.0                    py37_0
astroid                   2.3.3                    py37_0
autopep8                  1.4.4                      py_0
ca-certificates           2019.11.27                    0
certifi                   2019.11.28               py37_0
cffi                      1.13.2           py37h7a1dbc1_0
chardet                   3.0.4                 py37_1003
colorama                  0.4.1                    py37_0
conda                     4.7.12                   py37_0
conda-package-handling    1.6.0            py37h62dcd97_0
cryptography              2.8              py37h7a1dbc1_0
idna                      2.8                      py37_0
isort                     4.3.21                   py37_0
lazy-object-proxy         1.4.3            py37he774522_0
mccabe                    0.6.1                    py37_1
menuinst                  1.4.16           py37he774522_0
openssl                   1.1.1d               he774522_3
pip                       19.3.1                   py37_0
pycodestyle               2.5.0                    py37_0
pycosat                   0.6.3            py37hfa6e2cd_0
pycparser                 2.19                     py37_0
pylint                    2.4.4                    py37_0
pyopenssl                 19.1.0                   py37_0
pysocks                   1.7.1                    py37_0
python                    3.7.5                h8c8aaf0_0
pywin32                   223              py37hfa6e2cd_1
requests                  2.22.0                   py37_1
ruamel_yaml               0.15.46          py37hfa6e2cd_0
setuptools                42.0.2                   py37_0
six                       1.13.0                   py37_0
sqlite                    3.30.1               he774522_0
torrentool                1.0.2                    pypi_0    pypi
tqdm                      4.40.0                     py_0
urllib3                   1.25.7                   py37_0
vc                        14.1                 h0510ff6_4
vs2015_runtime            14.16.27012          hf0eaf9b_0
wheel                     0.33.6                   py37_0
win_inet_pton             1.1.0                    py37_0
wincertstore              0.2                      py37_0
wrapt                     1.11.2           py37he774522_0
yaml                      0.1.7                hc54c509_2
you-get                   0.4.1355                 pypi_0    pypi                        

[doko-desuka]

Thank you for the bug report. The problem is that CF updated its challenge mechanism some weeks ago, and this scraper hasn't been updated yet.

At the moment it's better to use @VeNoMouS' cloudscraper as it has received some updates for using a local solver (called native), and it also supports this new CF mechanism.
You can find that project in here: https://github.com/VeNoMouS/cloudscraper (the solver is in /cloudscraper/interpreters/native.py)

[VeNoMouS]

Lol I just woke up and I read this... and thought this was my repo, and was like ... wtf? lol

[doko-desuka]

Hehe if anyone else drops by I'll keep pointing them to yours. Cheers!