Unable to bypass indeed.com cloudflare

Before creating an issue, first upgrade cfscrape with pip install -U cfscrape and see if you're still experiencing the problem. Please also confirm your Node version (node --version or nodejs --version) is version 10 or higher.

Make sure the website you're having issues with is actually using anti-bot protection by Cloudflare and not a competitor like Imperva Incapsula or Sucuri. And if you're using an anonymizing proxy, a VPN, or Tor, Cloudflare often flags those IPs and may block you or present you with a captcha as a result.

Please confirm the following statements and check the boxes before creating an issue:

[x] I've upgraded cfscrape with pip install -U cfscrape
[x] I'm using Node version 10 or higher
[x] The site protection I'm having issues with is from Cloudflare
[x] I'm not using Tor, a VPN, or an anonymizing proxy

Python version number

Run python --version and paste the output below:

Python 3.10.4

cfscrape version number

Run pip show cfscrape and paste the output below:

Name: cfscrape                                    Version: 2.1.1                                    Summary: A simple Python module to bypass Cloudflare's anti-bot page. See https://github.com/Anorov/cloudflare-scrape for more information.           Home-page: https://github.com/Anorov/cloudflare-scrape                                              Author: Anorov                                    Author-email: anorov.vorona@gmail.com             License: UNKNOWN                                  Location: /data/data/com.termux/files/usr/lib/python3.10/site-packages                              Requires: requests                                Required-by:

Code snippet involved with the issue

import cfscrape                                                                                     scraper = cfscrape.create_scraper(delay=20)  # returns a CloudflareScraper instance                 # Or: scraper = cfscrape.CloudflareScraper()  # CloudflareScraper inherits from requests.Session
print(scraper.get("https://www.indeed.com/jobs?q=data+engineer&l=United+States").content)

Complete exception and traceback

(If the problem doesn't involve an exception being raised, leave this blank)

b'<!DOCTYPE html>\n<html lang="en-US">\n<head>\n    <title>Just a moment...</title>\n    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />\n    <meta http-equiv="X-UA-Compatible" content="IE=Edge" />\n    <meta name="robots" content="noindex,nofollow" />\n    <meta name="viewport" content="width=device-width,initial-scale=1" />\n    <link href="/cdn-cgi/styles/challenges.css" rel="stylesheet" />\n    \n\n</head>\n<body class="no-js">\n    <div class="main-wrapper" role="main">\n    <div class="main-content">\n        <h1 class="zone-name-title h1">\n            <img class="heading-favicon" src="/favicon.ico"\n                 onerror="this.onerror=null;this.parentNode.removeChild(this)" />\n            www.indeed.com\n        </h1>\n        <h2 class="h2" id="challenge-running">\n            Checking if the site connection is secure\n        </h2>\n        <noscript>\n            <div id="challenge-error-title">\n                <div class="h2">\n                    <span class="icon-wrapper">\n                        <div class="heading-icon warning-icon"></div>\n                    </span>\n                    <span id="challenge-error-text">\n                        Enable JavaScript and cookies to continue\n                    </span>\n                </div>\n            </div>\n        </noscript>\n        <div id="trk_jschal_js" style="display:none;background-image:url(\'/cdn-cgi/images/trace/managed/nojs/transparent.gif?ray=7445c81f9a4db6fa\')"></div>\n        <div id="challenge-body-text" class="core-msg spacer">\n            www.indeed.com needs to review the security of your connection before proceeding.\n        </div>\n        <form id="challenge-form" action="/jobs?q=data+engineer&amp;l=United+States&amp;__cf_chl_f_tk=kHBkWMHc0vIlPnY8zcQ7efXTdUkjTDCPolfCaqpOOC0-1662116843-0-gaNycGzNB9E" method="POST" enctype="application/x-www-form-urlencoded">\n            <input type="hidden" name="md" value="NQX6LSFqJ5u17EFgoo5FOpc1hhX0v24Robn6K4wQXF8-1662116843-0-AX-SXUPRprpZKPRqegt3ZbuYvSzECnKqkYKXlVwcXD4Fanq2IpPerRNrknNKucbV5s0dH7kenxXiOSg08Dfmv0t6nbWFwWMYU3BhB9j6Q0WpQvxHGRfwvwgFlqYdhTmndciUolBf6VQa8Gp368EhC6UAPTF84e-Ln4zqGGhr0E7NS-V4Mi19ONMj7eWqAfxwdbEydnae7OtQ34N5uJ9Zz0Tw3IyM7_P8qPCVDn3kU6LiaPTkmI0xkRt4LU0CvXKcH63eFblJ2sDxUZktI2szKj9bo5omJaHGbDLyrunQPY5sZTDI3MtQAgqboVcLof04Njtzg_pJC30C5EqNkk5xbx5tEsVZ44SWqtEsdhMKp4ewa9F3nq8qHqZ_IoBHeCRG62JGMROVdC_S1WtBGSNKLWUMuqieqJ40yn5mspDGc4wEgFoH24n-mL9kUMEBi8oaGBdWn2_48GnoP_VwEclX8odZ0dMIn8u5UY3Pa6KB9s6EVl_5PE8WidhPuF9vFgZ6jDtVUx0zNeQHb4yifUyKGmK81xZJ5SZT_mJqOWKI3UPD06t3r2IbGpBmyiIphzuRSbtVOfGInokpXoXsw6fvCtEHeN4QipwRVR5B0DmTHyhG4tS5idb5DFWBtVY5TGHBgPyH7oe3Is1YFGiDGUZV8bxwzH1EkhRXTCN1c2kcvhpKOFQs7yGc4yy65Es44vCuEz2jy8DmMiV8_Y1M1HZU1_z85remAZQDfhhDfCwCAX9n" />\n            <input type="hidden" name="r" value="YxnkFpHuxx2iWByjawRBmNbJVXaLPvXls5gbMsRvFw0-1662116843-0-Af0NLSEhFV4qeITDOMlHoxNfNLbRQUCxgncOtyTnEl5I4/z9gOdwMCQlGGcCXd4AQ3fLBlQt4ZY+OO3aYTMXYwvfo35BoxpJy457UNDrW46PlfoS6BsxDHgWBdkGW/SmK0ep2wENLkoI8L4z3I/wxdYRKh2cEbnORPsmQ/5p6GXw3X+Urmd9yPww5LvfdIhhv6uQWt+q2Luhc1qTami3uJToyGT+dG2B703BaQl7v0fRLNHNaqnqmdDO2vTzAdrtpoZiS5azFOSwn7z/BFNXA15njnCSzm0KgH3wNZb0CHKX6R29sY7K5tZBe+g3+5i2iOl3BpEGWhj2XoEpBMZt0hmWmteSFTz4i7PcRdMwhwbbUEZ1kRC3ivFCqU1sQUcd1KdPl6H7kM3hL/dBY+TSte/rD9eGgE2chdAAk5DwqQvutd+z1abqjYt9opGp3blRrNhNgQZJJtpMXXrvflnM64GfeDvnS4BsrfF8uDxKEaWrH8ajo2o5DqOsESJOG+U7P796w4smLQk1mVLfEHq1jJyoR3gSihIJg6mFfeR4anrzsbajntNk+onwzLGAOtPXP2kOl2TDfq6OhkuXUb7f7Xb6o8voz2jX96PDxpUfybNbdqclItistsuGpjnV4Kvytzut3yCxfyv2cXWKhxueY3PzRcgS9Vby2Fqb4mLEiFqDHyY0tgi+q79dez/dQtzN5Tv7oZcWfFz8OeBx27l578uklr2I1cd9fSMZrjxTciFgePPtME2Zy++Z4dQRKtg73z+2km2mhSlrLA1fKcNnAfLjcbbmGtAGz1ny3h+km3rV24w5si8wTSvN2FLS3DYHwwelleAWj0SQANQ9i9tuT+Ys/GoDVVnfqQ2L77kiQfpxCA+HaM3DyPmUUNxc8pwUGacd/Rz5smRAl2rCaV86/sEmSsPJ76espmEFlZlMk/UEX1UJmGhasZd+VdaqTvy5q9lYJWuREm6L8X8qepOCoTJA+/2PiWDnrt/AxKsHiH38aXVLsFJ6wWIeAhEwR9Ub7CnCFYuSyIAxZjpmUikx6hhkLLSj9H+L+/xrYaoOi/VhEu+pUOYB8nI7pAHs5F3XQEQhnc+4uLofGYip2xIagcn2awyTWgYRKBlo67ZyMbJKSF0N+2hsHTU9hIh/R/zxEv2yLWt2V8plYZznFhCzguc4+BiF3TXPEZuVnILTjTuRN279dXeARQZc3BmmOyxLKSj+00uJ/LaeEmIxdkDBhP/tyynFNaR3W0wmddnEKyfzLpMXvPOJl9xUKUSL4wEXE+OZCCZD/KI3w8hHJmz1nX7zjPwwxng8jIV1LlXvY20hzdRpHHsMiRMtqQzEGGLFwNDpxiJbyOJ3ljIEo5sg2LQUaVurol12QmBm3JznPOQULv3YRvDRlFLW3ZGTDEAoQDlEWufbO2hAWHbFcqCN+BMlFoWFtGDMKA0t3+gSmDvEeAPCDNMIMvL4LdWvjWIYt4reh1iBAc7CddpTCATevHPbj8dS9BLIaeAWwNJKo8GEjHaQssmZbcclDeQMJKJ4FIG8P0eGky5sr31mcsgUh7EcYraGC6xtUk5KAFWeH30IeC3ZrxOlbHKowPyT+K7tN83HGlCXG755bFckcQv2yJYEtIh/0hnOO9bOZwPqUSOBIgnsw1yG6B1VToc+u61P3lrkGk8E80ogCdv56a8eTPnH58MeWd11wvRuATxf2r0YV+tC7myr2DRdx7s282ACr1tNst0yWpcsLWrWEorK6g6DtiLQThVYwxblr3F96jlj69vc+rOArbLzgf9VnsFwbQAJX1Y3dy2DRthYymtOc77yukPnJz8MIN+/1kqeFEvj/+90p/hQUcXAuadxSVRDD3sGoHK8y+Q943um6etUA6SUVRLNcViWQIhJdxr84NApBOWMG6bAvP+ivaN4qr4l4f0Y/l+Nmv9hug/vuxiezjA="/>\n        </form>\n    </div>\n</div>\n<script>\n    (function(){\n        window._cf_chl_opt={\n            cvId: \'2\',\n            cType: \'managed\',\n            cNounce: \'23853\',\n            cRay: \'7445c81f9a4db6fa\',\n            cHash: \'2707162b80959cb\',\n            cUPMDTk: "\\/jobs?q=data+engineer&l=United+States&__cf_chl_tk=kHBkWMHc0vIlPnY8zcQ7efXTdUkjTDCPolfCaqpOOC0-1662116843-0-gaNycGzNB9E",\n            cFPWv: \'g\',\n            cTTimeMs: \'1000\',\n            cTplV: 4,\n            cTplB: \'cf\',\n            cRq: {\n                ru: \'aHR0cHM6Ly93d3cuaW5kZWVkLmNvbS9qb2JzP3E9ZGF0YStlbmdpbmVlciZsPVVuaXRlZCtTdGF0ZXM=\',\n                ra: \'TW96aWxsYS81LjAgKE1hY2ludG9zaDsgSW50ZWwgTWFjIE9TIFggMTBfMTJfNikgQXBwbGVXZWJLaXQvNTM3LjM2IChLSFRNTCwgbGlrZSBHZWNrbykgQ2hyb21lLzU5LjAuMzA3MS44NiBTYWZhcmkvNTM3LjM2\',\n                rm: \'R0VU\',\n                d: \'D9uwPhAqiQ7e1gwwR2J7/kjAcp+izE19Tt0hJAhvxHQv9GaP+yRGI2AXeYhJxRtzNueaFf4dw0nU8ntjtBzmgI+M0u62W1D+cc3wt8BIvSMlHn8pSqq9j4ace0tunv2FuSlN3W48hOLV0YC5o9R+om2DblQuRKEZrj9BrVqa//q7qVgTzRx/zzBSta7peAworo7ZGof9wG7YOERx742S6CqEMJ1JILExhbhccwAy8l1iqwZZq34n5wCOB6DwenMRWlF8PhZmTS10zKvi2S+iEye5+DJpTI+p120cK2Irrbtz1L6Qjxks4958xJ8dGO+KWw+G1SFqoBpzv+aSuWQ9v6bQaj69G/JHI3T9OeVLKncZM7AkYq9P0zUZ8Aj3qBfumVXYsfpF/QbaQkYIJyGybpO47+9K25saq2dOIcgYjEvkL7uTlTdr2CLzZFB6Ayksq4ZTahbxrr8YZrEUyJracwefGRBp6T2Fcc809/YdvdnFqaVF3Qufiv2Hya7akX+t093jnrmrZ8zz0gPVSv4KjgCDiaG6YBFyWDUX1l6215kRSo9hTXXKlnZ+L29rfEexmN8qXuCutEA2Y1mtkhy/h+tErxX1fCAm508FDmB0O+A//yDVforALo9srGKJf4Jd\',\n                t: \'MTY2MjExNjg0My40NzYwMDA=\',\n                m: \'KD9iOmRQmS+5eTJZBXVFRmRP4irwLwA4XtwfdXZyN2Y=\',\n                i1: \'xx9X1wvCLLY1u0fId1zqkQ==\',\n                i2: \'Xyy3YiDCbcaO2Dpgp96oBg==\',\n                zh: \'2kfjrLYfAVNem19Lin3oP+VKo7ladb55se1eVtiLLnQ=\',\n                uh: \'Xuxz+P/DBoAImGVWHucNBa1YDTqDzhW41a/s5bcbPAo=\',\n                hh: \'kscNdiThTx/CcYpUiV7EP4v/VgPxIHsoBIQQ6Rdzvmg=\',\n            }\n        };\n        var trkjs = document.createElement(\'img\');\n        trkjs.setAttribute(\'src\', \'/cdn-cgi/images/trace/managed/js/transparent.gif?ray=7445c81f9a4db6fa\');\n        trkjs.setAttribute(\'style\', \'display: none\');\n        document.body.appendChild(trkjs);\n        var cpo = document.createElement(\'script\');\n        cpo.src = \'/cdn-cgi/challenge-platform/h/g/orchestrate/managed/v1?ray=7445c81f9a4db6fa\';\n        window._cf_chl_opt.cOgUHash = location.hash === \'\' && location.href.indexOf(\'#\') !== -1 ? \'#\' : location.hash;\n        window._cf_chl_opt.cOgUQuery = location.search === \'\' && location.href.slice(0, -window._cf_chl_opt.cOgUHash.length).indexOf(\'?\') !== -1 ? \'?\' : location.search;\n        if (window.history && window.history.replaceState) {\n            var ogU = location.pathname + window._cf_chl_opt.cOgUQuery + window._cf_chl_opt.cOgUHash;\n            history.replaceState(null, null, "\\/jobs?q=data+engineer&l=United+States&__cf_chl_rt_tk=kHBkWMHc0vIlPnY8zcQ7efXTdUkjTDCPolfCaqpOOC0-1662116843-0-gaNycGzNB9E" + window._cf_chl_opt.cOgUHash);\n            cpo.onload = function() {\n                history.replaceState(null, null, ogU);\n            };\n        }\n        document.getElementsByTagName(\'head\')[0].appendChild(cpo);\n    }());\n</script>\n\n    <div class="footer" role="contentinfo">\n        <div class="footer-inner">\n            <div class="clearfix diagnostic-wrapper">\n                <div class="ray-id">Ray ID: <code>7445c81f9a4db6fa</code></div>\n            </div>\n            <div class="text-center">Performance &amp; security by <a rel="noopener noreferrer" href="https://www.cloudflare.com?utm_source=challenge&utm_campaign=m" target="_blank">Cloudflare</a></div>\n        </div>\n    </div>\n</body>\n</html>\n'

URL of the Cloudflare-protected page

LINK GOES HERE

URL of Pastebin/Gist with HTML source of protected page

[LINK GOES HERE]

Anorov / cloudflare-scrape