ziplokk1 / incapsula-cracker-py3

Python3 compatible way to bypass sites guarded with Incapsula
https://ziplokk1.github.io/incapsula-cracker-py3/
The Unlicense
195 stars 33 forks source link

API Request is being blocked. #32

Open muzaT opened 1 month ago

muzaT commented 1 month ago

I recently came across this python package and really appreciate the developers that they have put soo much effort and made it work flawless.

I tried to access a website, after inserting headers as they go in the browser. Web page content is fetched successfully but the issue is that, there is a button. Clicking on that button, triggers an API request, which returns json data. When I try it with this package, it gets blocked and returns below error:

<html>
    <head>
        <noscript>
            <title>Pardon Our Interruption</title>
        </noscript>

        <meta name="viewport" content="width=1000">
        <meta name="robots" content="noindex, nofollow">
        <meta http-equiv="cache-control" content="no-cache, no-store, must-revalidate">
        <meta http-equiv="pragma" content="no-cache">
        <meta http-equiv="expires" content="0">

        <style>
            .container { max-width: 800px; margin: auto; font-family: 'Helvetica Neue',Helvetica,Arial,sans-serif; color: #7a838c; }
            h1 { color: #2a2d30; font-weight: 500; }
            li { margin: 0 0 10px; }
            a { color: #428bca; }
            a:hover, a:focus { color: #2a6496; }
        </style>

        <script>
          var isSpa = new URLSearchParams(window.location.search).get('X-SPA') === '1' || window.isImpervaSpaSupport;
        </script>

        <!-- This head template should be placed before the following script tag that loads the challenge script -->
        <script>
          window.onProtectionInitialized = function(protection) {
            if (protection && protection.cookieIsSet && !protection.cookieIsSet()) {
              showBlockPage();
              return;
            }
            if (!isSpa) {
              window.location.reload(true);
            }
          };
          window.reeseSkipExpirationCheck = true;
        </script>

        <script>
          if (!isSpa) {
            var scriptElement = document.createElement('script');
            scriptElement.type = "text/javascript";
            scriptElement.src = "/Spurre-Onell-vp-Enter-feed-ere-Yourthe-away-riso/14883154119434262891?s=9lRPm776";
            scriptElement.async = true;
            scriptElement.defer = true;
            document.head.appendChild(scriptElement);
          }
        </script>

    </head>
    <body>

        <div class="container">
            <script>document.getElementsByClassName("container")[0].style.display = "none";</script>

            <h1>Pardon Our Interruption</h1>
<p>As you were browsing something about your browser made us think you were a bot. There are a few reasons this might happen:</p>
<ul>
<noscript><li>You've disabled JavaScript in your web browser.</li></noscript>
<li>You're a power user moving through this website with super-human speed.</li>
<li>You've disabled cookies in your web browser.</li>
<li>A third-party browser plugin, such as Ghostery or NoScript, is preventing JavaScript from running. Additional information is available in this <a title='Third party browser plugins that block javascript' href='http://ds.tl/help-third-party-plugins' target='_blank'>support article</a>.</li>
</ul>
<p>To regain access, please make sure that cookies and JavaScript are enabled before reloading the page.</p>

        </div>
        <script>
          function showBlockPage() {
            document.title = "Pardon Our Interruption";
            document.getElementsByClassName("container")[0].style.display = "block";
          }

          if (isSpa) {
            showBlockPage();
          } else {
            setTimeout(showBlockPage, 10000);
          }
        </script>
    </body>
</html>

Process finished with exit code 0

Code snippet:

session = IncapSession()

session.cookies.set('Cookie', 'visid_incap_2413658=OtjS12IzRQ+hKNsLDp4Shx2Q6mYAAAAAQUIPAAAAAADzUMUxSF+SiN0hHLBWY6rT; incap_ses_128_2413658=rkSPJMtdQ1YZXSCscL/GAR2Q6mYAAAAA0vLjT1iXPBwUoEM1S3dxqQ==; incap_ses_775_2413658=q1Rtb5x0Rhk1Ol4bg1rBCh+Q6mYAAAAAFW5niDtjVSQ0uF1tQS4Pqg==; _gcl_au=1.1.512602668.1726648355; _tt_enable_cookie=1; _ttp=G3ss00DM3AsDi8LjliofUogZ97b; _ga=GA1.1.771662499.1726648356; _scid=iIkg4MW064d6cqdRAxN8Wree7GrWNTlP; _fbp=fb.1.1726648355756.490626892793758522; _ScCbts=%5B%5D; _scid_r=gQkg4MW064d6cqdRAxN8Wree7GrWNTlPzguJmQ; nlbi_2413658_2147483392=tIHqG/lUvTLaYOb6eguELgAAAAC3J3vBZqQUgvfc3qBK4kQa; reese84=3:HcPPj2rdqpDwoWlYUaCduQ==:9wy/KN76yAp+IX321b5vQXWWhI6SR8gKljxSCdaDzDdO5+5tLigqXWyJ6HZSt9hjkjrChBgZWaWM8Up2yOM/6Bch9xTuQrMwF/Cy5MnP3rnL7ObFuHmpquPM0TAUjikDuFG22fDAAMkP6voelGgbwVRT0Zg8E36Dz4f7fIdCrd80wq0Dm3b9qTAKSzkOSERBDlCdz2deSP/jzQyn6kZb6xwvb/6MS6SD+dPw1SL84xOtNHuIF74vXvD7wSvM/xx2oXOkrFur+N1wCtq9Dbvox0VgoVudpwY+6138vbzPlijbdPs1THZbAaw7vVA7Xwfeklhqvl7EIeFvmZFCJTpz4jffQZh/S0NEuZmxo41LCmb/USZH+62+W4+zKAT9E7r2Wsf5W6GdzwtVY8hl/Or8VwtMcyy2RECnOYWOUr2VuP1OYQQwToDHlEx8ofgRbROAHE3WZuKhkb37Ov7nody6Kw==:CJsPUKj0+ufJL0hAvmlMwiPWLoKge3DS+z4JBO9YRIc=; ias=0; nlbi_2413658=PuJtcKXdChm2y1yleguELgAAAAB3AhLXlU/LPxXcc8cn127l; _sctr=1%7C1726603200000; __gads=ID=ee5a178be1b0f05b:T=1726648356:RT=1726648356:S=ALNI_MZyw5qAhL0xPStB1n0AO9wKPZbejw; __gpi=UID=00000f0545f704b8:T=1726648356:RT=1726648356:S=ALNI_MYrEpGfB3M0ndF1X9ENeU-rzw_IGA; __eoi=ID=9ff938aca5b93c6d:T=1726648356:RT=1726648356:S=AA-AfjYiQlITvzkWMzSsBbMpghn4; _cc_id=d385bcb2a3e0f8df6d9d5690b0cc21d0; panoramaId_expiry=1726734756911; panoramaId=e231123d24848fca3d292205f7dda9fb927a63c68d47f55451f18511fdcfb078; panoramaIdType=panoDevice; moe_uuid=39a77911-d741-4354-b8e0-5a4645d10df1; USER_DATA=%7B%22attributes%22%3A%5B%5D%2C%22subscribedToOldSdk%22%3Afalse%2C%22deviceUuid%22%3A%2239a77911-d741-4354-b8e0-5a4645d10df1%22%2C%22deviceAdded%22%3Atrue%7D; _gaexp=GAX1.2.xB4nT9jdK5vQz7UwLpR3yH.20074.0; OPT_IN_SHOWN_TIME=1726648362280; __rtbh.uid=%7B%22eventType%22%3A%22uid%22%2C%22id%22%3A%22771662499.1726648356%22%7D; __rtbh.lid=%7B%22eventType%22%3A%22lid%22%2C%22id%22%3A%22kAppdxPpSxBnMaq25HOn%22%7D; SOFT_ASK_STATUS=%7B%22actualValue%22%3A%22dismissed%22%2C%22MOE_DATA_TYPE%22%3A%22string%22%7D; lotame_domain_check=dubizzle.com; _ga_LRML1YM9GH=GS1.1.1726648355.1.1.1726648387.28.0.0; SESSION=%7B%22sessionKey%22%3A%2273eea8e6-6c92-480e-8a23-047177ef6aa4%22%2C%22sessionStartTime%22%3A%222024-09-18T08%3A32%3A40.070Z%22%2C%22sessionMaxTime%22%3A1800%2C%22customIdentifiersToTrack%22%3A%5B%5D%2C%22sessionExpiryTime%22%3A1726650187904%2C%22numberOfSessions%22%3A1%7D; UniqueLead=$1237-15945764')
response = session.get('https://dubai.dubizzle.com/api/v4/leads/1237/0a7018d4f4cc46428e05e644e8ded161/listing-profile/phone/', headers={
        "Accept": "application/json",
        'accept-encoding': 'gzip, deflate, br',
        'accept-language': 'en-US,en;q=0.9',
        'content-type': 'application/json;charset=UTF-8',
        'cookie': 'visid_incap_2413658=OtjS12IzRQ+hKNsLDp4Shx2Q6mYAAAAAQUIPAAAAAADzUMUxSF+SiN0hHLBWY6rT; incap_ses_128_2413658=rkSPJMtdQ1YZXSCscL/GAR2Q6mYAAAAA0vLjT1iXPBwUoEM1S3dxqQ==; incap_ses_775_2413658=q1Rtb5x0Rhk1Ol4bg1rBCh+Q6mYAAAAAFW5niDtjVSQ0uF1tQS4Pqg==; _gcl_au=1.1.512602668.1726648355; _tt_enable_cookie=1; _ttp=G3ss00DM3AsDi8LjliofUogZ97b; _ga=GA1.1.771662499.1726648356; _scid=iIkg4MW064d6cqdRAxN8Wree7GrWNTlP; _fbp=fb.1.1726648355756.490626892793758522; _ScCbts=%5B%5D; reese84=3:HcPPj2rdqpDwoWlYUaCduQ==:9wy/KN76yAp+IX321b5vQXWWhI6SR8gKljxSCdaDzDdO5+5tLigqXWyJ6HZSt9hjkjrChBgZWaWM8Up2yOM/6Bch9xTuQrMwF/Cy5MnP3rnL7ObFuHmpquPM0TAUjikDuFG22fDAAMkP6voelGgbwVRT0Zg8E36Dz4f7fIdCrd80wq0Dm3b9qTAKSzkOSERBDlCdz2deSP/jzQyn6kZb6xwvb/6MS6SD+dPw1SL84xOtNHuIF74vXvD7wSvM/xx2oXOkrFur+N1wCtq9Dbvox0VgoVudpwY+6138vbzPlijbdPs1THZbAaw7vVA7Xwfeklhqvl7EIeFvmZFCJTpz4jffQZh/S0NEuZmxo41LCmb/USZH+62+W4+zKAT9E7r2Wsf5W6GdzwtVY8hl/Or8VwtMcyy2RECnOYWOUr2VuP1OYQQwToDHlEx8ofgRbROAHE3WZuKhkb37Ov7nody6Kw==:CJsPUKj0+ufJL0hAvmlMwiPWLoKge3DS+z4JBO9YRIc=; ias=0; nlbi_2413658=PuJtcKXdChm2y1yleguELgAAAAB3AhLXlU/LPxXcc8cn127l; _sctr=1%7C1726603200000; _cc_id=d385bcb2a3e0f8df6d9d5690b0cc21d0; panoramaId_expiry=1726734756911; panoramaId=e231123d24848fca3d292205f7dda9fb927a63c68d47f55451f18511fdcfb078; panoramaIdType=panoDevice; moe_uuid=39a77911-d741-4354-b8e0-5a4645d10df1; USER_DATA=%7B%22attributes%22%3A%5B%5D%2C%22subscribedToOldSdk%22%3Afalse%2C%22deviceUuid%22%3A%2239a77911-d741-4354-b8e0-5a4645d10df1%22%2C%22deviceAdded%22%3Atrue%7D; _gaexp=GAX1.2.xB4nT9jdK5vQz7UwLpR3yH.20074.0; OPT_IN_SHOWN_TIME=1726648362280; __rtbh.uid=%7B%22eventType%22%3A%22uid%22%2C%22id%22%3A%22771662499.1726648356%22%7D; __rtbh.lid=%7B%22eventType%22%3A%22lid%22%2C%22id%22%3A%22kAppdxPpSxBnMaq25HOn%22%7D; SOFT_ASK_STATUS=%7B%22actualValue%22%3A%22dismissed%22%2C%22MOE_DATA_TYPE%22%3A%22string%22%7D; UniqueLead=$1237-15945764; skybar_sess_False=1; default_site=2; sid=mhh3b6u0dhf44r7q2oji1cvvou9a7f19; nlbi_2413658_2147483392=+JAvJYmuMznMnfcreguELgAAAADPTu7sOTLSpa1e1C6Y4+6u; _scid_r=kgkg4MW064d6cqdRAxN8Wree7GrWNTlPzguJmg; __gads=ID=ee5a178be1b0f05b:T=1726648356:RT=1726649146:S=ALNI_MZyw5qAhL0xPStB1n0AO9wKPZbejw; __gpi=UID=00000f0545f704b8:T=1726648356:RT=1726649146:S=ALNI_MYrEpGfB3M0ndF1X9ENeU-rzw_IGA; __eoi=ID=9ff938aca5b93c6d:T=1726648356:RT=1726649146:S=AA-AfjYiQlITvzkWMzSsBbMpghn4; lotame_domain_check=dubizzle.com; _ga_LRML1YM9GH=GS1.1.1726648355.1.1.1726649161.42.0.0; SESSION=%7B%22sessionKey%22%3A%2273eea8e6-6c92-480e-8a23-047177ef6aa4%22%2C%22sessionStartTime%22%3A%222024-09-18T08%3A32%3A40.070Z%22%2C%22sessionMaxTime%22%3A1800%2C%22customIdentifiersToTrack%22%3A%5B%5D%2C%22sessionExpiryTime%22%3A1726650961599%2C%22numberOfSessions%22%3A1%7D',
        'priority': 'u=1, i',
        'referer': 'https://dubai.dubizzle.com/motors/used-cars/chevrolet/camaro/2024/5/27/ss-fifty-50th-anniversary-first-owner-full-2-973---0a7018d4f4cc46428e05e644e8ded161/',
        'sec-ch-ua': '"Chromium";v="128", "Not;A=Brand";v="24", "Google Chrome";v="128"',
        'sec-ch-ua-mobile': '?0',
        'sec-ch-ua-platform': '"Windows"',
        'sec-fetch-dest': 'empty',
        'sec-fetch-mode': 'cors',
        'sec-fetch-site': 'same-origin',
        'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/128.0.0.0 Safari/537.36',
        'x-access-token': 'eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6ImUwOGEyOWFiLWI4YmUtNGE5Ni04M2M0LTExYmM3ZjIwZTdkNyIsImlzcyI6ImR1Yml6emxlLmNvbSIsImF1ZCI6WyJkdWJpenpsZS5jb20iXSwic3ViIjoiZTA4YTI5YWItYjhiZS00YTk2LTgzYzQtMTFiYzdmMjBlN2Q3IiwiZXhwIjoxNzI2NzM1NTQxLCJpYXQiOjE3MjY2NDkxNDEsImp0aSI6IjBlYjdjZjQyLTY0MGYtNGVkOS1iOTMxLTM1ZjQyYjFjYzY1NSIsInR5cCI6IkFjY2VzcyBKV1QgVG9rZW4iLCJmbGFncyI6eyJsb2dnZWRfaW4iOmZhbHNlLCJpc19zdGFmZiI6ZmFsc2UsImlzX3N1cGVydXNlciI6ZmFsc2UsImlzX3Byb3BlcnR5X2FnZW50IjpmYWxzZSwiaXNfbGFuZGxvcmQiOmZhbHNlLCJpc19tb3RvcnNfYWdlbnQiOmZhbHNlLCJpc19qb2JzX2FnZW50IjpmYWxzZSwiaXNfbGVhZF9ibG9jayI6ZmFsc2UsImlzX2xlYWRfc29mdF9ibG9jayI6ZmFsc2UsImlzX2NoYXRfYmxvY2siOmZhbHNlLCJpc19jaGF0X3NvZnRfYmxvY2siOmZhbHNlLCJoYXNfY2FsbF90cmFja2luZyI6ZmFsc2UsImNhbl9yZXBvcnQiOnRydWUsImhpZGVfcHVibGljX3Byb2ZpbGUiOmZhbHNlLCJpc19tb3RvcnNfYWdlbnRfZW1wbG95ZWVfdXNlciI6ZmFsc2UsImlzX3JlbnRhbF9jYXJzX2FnZW50IjpmYWxzZX0sInVzZXJfZGF0YSI6eyJ1c2VyX2lkIjpudWxsLCJnZW5kZXIiOm51bGwsIm5hdGlvbmFsaXR5IjpudWxsLCJlZHVjYXRpb24iOm51bGwsInJvbGUiOm51bGwsImRvYiI6bnVsbCwiYWdlIjpudWxsLCJmaXJzdF9uYW1lIjoiIiwibGFzdF9uYW1lIjoiIiwiZW1haWwiOiIiLCJwaG9uZSI6IiIsInZlcmlmaWNhdGlvbl9zdGF0dXMiOiJub3RfYXBwbGllZCIsInZlcmlmaWNhdGlvbl9zdGFydF9kYXRlIjpudWxsfX0.J9UjbnprI-uyW6XSefhjS_sSebSFZcRJl2wK3mMrsGvxKA_by5EnSMpN5TRS10_5ZSDUe3vMm9CtnjQo7gfP5NWfKFbZGCzIAA987rKXEbcvNiZ6077obZR0kpqDXGE5gp3fx3v-2KOy0XKzRU4Aaag8gMmUkRPWJ2hhdsJJlY1cZqCVGuWbefz_YK9IqN77vEC-qoKdW9cgNtWlzW9PdbO39C6-79BMSa9or0Uj0LfZdtC5-I3ZpCZXb7XmOuv2WJ9w_orN5eHebDvoLCouHjQCT0GKj0XovLUpFJI60lU9HmloMktBk-iqqtc2NsPpc4Fda7sw_pT_QmY13MasOIlJFVpn0oa_in5g2xPN-IVdXstc6n42SOG39egNxzp2nR1XNQo9kaJ_kSPgqbtN7V2ADvzeIGdExULtVOTBEoWrrv1-ybSB681xYtcrJfVUkoccRKSw6gslBU4oKps0oRTf1pXb8C5ePN8AtUG01EJW9qq9zqNoCg7_3-2Tr9Z1dmMyy9wpy4kq8fLhyUvWIlwjPcskAVJhx7vkkjuWtlQTPEcyPlRW6TPVOmZ4Z9g0tnZc8YYKPo29c5NK8S9AW9fYvnOUm1EvXjKDGrBanbujX_niGgX5HzAsqMFwOIB2z_Rbvw-hLdZEu5_ZiPd5oUtutOEUwz9Zob60Me5ZOzc',
        # Add other headers as needed
    })

print(response.text)

URL Example: https://dubai.dubizzle.com/motors/used-cars/chevrolet/camaro/2024/5/27/ss-fifty-50th-anniversary-first-owner-full-2-973---0a7018d4f4cc46428e05e644e8ded161/

Click Show Phone Number button.