search

thewhiteh4t / pwnedOrNot

OSINT Tool for Finding Passwords of Compromised Email Addresses
MIT License
2.19k stars 318 forks source link

Error:could not collect tokens | 403 Client Error #20

Closed stack00 closed 5 years ago

stack00 commented 5 years ago

HI , i was just trying to use this tool just as described in demo video and using 12345@gmail.com to test for data breaches and i am getting following error

] Created by : thewhiteh4t [>] Version : 1.1.7

[+] Checking for updates...

[+] Script is up-to-date...

[+] Bypassing Cloudflare Restriction... ERROR:root:'https://haveibeenpwned.com/api/v2/breachedaccount/test@example.com' returned an error. Could not collect tokens. Traceback (most recent call last): File "pwnedornot.py", line 273, in main() File "pwnedornot.py", line 64, in main cookies, user_agent = cfscrape.get_tokens('https://haveibeenpwned.com/api/v2/breachedaccount/test@example.com', user_agent='pwnedornot') File "/usr/local/lib/python3.7/dist-packages/cfscrape/init.py", line 182, in get_tokens resp.raise_for_status() File "/usr/lib/python3/dist-packages/requests/models.py", line 940, in raise_for_status raise HTTPError(http_error_msg, response=self) requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://haveibeenpwned.com/api/v2/breachedaccount/test@example.com root@kali:~/Downloads/breach/pwnedOrNot#

the platform i am using this on are as follows:

root@kali:~/Downloads/breach/pwnedOrNot# uname -r 4.19.0-kali3-amd64 root@kali:~/Downloads/breach/pwnedOrNot# uname -v

1 SMP Debian 4.19.20-1kali1 (2019-02-14)

Screenshot from 2019-04-21 07-59-49

jepunband commented 5 years ago

same issue here.

thewhiteh4t commented 5 years ago

@mohsin2928 @jepunband i have contacted Troy Hunt ( Creator of HaveIbeenPwned ), I am waiting for his reply, I will try to resolve this if the error is caused by my tool, because personally i am not getting this error, not even once, so i am not able to reproduce this error

stack00 commented 5 years ago

@jepunband since when are you getting this error, i used this tool today and got this @thewhiteh4t thanks a lot for looking into this and for your support , when i try to access https://haveibeenpwned.com/api/v2/breachedaccount/test@example.com i am getting an error that is attached. Screenshot from 2019-04-21 12-52-29

stack00 commented 5 years ago

Steps to reproduce it;

root@kali:~/Downloads/breach/pwnedOrNot# python3 pwnedornot.py -e test@gmail.com

afte this i hit enter and get the error message that i have attached earlier

jepunband commented 5 years ago

@mohsin2928 i downloaded and tried it today.

thewhiteh4t commented 5 years ago

@mohsin2928 you will always get the error if you try it with a browser because if you read the api documentation, all normal browser User Agents are blocked by default, API specifically needs tool / script name as the User Agent, and i am using pwnedornot as the User Agent

Screenshot from 2019-04-19 11-02-31

stack00 commented 5 years ago

@thewhiteh4t so how can i proceed now,what to do ?

thewhiteh4t commented 5 years ago

@mohsin2928 unfortunately you will have to wait until Troy Hunt replies...

Also here is my result for test@gmail.com

Screenshot from 2019-04-21 23-20-24

Medorna commented 5 years ago

Same issue here.

thewhiteh4t commented 5 years ago

@mohsin2928 @jepunband @Medorna i have updated the tool and removed CFScrape for now, please test and report if you are still getting errors...

stack00 commented 5 years ago

@thewhiteh4t thanks alot, its working fine as if now,i will keep using it and update the thread if i encounter any more bugs, but again thanks for working on this

jepunband commented 5 years ago

@mohsin2928 @jepunband i have contacted Troy Hunt ( Creator of HaveIbeenPwned ), I am waiting for his reply, I will try to resolve this if the error is caused by my tool, because personally i am not getting this error, not even once, so i am not able to reproduce this error

getting a different error now ..

Traceback (most recent call last): File "pwnedornot.py", line 272, in main() File "pwnedornot.py", line 73, in main check() File "pwnedornot.py", line 110, in check simple_out = json.loads(json_out) File "/usr/lib/python3.7/json/init.py", line 338, in loads s, 0) json.decoder.JSONDecodeError: Unexpected UTF-8 BOM (decode using utf-8-sig): line 1 column 1 (char 0)

thewhiteh4t commented 5 years ago

@mohsin2928 thankyou for using my tool and feedback!!

thewhiteh4t commented 5 years ago

@jepunband how can i reproduce this error?

jepunband commented 5 years ago

@jepunband how can i reproduce this error?

python3 pwnedornot.py -e jessy@gmail.com

Screenshot at 2019-04-22 16-59-15

thewhiteh4t commented 5 years ago

@mohsin2928 @jepunband i have updated again, fixed decode errors (found few more), and getting dumps is a bit faster now, please test and report, i have also removed install scripts and replaced with requirements file as CFScrape is not required anymore, overall tool is much lighter

jepunband commented 5 years ago

@mohsin2928 @jepunband i have updated again, fixed decode errors (found few more), and getting dumps is a bit faster now, please test and report, i have also removed install scripts and replaced with requirements file as CFScrape is not required anymore, overall tool is much lighter

hi still same error:

Traceback (most recent call last): File "pwnedornot.py", line 247, in main() File "pwnedornot.py", line 51, in main check() File "pwnedornot.py", line 88, in check simple_out = json.loads(json_out) File "/usr/lib/python3.7/json/init.py", line 338, in loads s, 0) json.decoder.JSONDecodeError: Unexpected UTF-8 BOM (decode using utf-8-sig): line 1 column 1 (char 0)

thewhiteh4t commented 5 years ago

@jepunband i tested with jessy@gmail.com i am not getting any errors :O

stack00 commented 5 years ago

@thewhiteh4t thanks for tool improvements, could you please mention the full command if we have to search for breach on a particular domain,its working fine on emails but for domains its showing an argument is missing !!!!

thewhiteh4t commented 5 years ago

@mohsin2928 in the README i have mentioned the command under examples

python3 pwnedornot.py -e test@gmail.com -d adobe.com

Screenshot from 2019-04-23 20-26-51

jepunband commented 5 years ago

unfortunately it didnt work for me ... same errors as before :

Traceback (most recent call last): File "pwnedornot.py", line 247, in main() File "pwnedornot.py", line 51, in main check() File "pwnedornot.py", line 88, in check simple_out = json.loads(json_out) File "/usr/lib/python3.7/json/init.py", line 338, in loads s, 0) json.decoder.JSONDecodeError: Unexpected UTF-8 BOM (decode using utf-8-sig): line 1 column 1 (char 0)

thewhiteh4t commented 5 years ago

@jepunband are you getting this error for every email address or a specific one? if you are getting this on every email, which operating system are you using and which shell are you using? because i told my friends to test it and we are not getting this error

jepunband commented 5 years ago

hi it happens on every email i tested. i'm on parrot sec. Linux parrot 4.19.0-parrot2-28t-amd64

thewhiteh4t commented 5 years ago

@jepunband i will test on parrot os, lets see what the problem is

thewhiteh4t commented 5 years ago

@jepunband I am not getting any errors on latest parrot sec os, so problem is not with the tool or parrot os, it's a problem in your configuration and i can't help you with that

cat /etc/os-release
PRETTY_NAME="Parrot GNU/Linux 4.6"
NAME="Parrot GNU/Linux"
ID=parrot
ID_LIKE=debian
VERSION="4.6"
uname -r
4.19.0-parrot1-13t-amd64

Screenshot from 2019-04-24 01-38-24

jepunband commented 5 years ago

hmmm, strange even with a fresh installation of parrot i still get the same errors.. : ( the difference is you're using 4.6 and i'm on 4.5.

thewhiteh4t commented 5 years ago

@jepunband found the problem, it's cloudflare ddos protection, it is blocking the request and giving error 503, so when the script proceeds data is basically empty and that's why you are getting a json decode error because there's nothing to decode

ostes commented 5 years ago

please if you know a solution pplease tell me . this is a output "Request Blocked by Cloudflare" my internet working good !

thewhiteh4t commented 5 years ago

@ostes execute git pull and try again pushed a small update

ostes commented 5 years ago

thanks i Tried [-] Error 403 : Request Blocked by Cloudflare

[+] Completed in 0.5606667995452881 seconds. root@kali:~/pwnedOrNot# git pull Already up to date. root@kali:~/pwnedOrNot#

ostes commented 5 years ago

same error

thewhiteh4t commented 5 years ago

@ostes version? and which country?

twm1016 commented 5 years ago

@thewhiteh4t Hi I come from Hong Kong and I has same problem of the "Error 403 : Request Blocked by Cloudflare" too. For the application, I am using ubuntu 16.04 LTS and running by VM. And you can check below share link for this.

https://drive.google.com/open?id=1zLLcnKkBxMHA0CkFyfWmeq6RlSSKnB-f

Please help. Thanks.

ostes commented 5 years ago

@ostes version? and which country?

thanks for your answer . I live in Argentina and I implemented over kali linux

HackingEnVivo commented 5 years ago

Hi, I get Error 403: Request Blocked by Cloudflare CLOUD

guilhergomes commented 5 years ago

Good morning, I have restricted access on cloudflare. I'm using Kali on Raspberry pi 3 b, using only one email, I was able to use more when I used the -f command this fault occurred

image

thewhiteh4t commented 5 years ago

@guilhergomes please share the exact command you used

guilhergomes commented 5 years ago

@guilhergomes please share the exact command you used root@kali:~/pwnedOrNot# python3 pwnedornot.py -f gui.txt

BaRRaKudaRain commented 5 years ago

Should you use VPN?

-----Original Message----- From: "twm1016" notifications@github.com Sent: ‎5/‎8/‎2019 2:31 PM To: "thewhiteh4t/pwnedOrNot" pwnedOrNot@noreply.github.com Cc: "Subscribed" subscribed@noreply.github.com Subject: Re: [thewhiteh4t/pwnedOrNot] Error:could not collect tokens | 403Client Error (#20)

@ostes Hi I come from Hong Kong and I has same problem of the "Error 403 : Request Blocked by Cloudflare" too. For the application, I am using ubuntu 16.04 LTS and running by VM. Please help. Thanks. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

twm1016 commented 5 years ago

@BaRRaKudaRain I didn’t connected any VPN

thewhiteh4t commented 5 years ago

@twm1016 @BaRRaKudaRain are you guys willing to send some emails?

i printed the response body for error 403 :

<!DOCTYPE html>
<html>
<head>
  <title>Request Blocked</title>
  <meta charset="UTF-8" />
  <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
</head>
<body>
  <h1>You have been blocked from accessing this resource on Have I Been Pwned</h1>

  <p>This may be due to violating one or more of <a href="https://haveibeenpwned.com/API/v2#AcceptableUse">the acceptable use terms of the API</a> or for not complying with <a href="https://haveibeenpwned.com/API/v2">the API specifications</a>. It may also be due to your traffic patterns being similar to other users who may have violated the acceptable use terms.</p>

  <p>Tips to avoid requests being blocked include:</p>
  <ol>
    <li>Stick well within the published rate limit</li>
    <li>Don't distribute requests over multiple IP addresses in an attempt to circumvent the rate limit</li>
    <li>Only query the email addresses of people who have a reasonable expectation that you should do so</li>
    <li>Avoid prolonged querying of the API over an extended period of time</li>
    <li>Clearly identify your app in the user agent string <a href="https://haveibeenpwned.com/API/v2#UserAgent">per the API docs</a></li>
    <li>If you reach out to ask for help after seeing this message and need to be directed to the previous point, you agree to <a href="https://haveibeenpwned.com/Donate">make a donation</a></li>
  </ol>
  <p>If you believe your request meets these requirements and was still blocked, please send this entire response body along with any communication you send regarding the error.</p>
  <div class="cf-error-details cf-error-1020">
  <h1>Access denied</h1>
  <p>This website is using a security service to protect itself from online attacks.</p>
  <ul class="cferror_details">
    <li>Ray ID: 4d54a3644d30633b</li>
    <li>Timestamp: 2019-05-11 13:53:17 UTC</li>
    <li>Your IP address: 101.109.35.96</li>
    <li class="XXX_no_wrap_overflow_hidden">Requested URL: haveibeenpwned.com/api/v2/breachedaccount/test@gmail.com </li>
    <li>Error reference number: 1020</li>
    <li>Server ID: FL_104F19</li>
    <li>User-Agent: pwnedornot</li>
  </ul>
</div>
</body>
</html>

If you believe your request meets these requirements and was still blocked, please send this entire response body along with any communication you send regarding the error

now I have followed the api docs properly and tool works for me and some other people but for many others its not working, if i use a https proxy with a useragent pwnedornot or pwned-or-not , i am also getting 403 for both, also take a look at this :

Screenshot from 2019-04-19 11-02-31

it's somewhat location based i guess, it works for some regions and in other places it does not, because you guys are not using any proxy or vpn and actually both are not required

thewhiteh4t commented 5 years ago

@twm1016 @BaRRaKudaRain i have made few changes, tell me if it works... cd pwnedOrNot git pull

New version is 1.2.2

Edit : reverted to old useragent, new version is 1.2.3, you can now see response body for error 403, workaround used in 1.2.2 is not working after some time

BaRRaKudaRain commented 5 years ago

I mean you need to try connect the VPN. I'm not sure, but it can be helpful. And sorry for my bad English.

-----Original Message----- From: "twm1016" notifications@github.com Sent: ‎5/‎12/‎2019 3:33 PM To: "thewhiteh4t/pwnedOrNot" pwnedOrNot@noreply.github.com Cc: "Ilya Lapa" barrakudarain@gmail.com; "Mention" mention@noreply.github.com Subject: Re: [thewhiteh4t/pwnedOrNot] Error:could not collect tokens | 403Client Error (#20)

@BaRRaKudaRain I didn’t connected any VPN — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

thewhiteh4t commented 5 years ago

Give it a go @BaRRaKudaRain, try different regions and try to run it without vpn also because useragent pwnedornot is blocked by the api and i have changed useragent in new update so it might work for you now

twm1016 commented 5 years ago

@twm1016 @BaRRaKudaRain i have made few changes, tell me if it works... cd pwnedOrNot git pull

New version is 1.2.2

Edit : reverted to old useragent, new version is 1.2.3, you can now see response body for error 403, workaround used in 1.2.2 is not working after some time

@thewhiteh4t I am still not working for this, and I showed same error as you was posted in past, should I send you those response by email? If yes where can I find your email? Thanks.

thewhiteh4t commented 5 years ago

@twm1016 no you don't have to send the response to me, i have included the email on which you have to send the response, troy@troyhunt.com

twm1016 commented 5 years ago

@thewhiteh4t OK, Thanks for the helping.

@twm1016 no you don't have to send the response to me, i have included the email on which you have to send the response, troy@troyhunt.com

thewhiteh4t commented 5 years ago

Conducted multiple tests with multiple proxies :

Argentina --> 5 proxies tested [ https ] --> [5/5] Error 403 | Request Blocked Hong Kong --> 5 proxies tested [https] --> [5/5] Error 403 | Request Blocked USA --> 5 proxies tested [https] --> [5/5] Error 200 | OK India --> 5 proxies tested [https] --> [5/5] Error 200 | OK

haveibeenpwned API is blocking requests by region, i can't do much about this :(