obheda12 / GitDorker

A Python program to scrape secrets from GitHub through usage of a large repository of dorks.
2.3k stars 419 forks source link

error occurred: https://docs.github.com/v3/search/ #2

Closed xaeroborg closed 4 years ago

xaeroborg commented 4 years ago

Hello Omar, I am getting this error when I run the following command python3 /home/GitDorker/GitDorker.py -tf /home/git_token.txt -org target -d /home/GitDorker/Dorks/alldorks.txt -o gitdorks_target.txt

gitdorker_error

0xth commented 4 years ago

Same issue

0xth commented 4 years ago

error occurred: https://docs.github.com/rest

obheda12 commented 4 years ago

check the alldorks file. Are there any blank lines? also change "target" to an actual company name.

xaeroborg commented 4 years ago

Actually I used the actual target name instead of "target" but the command syntax is the same, anyways I made it work. @0xth try this python3 /home/GitDorker/GitDorker.py -tf /home/git_token.txt -q target.com -d /home/GitDorker/Dorks/alldorks.txt -o gitdorks_target.txt

@obheda12 your tool works amazing, its the command syntax in your blog that is making such error or correct me if I am wrong. But the initial syntax I tried and opened the issue with above is exactly the same from your blog.

obheda12 commented 4 years ago

what did you put for the "-org " field inititally? GitHub has a different system for organizations. You can't input a domain in "-org" but you can with "-q" as you did in your above command.

Hope that clears up any confusion and glad you like it :)

obheda12 commented 4 years ago

just to clear up any confusion: say I want to search on tesla's github organization. Tesla on github is not "tesla" rather it is "teslamotors". Therefore the following command would be as such:

python3 GitDorker.py -tf tokensfile -org teslamotors -d dorksfile

If i wanted to search on tesla's domain, "tesla.com", then I will specify the command as you did @xaeroborg. However please note that searching on a domain vs within an organization's account will yield different results.

obheda12 commented 4 years ago

If there are no further concerns I will be closing this issue.

xaeroborg commented 4 years ago

just to clear up any confusion: say I want to search on tesla's github organization. Tesla on github is not "tesla" rather it is "teslamotors". Therefore the following command would be as such:

python3 GitDorker.py -tf tokensfile -org teslamotors -d dorksfile

If i wanted to search on tesla's domain, "tesla.com", then I will specify the command as you did @xaeroborg. However please note that searching on a domain vs within an organization's account will yield different results.

Thanks for the tip/suggestion @obheda12

xaeroborg commented 4 years ago

If there are no further concerns I will be closing this issue.

Yes you may close the issue.

xaeroborg commented 4 years ago

what did you put for the "-org " field inititally? GitHub has a different system for organizations. You can't input a domain in "-org" but you can with "-q" as you did in your above command.

Hope that clears up any confusion and glad you like it :)

for the -org I used "target" instead of "target.com" for example -org google

Anyways your explanation above has cleared everything.

obheda12 commented 4 years ago

Glad to hear it, happy hunting! Keep me updated if you guys get any bounties!

For now I'm closing this issue

slowmistio commented 4 years ago

the same error

[-] error occurred: https://developer.github.com/v3/#abuse-rate-limits [-] error occurred: https://developer.github.com/v3/#abuse-rate-limits [-] error occurred: https://developer.github.com/v3/#abuse-rate-limits [-] error occurred: https://developer.github.com/v3/#abuse-rate-limits [-] error occurred: https://developer.github.com/v3/#abuse-rate-limits [-] error occurred: https://developer.github.com/v3/#abuse-rate-limits [-] error occurred: https://developer.github.com/v3/#abuse-rate-limits [-] error occurred: https://developer.github.com/v3/#abuse-rate-limits [-] error occurred: https://developer.github.com/v3/#abuse-rate-limits [-] error occurred: https://developer.github.com/v3/#abuse-rate-limits [-] error occurred: https://developer.github.com/v3/#abuse-rate-limits [-] error occurred: https://developer.github.com/v3/#abuse-rate-limits

yassineaboukir commented 4 years ago

the same error

[-] error occurred: https://developer.github.com/v3/#abuse-rate-limits [-] error occurred: https://developer.github.com/v3/#abuse-rate-limits [-] error occurred: https://developer.github.com/v3/#abuse-rate-limits [-] error occurred: https://developer.github.com/v3/#abuse-rate-limits [-] error occurred: https://developer.github.com/v3/#abuse-rate-limits [-] error occurred: https://developer.github.com/v3/#abuse-rate-limits [-] error occurred: https://developer.github.com/v3/#abuse-rate-limits [-] error occurred: https://developer.github.com/v3/#abuse-rate-limits [-] error occurred: https://developer.github.com/v3/#abuse-rate-limits [-] error occurred: https://developer.github.com/v3/#abuse-rate-limits [-] error occurred: https://developer.github.com/v3/#abuse-rate-limits [-] error occurred: https://developer.github.com/v3/#abuse-rate-limits

As mentioned above:

obheda12 commented 4 years ago

Per yassines response, please incorporate additional tokens so GitDorker may rotate between them. You have been rate limited as GitHub only allows 30 requests every minute per account.

In order to circumvent this you May use multiple tokens form unique accounts in the token file and change the sleep function from sleeping every 30 requests to the number of requests enabled by having multiple tokens. For example 3 unique tokens from 3 unique accounts would enable you to have 90 requests per minute.

obheda12 commented 4 years ago

I will be closing this if there are no further comments

daddyhack commented 4 years ago

error occurred: https://docs.github.com/rest

i am also getting this error. have added multiple tokens.

yassineaboukir commented 4 years ago

error occurred: https://docs.github.com/rest

i am also getting this error. have added multiple tokens.

Perhaps try to run it without mutithreading.

obheda12 commented 4 years ago

Please note that utilizing personal access tokens from the same account will NOT increase the rate limit. Rate limits in github are tied to unique accounts. You must use personal access tokens from multiple unique accounts should you like to bypass the 30 request per minute limit.

Regarding @yassineaboukir comment, he is correct. The multithreading can sparse out the tokens. I will revisit this functionality in the future to better it. For now please try to keep threads to one if you are experiencing rate limits.

obheda12 commented 4 years ago

Closing this issue as additional comments are becoming redundant

monustark commented 2 years ago

error occurred: https://docs.github.com/v3/search/ In Progress $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$ 0/241

problem