opsdisk / pagodo

pagodo (Passive Google Dork) - Automate Google Hacking Database scraping and searching
GNU General Public License v3.0
2.66k stars 486 forks source link

Rotate through all given proxies before letting yagoogle handle the waiting #70

Open sporetec opened 1 year ago

sporetec commented 1 year ago

This rework lets pagodo:

opsdisk commented 1 year ago

Thanks for submitting a PR @sporetec! Just a heads up, it may be a week or two until I can take a good look at it.

opsdisk commented 1 year ago

Hey @sporetec - haven't forgotten about this! Hoping to dig into the PR soon.

opsdisk commented 1 year ago

@sporetec - Would it be possible to tackle this feature first?

"Rotate through all given proxies before letting yagoogle handle the 429 to reduce execution time"

To be clear, you don't want the script to automatically go to sleep if an HTTP 429 is detected, especially if there are other proxies that can be tried?

As it is currently written, since yagooglesearch_manages_http_429s is not set during the instantiation of a yagooglesearch.SearchClient object (like is done here https://github.com/opsdisk/pagodo/blob/master/pagodo.py#L200), it defaults to True https://github.com/opsdisk/yagooglesearch/blob/master/yagooglesearch/__init__.py#L82

DannyCork commented 3 months ago

would be good to get this into master