opsdisk / pagodo

pagodo (Passive Google Dork) - Automate Google Hacking Database scraping and searching
GNU General Public License v3.0
2.83k stars 495 forks source link

Rotate through all given proxies before letting yagoogle handle the waiting #70

Open sporetec opened 2 years ago

sporetec commented 2 years ago

This rework lets pagodo:

opsdisk commented 2 years ago

Thanks for submitting a PR @sporetec! Just a heads up, it may be a week or two until I can take a good look at it.

opsdisk commented 2 years ago

Hey @sporetec - haven't forgotten about this! Hoping to dig into the PR soon.

opsdisk commented 2 years ago

@sporetec - Would it be possible to tackle this feature first?

"Rotate through all given proxies before letting yagoogle handle the 429 to reduce execution time"

To be clear, you don't want the script to automatically go to sleep if an HTTP 429 is detected, especially if there are other proxies that can be tried?

As it is currently written, since yagooglesearch_manages_http_429s is not set during the instantiation of a yagooglesearch.SearchClient object (like is done here https://github.com/opsdisk/pagodo/blob/master/pagodo.py#L200), it defaults to True https://github.com/opsdisk/yagooglesearch/blob/master/yagooglesearch/__init__.py#L82

DannyCork commented 8 months ago

would be good to get this into master