search

opsdisk / yagooglesearch

Yet another googlesearch - A Python library for executing intelligent, realistic-looking, and tunable Google searches.
BSD 3-Clause "New" or "Revised" License
249 stars 43 forks source link

added captcha handle with unicaps #17

Closed tjengbudi closed 1 year ago

tjengbudi commented 1 year ago

hello is there any chance to added captcha handle with unicaps library? it might nice to have alternative to escaped blocked by google

what you think about this?

opsdisk commented 1 year ago

Hi @tjengbudi - Although using something like https://github.com/sergey-scat/unicaps to solve captchas is possible, it's not a capability I want to support at this time. I'd recommend:

tjengbudi commented 1 year ago

I tried using proxy. However I fetch from free proxy and never get success to connect.

I already see the 429 error and try with something to change in the get_page method. But I am not success with that.

Unicaps or something like this is optional. Something really free of use will be nice. But I also need something really fast to implement to cover before trying more with proxy and/or others

On Sat, Dec 10, 2022, 7:34 PM opsdisk @.***> wrote:

Hi @tjengbudi https://github.com/tjengbudi - Although using something like https://github.com/sergey-scat/unicaps to solve captchas is possible, it's not a capability I want to support at this time. I'd recommend:

— Reply to this email directly, view it on GitHub https://github.com/opsdisk/yagooglesearch/issues/17#issuecomment-1345255352, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAXGEP6LEDBMUUEZNSIXGXLWMR2DRANCNFSM6AAAAAASZ5TREU . You are receiving this because you were mentioned.Message ID: @.***>

opsdisk commented 1 year ago

Free proxies are abused and probably already blocked by Google, so I'd stay away from them. Sounds like you have some options to explore/investigate. I'll keep this issue open for a while if there are any questions around yagooglesearch.

tjengbudi commented 1 year ago

Could you recommend proxy service that can used with this?

On Sat, Dec 10, 2022, 8:24 PM opsdisk @.***> wrote:

Free proxies are abused and probably already blocked by Google, so I'd stay away from them. Sounds like you have some options to explore/investigate. I'll keep this issue open for a while if there are any questions around yagooglesearch.

— Reply to this email directly, view it on GitHub https://github.com/opsdisk/yagooglesearch/issues/17#issuecomment-1345264499, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAXGEP4BNJPE7HQIBKTDIMDWMR777ANCNFSM6AAAAAASZ5TREU . You are receiving this because you were mentioned.Message ID: @.***>

opsdisk commented 1 year ago

In general, you'll have to do it yourself. You can deploy your own Virtual Private Servers and use ssh -D for SOCKS proxies or leverage something like this https://github.com/Ge0rg3/requests-ip-rotator

tjengbudi commented 1 year ago

I try several way. 1: using tor as proxy. However Google always found it and block 2: using Unicaps. First time now working it cause the parameter. I found that your params always used num= it will always got blocked by google After remove num parameter, captcha can resolved. But when create new instance it will be blocked again. So I use first create instance, and force change the query and update the url. 2query fetch after resolved captcha. Next trial might be saved the cookies

opsdisk commented 1 year ago
  1. You'll almost always get blocked if you're using Tor.
  2. It's open source, feel free to change it so it works in your specific case.
opsdisk commented 1 year ago

Closing this one out.