Closed photkey closed 7 months ago
My current solution is to use a new proxy for data collection if the result is empty. I will try this up to 10 times. This method can solve the current problem in most cases, but it is not perfect. For example, if there are keywords that do not have any results or if I encounter 10 problematic proxy IPs in a row. It would be best if yagooglesearch could visually indicate whether there is a network issue when returning results, similar to the mentioned HTTP_ERROR
.
Hi @photkey apologies it took a while to get back to you. Two quick things...
Do you get back a specific HTTP error code (401, 404, etc.) when results are empty? The HTTP 429 is easy because that relates to interacting with Google itself. If empty results are returned, that may be because there are literally no results...and how is yagooglesearch supposed to know the difference between empty results and your hypothesis that it is networked related?
Your current solution to have your script handle the empty results is correct in my opinion. Your script understands the context of the network, proxies, timing, etc. much better than yagooglesearch does. Again, check out https://github.com/opsdisk/yagooglesearch#multiple-proxies
If I'm not understanding your question or situation correctly, let me know.
Any updates on this @photkey ?
@opsdisk Sorry, no new developments, I just solved the problem with the previously mentioned method, it's not perfect, but the script still works!
Great to hear, I'll close this issue out then.
I used a proxy IP for the search, and sometimes when using the exact same keywords, the search results are empty, while other times there are results. I suspect it is due to the quality of the proxy IP, but I am not sure how to confirm the cause of the problem or how to resolve it. Can this library add a return similar to
HTTP_429_DETECTED
, which is a prompt returned in case of network issues, for example:HTTP_ERROR
? This can be used in the script to handle this result, such as changing the proxy