search

WilianZilv / angoole

Google bridge for Facebook Messenger
41 stars 4 forks source link

Add proxy to avoid Google limitations #1

Open renandecarlo opened 4 years ago

renandecarlo commented 4 years ago

The SERP npm module provides an easy way to use proxies when getting the results. This can probably be used together with a proxy API to avoid Google limitations.

macabeus commented 4 years ago

We could use duckduckgo, bing and others providers in order to avoid Google's limitations.

renandecarlo commented 4 years ago

Bing and Yahoo also do have limits, duckduckgo would definitely be a better alternative than proxies though, if it allows crawling.

renandecarlo commented 4 years ago

DuckDuckGo has an official API and it seems to have a more abroad limitation. It does not implicitly say how much is it though. It's very straight forward, you just access the api url: https://api.duckduckgo.com/?q=DuckDuckGo&format=json&pretty=1

Though, doing it like so doesn't include the visual feedback, which is needed (or not) for the Messenger response. We could, either (1) try scraping the ddg website directly and see if it have any limits; (2) use the official API and create a temporary local html to show the results, and get the imagem from it; or (3) return the results in text to the user

macabeus commented 4 years ago

I think that the third option is better, because following this way the user doesn't need to download an image on the cellphone (that could be bad on 2G internet, for example).

vitordino commented 4 years ago

probably we could be inspired by some CLI work on this field:

renandecarlo commented 4 years ago

@vitordino these projects use the user IP addresses, so there shouldn't be any limitations. This one, however, needs to do every request from a server, so the limits apply