search

FanfictionBot / reddit-ffn-bot

A reddit auto reply bot.
53 stars 18 forks source link

Google backend will not work forever #107

Open tusing opened 6 years ago

tusing commented 6 years ago

When the bot receives a request, it searches for that request in Google (adding on, for example, "site:fanfiction.net" to the query if the request was for fanfiction.net).

The script we use to search Google will not work forever. Google doesn't really have an API and it's cracking down on script-based access. To ensure the stability of the bot, we'll want to use a sanctioned API-based method of crawling Google.

An option might be Google's Custom Search APIs, where we define Custom Search over the two or three websites we support. We would then replace our call to the Google script with a call to Custom Search.

ThrawnCA commented 5 years ago

Are there alternative search engines that could do the job? StartPage, DuckDuckGo, etc?

Thatgirl318 commented 4 years ago

Is there any reason we can't use the fanfiction search funciton? Is it just bad or lack of API also?

Thatgirl318 commented 4 years ago

We might also be able to use the Custom Search Restriction json API

tusing commented 4 years ago

The reason is that I have no time to work on this anymore :( But I will gladly take pull requests that fix the search functionality! Would you be willing to work on this?

Thatgirl318 commented 4 years ago

Yeah totally (once finals are done). I just didn't want to go down a path you tried and already abandoned.

Thatgirl318 commented 4 years ago

Would it be possible for me to get the logs?