search

AskToni / asktoniapi

Life changing food recommendations
4 stars 1 forks source link

Getting new data #28

Open wfengk opened 6 years ago

wfengk commented 6 years ago

Need feedback

So right now we are at about ~1000 restaurants, and we will need to get more data eventually.

I was thinking that each time we use the search to find new restaurants, we could query yelp and add that new restaurant to our database

asktoni

Thoughts?

RedTn commented 6 years ago

Hmm, it seems like the crawler should be doing this. From a technical standpoint, other than the 3 RTT calls for each missing restaurant (Client -> Ask Toni, Client -> Yelp, Client -> Ask Toni), the comparison logic is going to be way too complex. Restaurant with same names, then compare address, what if we want to overwrite AskToni version?

wfengk commented 6 years ago

I could build the Yelp querying functionality as a REST endpoint? that way you just need to make one additional call?

My problem with the crawler is that I wont get any more data after the 1000 restaurants (yelp cap) unless I search for one specifically