Lazy Geocoding

[marmat]

A possibility to reduce the amount of geocoding requests is to geocode the locations only if the user actually clicks on the "Map" tab (otherwise the user doesn't need the locations anyway). Furthermore it would reduce the peaks on the Maps API server side since the clients wouldn't geocode the location of a freshly opened hangout all at the same time.

A problem might be that just-in-time geocoding could be a bit slow. Any ideas for taking care of that?

We could:

• start geocoding in the background as soon as the "Map" button gets hovered (by accident or on purpose) - probably not a big improvement
• "animate" the insertion of users on the map (that means instead of waiting until all hangouts are fetched, show them one by one, as soon as their specific location arrives)

[mohamedmansour]

@kaktus621 @johnbc alright, we need this :( Google reduced the load of lookupUsers from 40 to 12, perhaps we can use both approaches to get user data.

1) Use lookupUsers majority of the times + Google Maps API to geocode 2) Lookup the profiles each with profiles/get, for remaining.

Somehow those two functionalities should be in sync with each other. More work for the devs :)

[marmat]

I was just going over some of the current code and one thing is a bit unclear to me: why the split to loadPeople and loadLocations? The methods look like they would achieve more or less the same thing to me. Was there any specific reason in doing this?

Couldn't we just fetch the location (either using profiles/get or Maps geocoding) in the cachePeople method? How about something like this:

1. Init
2. Load Maps API
3. Start updates only for loadPeople every x seconds

As for the loadPeople method:

1. gather newParticipants and pass them to cachePeople
2. cachePeople iterates over every participant and tries to get his location
   • by checking in the local sql cache
   • if that fails, by making a profiles/get call
   • if that fails, by using maps geocoding
   • if that fails, we store some kind of error and blacklist this id for further location fetchings
3. the gathered location will be stored in the local cache (self.cache)
4. both profiles/get and maps geocoding will store their result normalized in the sql cache

[mohamedmansour]

• @johnbc who did that part

I like that approach, I am just worrysome regarding profiles/get call, how many calls are we going to make throughout the day. We should somehow incorporate some sort of progress.

Off topic: My Hangouts is doing a lot of processing, I am wondering if we could incorporate WebWorkers in some areas.

[ghost]

Hey, Martin.

The first time I wrote this, this is what I did more or less. I changed it because people and locations are different dimensions of data and really shouldn't be conflated -- addresses could come from people, business, etc. In fact loadPeople doesn't belong in this code at all and should be moved somewhere else if we ever want to do more non-map related stuff with 'people data'

The code has change quite a bit since that initial writing, and it is always worth examining it :) I think though, that we should keep geo-location of addresses distinct from other data.

Thanks.

@mohamedmansour @kaktus621

[marmat]

Thanks @johnbc for the explanation :-)

There's a small problem with the strict separation of people-data and addresses, though: if we want to use the profiles/get geocoding, we do need some kind of people data (at least the user's ID) associated with the address... Maybe we could do the profiles/get geocoding during the loadPeople phase (wherever it's going to be) and only if there's no location data afterwards, we try to fetch the location using the maps API geocoding service.

@mohamedmansour If the caching works, it shouldn't be too much. Probably far less than MapMyCircles, right? I don't think this will cause any problem, but monitoring/limiting the amount of queries is still definitely a good idea.