Spike: location data process [20,13,1.54]

[hellafitz]

From: #2453

This ticket is to tackle the root cause of our location problems, by:

• Reviewing how we collect location data into the app in the first place, including how we deal with disputed names/locations. Toward the goal of feeding in consistent location data and reducing unneeded variation.

Deliverable:

• document describing the above, as well as recommended next step(s) toward the identified goal

context:

• preliminary thought is be more like TF, providing a drop down for user plus options to add locations not on the list (although we're not sure what we'd do if someone provided one)
• theory: move the normalization upfront, remove the normalization in the middle, get better dashboard -accuracy at the end as a result.
• for KPI and reporting reasons, location classification should be the same across both TF and TG

[hellafitz]

Thought exercise for Program / CET: If we told you that we'd changed the platform's location system overnight, what would you go in and check that morning?

ex: as a student, can I find a mentor near me ('find nearby mentors' button)? As an RA, am I still seeing the students in my region? As an Admin, can I...? As a Judge who had mentored, do I correctly not see submissions from my area?

• Do we as an organization have a list of countries or geographies that we support regardless of whether the location API/database does?
• There used to be something that queried IP address to propose a user's location. Is that still needed? We'd like to throw it out.

[hellafitz]

re: List of countries, "the best bet is to pull the full country list of where we have registrants from for both programs. I think with the current version of the platform the only disputed country is Kosovo. Anything Google and Bing doesn't recognize as official gives us trouble." "(...) We also recognize Hong Kong."

[stenington]

Reviewing how we collect location data into the app in the first place, including how we deal with disputed names/locations. Toward the goal of feeding in consistent location data and reducing unneeded variation.

Here's my take on that: https://docs.google.com/document/d/1-PT21KBr5eanJaQ0jC7Vt-E3yRY51-3IwTp5gMchIuo/edit#

My recommended next step would be to get a second opinion on it. Someone else should do their own assessment and we can compare how much we agree or disagree on it. I think that would lead to productive conversations.

[hellafitz]

Updated Deliverable:

• Add to the existing document

Suggested steps

• Read the doc
• Ask a lot of Qs
• To some extent, follow the rabbit hole of existing tickets
• Look for external examples of people solving this (but watch out for overly simplistic descriptions of "this")
• write up resulting recs and remaining questions
• Overall, spend approx 1/3 time on understanding how we're currently handling location, approx 2/3 on how we could/should

[shaun-technovation]

If I'm understanding things correctly, it seems that when we try to search via the admin we're comparing a hard-coded list of location/string data with user-entered, Google/Bing-assisted location/string data, and these strings/locations don't always match up.

Another option to do this, but disclaimer first: I don't have any real world experience with location data. When I was looking at solutions it seems that the ideal way to search for locations is to use coordinates (lat/long) - addresses make sense to us, but coordinates make sense to computers; I read something like that somewhere.

We're already storing lat/long coordinates in the database (via the Geocoder library we're using), but we're not searching against them in the admin.

I looked at the doc Mike put together, and I like the idea of having a hardcoded list of locations, that both the users and admins are using/searching against, that way it's consistent.

Ultimately I think either option would give us better results than what we have now, and would eliminate our location patches/band-aids. I also think a hybrid approach would be nice too, allowing admins to find users based off hardcoded location data or by lat/long via geocoding.

---

Some random thoughts on the standardizing locations approach:

I think this would be a nice user experience using auto-suggest/completion, basically taking what we do in the admin to the user-facing side, but with more robust data to select from.

I like this approach because it's simple and it would be the same experience on both the user side and admin side.

One thing we would have to address is existing users and how to update their location. For users who still use the platform we could prompt them to update their location, but for users who aren't using the platform anymore it might be trickier, but maybe not a big deal?. It also might be tricky to find users by location who don't have a standardized location in the admin.

---

Some very random things/thoughts on the lat/long option if we want to explore this option more:

We could create a spike to hookup the backend to search via lat/long, and see how it works.

The front-end experience wouldn't *have to change.

We could remove at least three location related gems we're using from the admin side.

Lat/long gives us one more param to search off of. This would help with distance related queries.

We should be caching results (from the user and admin side).

We could re-geocode and reverse geocode for better more standardized locations.

There are other options for geocoding, OpenStreetMap has a free geocoding API, called Nominatim: https://nominatim.openstreetmap.org/ui/search.html

We could try out the Nominatim GUI to test for locations we've had trouble with.

Here is the usage policy (there are some things to be aware of, like a 1 request per second policy): https://operations.osmfoundation.org/policies/nominatim/

Another geocoding option (linked from Nominatim) that offers free/paid plans: https://opencagedata.com/

Looks like the geocoding library we're using (Geocoder) supports both Nominatim and OpenCage, and a bunch of others: https://rdoc.info/github/alexreisner/geocoder/master/Geocoder/Lookup

Random articles I ran into about OpenStreetMap: https://geoawesomeness.com/why-would-you-use-openstreetmap-if-there-is-google-maps/ https://geoawesomeness.com/openstreetmap-mapping-standard/

Regarding OpenStreetMap:

Prominent users include Facebook, Wikimedia Maps, Apple, Microsoft, Amazon Logistics, Uber, Craigslist, Snapchat, OsmAnd, Geocaching, MapQuest Open, JMP statistical software, and Foursquare. https://en.wikipedia.org/wiki/OpenStreetMap

[stenington]

Cool, lots to unpack here but I feel like we're generally on the same page.

I'm not sure what the next step is in moving this ticket forward, but just commenting to say I've read it and I think there are discussions we can continue based on this. 👍

[hellafitz]

Next Steps:

• discussion of macro approach (now or later) [Team Time?]
• spike 1: find the static datasource for options to select from (concern: may degrade the UX)
• spike 2: services we could put behind geocoder (Q: is the service consistent in how it looks at the world?)
• spike 3, if needed: lat/long geolocation stuff (libraries, how to work through admin, etc); Q: how would we group users by US state and country if using lat/long?

spikes 2 & 3 need some refinement