foursquare / fsqio

A monorepo that holds all of Foursquare's opensource projects
Apache License 2.0
252 stars 54 forks source link

Twofishes does not handle misspelled city names at all #35

Open steveha-ziprecruiter opened 7 years ago

steveha-ziprecruiter commented 7 years ago

Do a Twofishes lookup on: "Baton Rouge, LA" Result: correctly resolves

Do a Twofishes lookup on: "Baton Roug, LA" Result: resolves to "Los Angeles, CA, United States"

Any misspelling in the city name causes a failure. We first saw this when someone transposed the 'u' and the 'g' to get "Baton Rogue, LA" which won't even trip a spell checker since "Rogue" is a word.

Similarly, "Terra Haute, IN" resolves to the country of India. "Terre Haute, IN" correctly resolves.

The basic problem is that any misspelling in the city name causes the city name to be completely ignored. If you are lucky it will at least get the state right (example: "Minneapolid, MN" resolves to Minnesota, United States). But Los Angeles is over 1800 miles away from Louisiana so it's a very poor result.

I speculate that a misspelled city name is ignored because Twofishes is designed to ignore extra stuff as in "Empire State Building, 350 5th Ave, New York, NY" which resolves to New York, NY, United States.