search

pelias / api

HTTP API for Pelias Geocoder
http://pelias.io
MIT License
219 stars 162 forks source link

Increase `focus.point` weight #1568

Open Joxit opened 3 years ago

Joxit commented 3 years ago

Hi there, this is the PR for #1567

I kept in mind what @orangejulius says in https://github.com/pelias/api/issues/1567#issuecomment-944391612:

Funnily enough, right now we can have two problems that are almost opposites, at the same time:

  • nearby, exact text matches can score below a far away populated/popular place that is a poor text match
  • distant, exact text matches for popular places can score below a nearby place with a good text match

I did several tests to find out what would be the best configuration.

Results with population and popularity boost

I started by decreasing population and popularity boosts as I said in my issue. I tried with three values, 8, 10 and 12 and ran acceptance tests (issues are from autocomplete_acushnet_antiques.json and autocomplete_focus.json).

With boost decreased to 10 for popularity and population I had 2 improvements, 1 regression and 2 changes.

Improvements

/v1/autocomplete?focus.point.lat=41.652889&focus.point.lon=-70.922898&text=antiq

before after
Antique, Philippines (14020.113km) Acushnet River Antiques Showroom, New Bedford, MA, USA (0km) :heavy_check_mark:
Province of Antique, AQ, Philippines (14008.427km) Antiques, Tiverton, RI, USA (23.22km)
Acushnet River Antiques Showroom, New Bedford, MA, USA (0km) :x: Seaview Antiques, Plymouth, MA, USA (40.358km)
Antiques, Tiverton, RI, USA (23.22km) Larchmere Antique District, Cleveland, OH, USA (888.459km)
Seaview Antiques, Plymouth, MA, USA (40.358km) Province of Antique, AQ, Philippines (14008.427km)
Larchmere Antique District, Cleveland, OH, USA (888.459km) Antique, Philippines (14020.113km)
Armory Antique Marketplace, Newport, RI, USA (37.73km) Armory Antique Marketplace, Newport, RI, USA (37.73km)
Antiquarian House, Plymouth, MA, USA (40.241km) Antiquarian House, Plymouth, MA, USA (40.241km)
Cobwebs Antiques & Jewelry, Boston, MA, USA (75.016km) Cobwebs Antiques & Jewelry, Boston, MA, USA (75.016km)
Bridge Antiques, Weymouth Town, MA, USA (65.747km) Bridge Antiques, Weymouth Town, MA, USA (65.747km)

/v1/autocomplete?focus.point.lat=41.652889&focus.point.lon=-70.922898&text=antique

before after
Antique, Philippines (14020.113km) Acushnet River Antiques Showroom, New Bedford, MA, USA (0km) :heavy_check_mark:
Province of Antique, AQ, Philippines (14008.427km) Antiques, Tiverton, RI, USA (23.22km)
Acushnet River Antiques Showroom, New Bedford, MA, USA (0km) :x: Seaview Antiques, Plymouth, MA, USA (40.358km)
Antiques, Tiverton, RI, USA (23.22km) Larchmere Antique District, Cleveland, OH, USA (888.459km)
Seaview Antiques, Plymouth, MA, USA (40.358km) Province of Antique, AQ, Philippines (14008.427km)
Larchmere Antique District, Cleveland, OH, USA (888.459km) Antique, Philippines (14020.113km)
Armory Antique Marketplace, Newport, RI, USA (37.73km) Armory Antique Marketplace, Newport, RI, USA (37.73km)
Cobwebs Antiques & Jewelry, Boston, MA, USA (75.016km) Cobwebs Antiques & Jewelry, Boston, MA, USA (75.016km)
Bridge Antiques, Weymouth Town, MA, USA (65.747km) Bridge Antiques, Weymouth Town, MA, USA (65.747km)
Genuine Antique Lighting, Boston, MA, USA (77.334km) Genuine Antique Lighting, Boston, MA, USA (77.334km)

There is no doubt, this is an improvement

Regressions

/v1/autocomplete?focus.point.lat=40.744131&focus.point.lon=-73.990424&text=San Francisco

before after
San Francisco, CA, USA (4135.156km) :heavy_check_mark: South San Francisco, CA, USA (4132.782km)
South San Francisco, CA, USA (4132.782km) San Francisco, QZ, Philippines (13779.823km)
City of San Francisco, San Francisco, CA, USA (4136.075km) South Beach, San Francisco, CA, USA (4131.75km)
San Francisco County, San Francisco, CA, USA (4136.075km) San Francisco Gotera, MO, El Salvador (3308.524km)
Cuautitlán Izcalli, MEX, Mexico (3340.891km) San Francisco Cahuacúa, OAX, Mexico (3474.854km)
San Francisco de Macorís, DU, Dominican Republic (2414.237km) San Francisco del Valle, OC, Honduras (3270.844km)
Campeche, Mexico (2807.209km) San Fco. Del Norte, CI, Nicaragua (3308.078km)
S. Francisco del Rincon, GUA, Mexico (3424.249km) San Francisco, CA, USA (4135.156km) :x:
San Francisco, AS, Philippines (14187.026km) San Francisco La Unión, QZ, Guatemala (3338.114km)
San Francisco, QZ, Philippines (13779.823km) San Francisco Zapotitlán, SU, Guatemala (3371.286km)

This one is very weird, South San Francisco took the place of San Francisco...

Change

/v1/autocomplete?focus.point.lat=41.05343&focus.point.lon=-73.53873&text=New York

before after
New York County, NY, USA (47.631km) New Castle, NY, USA (24.692km)
New York City, Manhattan, New York City, NY, USA (54.514km) New Rochelle, NY, USA (25.49km)
City of New York, Brooklyn, NY, USA (54.878km) New Salem, Port Washington, NY, USA (28.565km)
New Rochelle, NY, USA (25.49km) New City, NY, USA (39.498km)
New York, NY, USA (54.989km) :o: New Cassel, NY, USA (32.589km)
City of New Rochelle, NY, USA (24.928km) New Square, NY, USA (42.21km)
New Castle, NY, USA (24.692km) Town of New Castle, NY, USA (24.424km)
New City, NY, USA (39.498km) City of New Rochelle, NY, USA (24.928km)
East New York, Brooklyn, New York, NY, USA (51.821km) New York County, NY, USA (47.631km)
Town of New Castle, NY, USA (24.424km) North New Hyde Park, NY, USA (36.402km)

Here the new position of New York City is 42 ! This may be considered as a regression, that's why I will not chose this configuration.

/v1/autocomplete?focus.point.lat=41.05343&focus.point.lon=-73.53873&text=New York, NY

before after
New York County, NY, USA (47.631km) New York County, NY, USA (47.631km)
New York, NY, USA (54.989km) :o: East New York, Brooklyn, New York, NY, USA (51.821km)
New York City, Manhattan, New York City, NY, USA (54.514km) New York, NY, USA (54.989km) :x:
City of New York, Brooklyn, NY, USA (54.878km) New York City, Manhattan, New York City, NY, USA (54.514km)
East New York, Brooklyn, New York, NY, USA (51.821km) City of New York, Brooklyn, NY, USA (54.878km)
Deutsche Schule New York, White Plains, NY, USA (16.891km) Deutsche Schule New York, White Plains, NY, USA (16.891km)
Keio Academy of New York, Purchase, NY, USA (15.338km) Keio Academy of New York, Purchase, NY, USA (15.338km)
Port Chester, New York 10573, Port Chester, NY, USA (12.209km) Port Chester, New York 10573, Port Chester, NY, USA (12.209km)
NYLO New York City, New York, NY, USA (47.886km) NYLO New York City, New York, NY, USA (47.886km)
New York Mills, NY, USA (270.079km) New York University, New York, NY, USA (52.843km)

Here New York is loosing one place, but still good enough. At this point I decided to change my mind and increase focus point instead of decreasing population/popularity.

Results with focus point increase weight

So now I want to have the same scale between focus point boosts and population/popularity, so I set the weight of focust point to 20. With this configuration I had 2 improvements, 0 regression :tada: and 1 change.

Improvements

/v1/autocomplete?focus.point.lat=41.652889&focus.point.lon=-70.922898&text=antiq

before after
Antique, Philippines (14020.113km) Acushnet River Antiques Showroom, New Bedford, MA, USA (0km) :heavy_check_mark:
Province of Antique, AQ, Philippines (14008.427km) Antique, Philippines (14020.113km)
Acushnet River Antiques Showroom, New Bedford, MA, USA (0km) :x: Province of Antique, AQ, Philippines (14008.427km)
Antiques, Tiverton, RI, USA (23.22km) Antiques, Tiverton, RI, USA (23.22km)
Seaview Antiques, Plymouth, MA, USA (40.358km) Seaview Antiques, Plymouth, MA, USA (40.358km)
Larchmere Antique District, Cleveland, OH, USA (888.459km) Armory Antique Marketplace, Newport, RI, USA (37.73km)
Armory Antique Marketplace, Newport, RI, USA (37.73km) Antiquarian House, Plymouth, MA, USA (40.241km)
Antiquarian House, Plymouth, MA, USA (40.241km) Larchmere Antique District, Cleveland, OH, USA (888.459km)
Cobwebs Antiques & Jewelry, Boston, MA, USA (75.016km) Bridge Antiques, Weymouth Town, MA, USA (65.747km)
Bridge Antiques, Weymouth Town, MA, USA (65.747km) Cobwebs Antiques & Jewelry, Boston, MA, USA (75.016km)

/v1/autocomplete?focus.point.lat=41.652889&focus.point.lon=-70.922898&text=antique

before after
Antique, Philippines (14020.113km) Acushnet River Antiques Showroom, New Bedford, MA, USA (0km) :heavy_check_mark:
Province of Antique, AQ, Philippines (14008.427km) Antique, Philippines (14020.113km)
Acushnet River Antiques Showroom, New Bedford, MA, USA (0km) :x: Province of Antique, AQ, Philippines (14008.427km)
Antiques, Tiverton, RI, USA (23.22km) Antiques, Tiverton, RI, USA (23.22km)
Seaview Antiques, Plymouth, MA, USA (40.358km) Seaview Antiques, Plymouth, MA, USA (40.358km)
Larchmere Antique District, Cleveland, OH, USA (888.459km) Armory Antique Marketplace, Newport, RI, USA (37.73km)
Armory Antique Marketplace, Newport, RI, USA (37.73km) Larchmere Antique District, Cleveland, OH, USA (888.459km)
Cobwebs Antiques & Jewelry, Boston, MA, USA (75.016km) Bridge Antiques, Weymouth Town, MA, USA (65.747km)
Bridge Antiques, Weymouth Town, MA, USA (65.747km) Cobwebs Antiques & Jewelry, Boston, MA, USA (75.016km)
Genuine Antique Lighting, Boston, MA, USA (77.334km) Genuine Antique Lighting, Boston, MA, USA (77.334km)

There is no doubt, this is still an improvement

Regressions

No regressions :heavy_check_mark:

Change

/v1/autocomplete?focus.point.lat=41.05343&focus.point.lon=-73.53873&text=New York

before after
New York County, NY, USA (47.631km) New York County, NY, USA (47.631km)
New York City, Manhattan, New York City, NY, USA (54.514km) New Rochelle, NY, USA (25.49km)
City of New York, Brooklyn, NY, USA (54.878km) New York City, Manhattan, New York City, NY, USA (54.514km)
New Rochelle, NY, USA (25.49km) City of New York, Brooklyn, NY, USA (54.878km)
New York, NY, USA (54.989km) :o: City of New Rochelle, NY, USA (24.928km)
City of New Rochelle, NY, USA (24.928km) New York, NY, USA (54.989km) :x:
New Castle, NY, USA (24.692km) New Castle, NY, USA (24.692km)
New City, NY, USA (39.498km) New City, NY, USA (39.498km)
East New York, Brooklyn, New York, NY, USA (51.821km) East New York, Brooklyn, New York, NY, USA (51.821km)
Town of New Castle, NY, USA (24.424km) Town of New Castle, NY, USA (24.424km)

Here New York is loosing one place, but still good enough.

Resume

So instead of decreasing population and popularity boosts, I chose to increase focus point weight. The overall result seems good and close to current responses.

Results are also better in my initial test with Vars (the lang is important here).

/v1/autocomplete?lang=fr&focus.point.lat=48.03661925338169&focus.point.lon=6.580299229512&text=vars

before after
Varsovie, MZ, Pologne (1127.93km) Varsovie, MZ, Pologne (1127.93km)
Finlande propre, Turku, Finlande propre, Finlande (1711.878km) Finlande propre, Turku, Finlande propre, Finlande (1711.878km)
Varsovie, Johns Creek, GA, USA (7334.865km) Ham-sous-Varsberg, France (127.305km)
Ham-sous-Varsberg, France (127.305km) Varsovie, Johns Creek, GA, USA (7334.865km)
Varsity Center, Lehigh Acres, FL, USA (7732km) Varsity Center, Lehigh Acres, FL, USA (7732km)
Varsity Lakes, QLD, Australie (16424.897km) Varsity Lakes, QLD, Australie (16424.897km)
Varsovie, Throop, PA, USA (6225.986km) Varsberg, France (126.335km)
Varchets, MT, Bulgarie (1404.336km) Vars, France (96.218km) :o:
Obshtina Varshets, MT, Bulgarie (1406.591km) Varsovie, Throop, PA, USA (6225.986km)
Varsberg, France (126.335km) Varchets, MT, Bulgarie (1404.336km)

:memo: TODO: write acceptance test for Vars

related #1567

orangejulius commented 3 years ago

Wow, quite comprehensive testing. We'll run this through our own tests as well and report back. Thanks for taking the time to try out lots of options.

The South San Francisco situation is something I've looked at in the past, and unfortunately it's a result of the issue with scoring multiple alt names. San Francisco (as expected) has more alt-names than South San Francisco.

So hopefully we can improve things even more someday when we fix scoring issues.

orangejulius commented 3 years ago

Hey this looks pretty good!

We tested it out and there don't really look to be any unexpected effects. Queries with a focus point do now prefer things closer to the focus point by a tiny bit. The acceptance tests provide the best examples, especially with the Union Square tests that have always been challenging to make pass:

image

The flip side is that some city/locality/high population results come up lower. Both the following examples are only slight regressions but worth looking more closely at.

image

Also NYC and SF are hard queries for various reasons (aliases, duplicate names, etc), so while they're obviously very prominent examples, if they are the only things affected then that's not so bad.