search

pelias / openstreetmap

Import pipeline for OSM in to Pelias
MIT License
112 stars 72 forks source link

duplicate terms in name affecting scoring #507

Open missinglink opened 4 years ago

missinglink commented 4 years ago

It seems as though having duplicate tokens in a name is causing elasticsearch to score the result higher (this is due to how the TF/IDF scoring works)

While it's impossible to have exact duplicate names (this is taken care of by pelias/model in a 'post' step), it is possible to have two terms which are very similar such as this:

Screenshot 2019-11-15 at 16 18 39

https://www.openstreetmap.org/way/432890745

I'm opening this issue so I don't forget, we can either try to solve this during import or during search.

A query such as /v1/search?text=whole foods market, NY illustrates the issue (although there may be other things at play here)

Screenshot 2019-11-15 at 16 22 28 Screenshot 2019-11-15 at 16 22 16
orangejulius commented 4 years ago

Oh yeah, this is a good one. It's kinda inherent to the scoring mechanisms of Elasticsearch.

Maybe we can solve this by modifying the scoring settings of Elasticsearch, which is easier and more flexible in newer versions. We might be able to split altnames into separate fields in a clever way as well. But I think this sort of case will be pretty hard, though not impossible, to solve.

missinglink commented 4 years ago

Last night I went into a deep-dive on tuning the BM25 algorithm, I suspect that we could reduce k1 to either 0 or near 0 to resolve this issue.

It's a big topic in itself so I'll start a discussion over on pelias/schema

[edit] I think maybe we actually want to set k1=1?

orangejulius commented 4 years ago

Since we've decided this issue is mostly unsolvable without major changes to how we index data in Elasticsearch, let's talk mitigation:

In this particular case, one name is a prefix of the other. Is it worth it to detect that and skip the shorter version?

In the other main case I've noticed with OSM data, the names only differ by whitespace. That one might be trickier, but we could probably still do it.