Improve search algorithm

rodekruis / helpful-information

A web-app for People Affected to lookup useful organizations and information.

Apache License 2.0

2 stars 2 forks source link

Improve search algorithm #435

Open elwinschmitz opened 2 years ago

elwinschmitz commented 2 years ago

Currently the search-feature uses a (case-INsensitive) RegEx-matching algorithm like: If <search input> can be found as-is ANYWHERE in the Answer- OR Question-fields of a Q&A-set. So for example:

"abc xyz" would match on a Question/Answer of: "ABC xyz" or "123aBc XyZ ... 78910"
"abc xyz" would NOT match on a Question/Answer of: "abcdef" or "abcdefghijklmnopqrstuvw xyz"

Maybe more intuitive would be to match ANY keyword (by splitting the input on spaces) found anywhere in the Question/Answer. So for example:

"abc xyz" would match on a Question/Answer of: "test-xyz test test abc test"

See #286 See #433

elwinschmitz commented 2 years ago

Also:

Add a "relevance"-score to each result, so we can sort the results on it.

For example a level of relevance could be: ("number of matches in the answer" * 10) + ("number of matches in the question" * 1) =

Other things to consider:

names/descriptions of (Sub-)Categories
Sub-Question's influence on parent Question's relevance?
More recent result more relevant?
Show results from the same (Sub-)Category grouped together?

jmargutt commented 1 year ago

There are fancy semantic search models that can be deployed without re-training them, e.g. https://www.sbert.net/examples/applications/semantic-search/README.html Much more robust than RegEx. But need to be hosted, cannot run on client-side