Open FuriusBaco opened 4 months ago
what kind of dictionary you suggesting? Could you provide reproducible case - config, data to populate with the forum index and data to populate with the standard dictionary of Russian and query to both indexes with the result sets there it is clear why idx_right
should be used but not the idx_forum
.
Do you mean that you create and full another index with right content when use this form of CALL SUGGEST(word, 'idx_right,idx_forum')
and if the 1st search in the idx_right
term levinstein distance is closer to N (some new param) then not to search in the idx_forum
?
As if search into both idx_right
and idx_forum
what is the condition why to use idx_right
but not idx_forum
if levinstein distance is closer for term from idx_forum
?
I can't fill in the indexes. Any forum will do.
A dictionary is a reliable source, so it should be used first. By the way, such a "CALL SUGGEST(words, 'idx_right,idx_forum')" format would also be very good! For example, I have a database with 1.7 million words. There is a similar one on Github (https://github.com/danakt/russian-words ). Manticore has its own dictionary (ru.pak, for example).
Now I'm making a Speller using just such a system - first a dictionary, then a forum. After some time, I will be able to tell you about the results.
You can make two modes - the priority of dictionaries as they are enumerated and as the Levenshtein distance increases, or a SORT BY operator or something similar.
Proposal:
Please consider checking words through SUGGEST first using the standard dictionary of Russian or any other language and first suggesting substitutions from there, and only then looking for them in the index. Optional, of course.
Yes, for academic texts where there are no errors, the current implementation is quite suitable. But for the forum, it doesn't always work well. For example, the word “pamagite” is replaced by “pomagite”, “pamagiti”, “pamagitya”, “poamagite”, etc.
This will also be good for the words in the query that are not on the site, but there may still be useful information in the query.
Checklist:
To be completed by the assignee. Check off tasks that have been completed or are not applicable.