openvanilla / McBopomofo

小麥注音輸入法
http://mcbopomofo.openvanilla.org/
MIT License
626 stars 76 forks source link

Refactor lookup-by-value, fix a bug, and add unit tests #374

Closed lukhnos closed 1 year ago

lukhnos commented 1 year ago

@shuang886 I refactored the code introduced in #372 and I fixed an issue. While the parseless db uses prefix match for looking up keys by value, the actual parseless LM shouldn't use prefixes.

To reproduce the issue, edit manually data.txt to make the term 鑰匙圈 have a higher score than 鑰匙 (say -1.0), then try to add the phrase 鑰匙系統 via the system service. It turns out that without the fix, the reading ㄧㄠˋ-ㄕˊ-ㄑㄩㄢ-ㄒㄧˋ-ㄊㄨㄥˇ is added. It's just that 鑰匙圈 normally has a score way lower than 鑰匙, and so this bug doesn't manifest itself easily.