search

sailfish-keyboard / presage

Fork of Presage (http://presage.sourceforge.net/)
GNU General Public License v2.0
6 stars 10 forks source link

performance optimization #1

Closed rinigus closed 6 years ago

rinigus commented 6 years ago

Presage has to do a lot of work and can become relatively slow. Most of the time is spent querying the database, so optimization of data access should bring major improvements.

While several LM exist optimized for getting probability full n-word sequence, Presage needs to get n-gram counts from prefix search, which makes it tricky.

The current perf flame diagram attached p

rinigus commented 6 years ago

Caching unigram count sum removed the corresponding call from the flamegraph and reduced one test benchmark from 95 to 74 seconds

Commit https://github.com/rinigus/presage/commit/090d5e77d28e06a95a1def47692fa152ae458818

rinigus commented 6 years ago

Using Marisa and counts file, I managed to get test run to 25s. About 15s of these are spent by SQLite learning predictor with large time probably used for flushing SQLite journal.

rinigus commented 6 years ago

Closing this for now. Let's reopen if learning predictor will become an issue or some other problem will resurface