Upgrade Simplemma & limit its memory usage

NatLibFi / Annif

Annif is a multi-algorithm automated subject indexing tool for libraries, archives and museums.

https://annif.org

Other

197 stars 41 forks source link

Upgrade Simplemma & limit its memory usage #724

Closed osma closed 2 weeks ago

osma commented 1 year ago

This PR upgrades Simplemma to the latest version 1.1.1 and adds some wrapper code to make it possible to use Simplemma language detection with a limit on the number of languages simultaneously loaded into memory. The number of languages is currently hardcoded at 5.

This will enable adding language detection functionality to the REST API in PR #659 , which has not been merged yet due to concerns about memory usage.

codecov[bot] commented 1 year ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 99.64%. Comparing base (337ee70) to head (61c9409). Report is 8 commits behind head on main.

Additional details and impacted files

```diff @@ Coverage Diff @@ ## main #724 +/- ## ======================================= Coverage 99.64% 99.64% ======================================= Files 91 93 +2 Lines 6831 6854 +23 ======================================= + Hits 6807 6830 +23 Misses 24 24 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.