Fix missing limit parameter in STWFSA backend

juhoinkinen commented 1 year ago

I noticed that if a STWFSA project configuration did not set the limit paramter, the operations based on suggest functionality (CLI commands suggest, eval, index...) were crashing:

echo kissa | annif suggest stwfsa-fi  
Traceback (most recent call last):
  File "/home/local/jmminkin/.cache/pypoetry/virtualenvs/annif-ul-EXdhi-py3.10/bin/annif", line 6, in <module>
    sys.exit(cli())
  File "/home/jmminkin/.cache/pypoetry/virtualenvs/annif-ul-EXdhi-py3.10/lib/python3.10/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/home/jmminkin/.cache/pypoetry/virtualenvs/annif-ul-EXdhi-py3.10/lib/python3.10/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/home/jmminkin/.cache/pypoetry/virtualenvs/annif-ul-EXdhi-py3.10/lib/python3.10/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/jmminkin/.cache/pypoetry/virtualenvs/annif-ul-EXdhi-py3.10/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/jmminkin/.cache/pypoetry/virtualenvs/annif-ul-EXdhi-py3.10/lib/python3.10/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/home/jmminkin/.cache/pypoetry/virtualenvs/annif-ul-EXdhi-py3.10/lib/python3.10/site-packages/click/decorators.py", line 26, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/home/jmminkin/.cache/pypoetry/virtualenvs/annif-ul-EXdhi-py3.10/lib/python3.10/site-packages/flask/cli.py", line 357, in decorator
    return __ctx.invoke(f, *args, **kwargs)
  File "/home/jmminkin/.cache/pypoetry/virtualenvs/annif-ul-EXdhi-py3.10/lib/python3.10/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/home/local/jmminkin/git/Annif/annif/cli.py", line 284, in run_suggest
    suggestions = project.suggest([text], backend_params).filter(limit, threshold)[
  File "/home/local/jmminkin/git/Annif/annif/project.py", line 253, in suggest
    return self._suggest_with_backend(texts, backend_params)
  File "/home/local/jmminkin/git/Annif/annif/project.py", line 143, in _suggest_with_backend
    return self.backend.suggest(texts, beparams)
  File "/home/local/jmminkin/git/Annif/annif/backend/backend.py", line 130, in suggest
    return self._suggest_batch(texts, params=beparams)
  File "/home/local/jmminkin/git/Annif/annif/backend/backend.py", line 118, in _suggest_batch
    limit=int(params.get("limit")),
TypeError: int() argument must be a string, a bytes-like object or a real number, not 'NoneType'

The default limit parameter was not obtainable from AnnifBackend.DEFAULT_PARAMETERS, because StwfsaBackend class defined its own DEFAULT_PARAMETERS field (without the limit parameter), but not a default_params() method. Other backends having their own DEFAULT_PARAMETERS field had that method to include the AnnifBackend.DEFAULT_PARAMETERS in the parameters.

I changed the AnnifBackend.default_params() method to include both DEFAULT_PARAMETERS from AnnifBackend (base) class and from the instances of the concrete backend (derived) classes, like most backends were already doing. This way there is no need to duplicate the method in so many backend classes.

In the fasttext backend the default_params() method includes also DEFAULT_PARAMETERS from mixins.ChunkingBackend, so it is needed to be retained to override the new AnnifBackend.default_params() method.

sonarcloud[bot] commented 1 year ago

Kudos, SonarCloud Quality Gate passed!

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

No Coverage information
0.0% Duplication

codecov[bot] commented 1 year ago

Codecov Report

Patch coverage: 100.00% and project coverage change: -0.01 :warning:

Comparison is base (320af2b) 99.67% compared to head (bf5f844) 99.66%.

Additional details and impacted files

```diff @@ Coverage Diff @@ ## main #718 +/- ## ========================================== - Coverage 99.67% 99.66% -0.01% ========================================== Files 89 89 Lines 6380 6356 -24 ========================================== - Hits 6359 6335 -24 Misses 21 21 ``` | [Impacted Files](https://app.codecov.io/gh/NatLibFi/Annif/pull/718?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=NatLibFi) | Coverage Δ | | |---|---|---| | [annif/backend/dummy.py](https://app.codecov.io/gh/NatLibFi/Annif/pull/718?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=NatLibFi#diff-YW5uaWYvYmFja2VuZC9kdW1teS5weQ==) | `100.00% <ø> (ø)` | | | [annif/backend/nn\_ensemble.py](https://app.codecov.io/gh/NatLibFi/Annif/pull/718?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=NatLibFi#diff-YW5uaWYvYmFja2VuZC9ubl9lbnNlbWJsZS5weQ==) | `100.00% <ø> (ø)` | | | [annif/backend/omikuji.py](https://app.codecov.io/gh/NatLibFi/Annif/pull/718?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=NatLibFi#diff-YW5uaWYvYmFja2VuZC9vbWlrdWppLnB5) | `97.46% <ø> (-0.13%)` | :arrow_down: | | [annif/backend/svc.py](https://app.codecov.io/gh/NatLibFi/Annif/pull/718?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=NatLibFi#diff-YW5uaWYvYmFja2VuZC9zdmMucHk=) | `100.00% <ø> (ø)` | | | [annif/backend/yake.py](https://app.codecov.io/gh/NatLibFi/Annif/pull/718?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=NatLibFi#diff-YW5uaWYvYmFja2VuZC95YWtlLnB5) | `98.18% <ø> (-0.07%)` | :arrow_down: | | [tests/test\_backend\_stwfsa.py](https://app.codecov.io/gh/NatLibFi/Annif/pull/718?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=NatLibFi#diff-dGVzdHMvdGVzdF9iYWNrZW5kX3N0d2ZzYS5weQ==) | `100.00% <ø> (ø)` | | | [annif/backend/backend.py](https://app.codecov.io/gh/NatLibFi/Annif/pull/718?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=NatLibFi#diff-YW5uaWYvYmFja2VuZC9iYWNrZW5kLnB5) | `100.00% <100.00%> (ø)` | | | [annif/backend/mllm.py](https://app.codecov.io/gh/NatLibFi/Annif/pull/718?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=NatLibFi#diff-YW5uaWYvYmFja2VuZC9tbGxtLnB5) | `100.00% <100.00%> (ø)` | | | [annif/backend/pav.py](https://app.codecov.io/gh/NatLibFi/Annif/pull/718?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=NatLibFi#diff-YW5uaWYvYmFja2VuZC9wYXYucHk=) | `100.00% <100.00%> (ø)` | |

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.

NatLibFi / Annif

Fix missing limit parameter in STWFSA backend #718

Codecov Report