NatLibFi / Annif

Annif is a multi-algorithm automated subject indexing tool for libraries, archives and museums.
https://annif.org
Other
197 stars 41 forks source link

Python 3.11 support #727

Closed juhoinkinen closed 1 year ago

juhoinkinen commented 1 year ago

Allow installing Annif on Python 3.11, and run unit tests on it in the CI/CD pipeline.

Omikuji dependency does not work Python 3.11, see https://github.com/NatLibFi/Annif/issues/703#issuecomment-1614513903. Other dependencies seem to work (i.e. the unit tests pass).

tomli is now a (direct) dependency only on Python <3.11, on Python 3.11 tomllib of standard library is used instead (PEP 690). However, also on Python 3.11 tomli is still installed as it is a dependency of black, coverage, pytest and schemathesis.

Closes #703, includes merge of the branch of #726.

codecov[bot] commented 1 year ago

Codecov Report

Patch coverage: 100.00% and no project coverage change.

Comparison is base (320af2b) 99.67% compared to head (f928844) 99.67%. Report is 18 commits behind head on main.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #727 +/- ## ======================================= Coverage 99.67% 99.67% ======================================= Files 89 89 Lines 6380 6400 +20 ======================================= + Hits 6359 6379 +20 Misses 21 21 ``` | [Files Changed](https://app.codecov.io/gh/NatLibFi/Annif/pull/727?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=NatLibFi) | Coverage Δ | | |---|---|---| | [annif/backend/dummy.py](https://app.codecov.io/gh/NatLibFi/Annif/pull/727?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=NatLibFi#diff-YW5uaWYvYmFja2VuZC9kdW1teS5weQ==) | `100.00% <ø> (ø)` | | | [annif/backend/ensemble.py](https://app.codecov.io/gh/NatLibFi/Annif/pull/727?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=NatLibFi#diff-YW5uaWYvYmFja2VuZC9lbnNlbWJsZS5weQ==) | `100.00% <ø> (ø)` | | | [annif/backend/nn\_ensemble.py](https://app.codecov.io/gh/NatLibFi/Annif/pull/727?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=NatLibFi#diff-YW5uaWYvYmFja2VuZC9ubl9lbnNlbWJsZS5weQ==) | `100.00% <ø> (ø)` | | | [annif/backend/omikuji.py](https://app.codecov.io/gh/NatLibFi/Annif/pull/727?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=NatLibFi#diff-YW5uaWYvYmFja2VuZC9vbWlrdWppLnB5) | `97.46% <ø> (-0.13%)` | :arrow_down: | | [annif/backend/svc.py](https://app.codecov.io/gh/NatLibFi/Annif/pull/727?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=NatLibFi#diff-YW5uaWYvYmFja2VuZC9zdmMucHk=) | `100.00% <ø> (ø)` | | | [annif/backend/yake.py](https://app.codecov.io/gh/NatLibFi/Annif/pull/727?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=NatLibFi#diff-YW5uaWYvYmFja2VuZC95YWtlLnB5) | `98.18% <ø> (-0.07%)` | :arrow_down: | | [tests/test\_backend\_stwfsa.py](https://app.codecov.io/gh/NatLibFi/Annif/pull/727?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=NatLibFi#diff-dGVzdHMvdGVzdF9iYWNrZW5kX3N0d2ZzYS5weQ==) | `100.00% <ø> (ø)` | | | [annif/\_\_init\_\_.py](https://app.codecov.io/gh/NatLibFi/Annif/pull/727?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=NatLibFi#diff-YW5uaWYvX19pbml0X18ucHk=) | `100.00% <100.00%> (ø)` | | | [annif/backend/backend.py](https://app.codecov.io/gh/NatLibFi/Annif/pull/727?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=NatLibFi#diff-YW5uaWYvYmFja2VuZC9iYWNrZW5kLnB5) | `100.00% <100.00%> (ø)` | | | [annif/backend/mllm.py](https://app.codecov.io/gh/NatLibFi/Annif/pull/727?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=NatLibFi#diff-YW5uaWYvYmFja2VuZC9tbGxtLnB5) | `100.00% <100.00%> (ø)` | | | ... and [5 more](https://app.codecov.io/gh/NatLibFi/Annif/pull/727?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=NatLibFi) | |

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

osma commented 1 year ago

If the unit tests pass on Python 3.11 (except Omikuji, but that is not even installed for 3.11), then I don't see why we should mark the CI tests as experimental. It would be a different situation if some core tests were failing on 3.11 and we couldn't fix them right now.

sonarcloud[bot] commented 1 year ago

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

No Coverage information No Coverage information
0.0% 0.0% Duplication

osma commented 1 year ago

The Python 3.11 parts of this PR look good to me, but since this includes all the commits from PR #726, we need to make sure those are good as well before merging this. I noticed a potential problem with TF/Keras, see https://github.com/NatLibFi/Annif/pull/726#issuecomment-1680047565