search

NatLibFi / Annif

Annif is a multi-algorithm automated subject indexing tool for libraries, archives and museums.
https://annif.org
Other
188 stars 41 forks source link

adding stable gensim for supporting py3.12 #789

Closed tavallaie closed 3 weeks ago

tavallaie commented 1 month ago

adding last stable commit of gensim for support python 3.12 solving: #779 I wrote similar problem here. https://blog.techbend.io/overcoming-the-hazm-compatibility-challenge-with-python-312

sonarcloud[bot] commented 1 month ago

Quality Gate Passed Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
No data about Duplication

See analysis details on SonarCloud

osma commented 1 month ago

Thank you for the PR @tavallaie !

Can you elaborate on what benefit this PR brings? In my understanding, Annif already mostly works in Python 3.12, though some dependencies will need to be compiled instead of installed from wheels. We use Python 3.12 in the new Annif-tutorial image and basically everything is working, even TensorFlow (though that needed some manual intervention).

According to the CI/CD pipeline, the newer Gensim cannot be installed on Python 3.9. So we would have to drop support for 3.9.

How is this "stable gensim"? It seems to be referencing a specific commit, not a proper release.

tavallaie commented 1 month ago

The gensim version 4.3.2 uses the deprecated triu module from scipy, making it incompatible with Python 3.12. In the version I added, gensim uses triu from numpy, but this update has not yet been added to PyPI.

osma commented 1 month ago

@tavallaie Python 3.12 with gensim 4.3.2 seems to work in practice at least for the purposes of Annif. See e.g. this CI/CD run for the Annif 1.1.0 release, which succesfully installed dependencies (including scipy and gensim) and ran the test suite.

I'm sure you are correct in that there are compatibility issues between the software packages and versions that you mentioned, but I see little harm in practice from the perspective of Annif users. I think it would be best to wait for a formal gensim release on PyPI and then upgrade to it. We probably need to drop Python 3.9 support in the same go if the new gensim release doesn't support it anymore.

tavallaie commented 1 month ago

I understand I can go with running more test so assure not harming py3.9 ususersBut if you think it is not necessary I can close the pull request.

osma commented 1 month ago

I'm curious if you had an actual problem running Annif on Python 3.12? What made you create this PR?

tavallaie commented 1 month ago

I encountered an issue with Gensim in another project and found a temporary fix. Noticing similar issues (genism version and python 3.12) in your project, I applied the same fix here. I do this to get involved in other projects and learn new things. Thanks!