anki-morphs crashing on numpy error

[zeroeightysix]

Probably a downstream issue, but I'm having a hard time figuring out why this is happening specifically.

Trying to open the anki-morphs settings or doing a recalc will yield the following error:

Anki 24.06.3 (d678e393) (src) (ao)
Python 3.12.6 Qt 6.7.2 PyQt 6.7.1
Platform: Linux-6.10.10-arch1-1-x86_64-with-glibc2.40

Traceback (most recent call last):
  File "/usr/lib/python3.12/site-packages/aqt/__init__.py", line 146, in open
    instance = creator(*args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ridan/.local/share/Anki2/addons21/472573498/settings/settings_dialog.py", line 54, in __init__
    self._note_filters_tab = NoteFiltersTab(
                             ^^^^^^^^^^^^^^^
  File "/home/ridan/.local/share/Anki2/addons21/472573498/settings/settings_note_filters_tab.py", line 72, in __init__
    self._morphemizers = get_all_morphemizers()
                         ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ridan/.local/share/Anki2/addons21/472573498/morphemizers/morphemizer.py", line 69, in get_all_morphemizers
    for spacy_model in spacy_wrapper.get_installed_models():
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ridan/.local/share/Anki2/addons21/472573498/morphemizers/spacy_wrapper.py", line 354, in get_installed_models
    import spacy.util  # pylint:disable=import-outside-toplevel
    ^^^^^^^^^^^^^^^^^
  File "/home/ridan/.local/share/Anki2/addons21/spacyenv/lib/python3.12/site-packages/spacy/__init__.py", line 6, in <module>
    from .errors import setup_default_warnings
  File "/home/ridan/.local/share/Anki2/addons21/spacyenv/lib/python3.12/site-packages/spacy/errors.py", line 3, in <module>
    from .compat import Literal
  File "/home/ridan/.local/share/Anki2/addons21/spacyenv/lib/python3.12/site-packages/spacy/compat.py", line 39, in <module>
    from thinc.api import Optimizer  # noqa: F401
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ridan/.local/share/Anki2/addons21/spacyenv/lib/python3.12/site-packages/thinc/api.py", line 1, in <module>
    from .backends import (
  File "/home/ridan/.local/share/Anki2/addons21/spacyenv/lib/python3.12/site-packages/thinc/backends/__init__.py", line 17, in <module>
    from .cupy_ops import CupyOps
  File "/home/ridan/.local/share/Anki2/addons21/spacyenv/lib/python3.12/site-packages/thinc/backends/cupy_ops.py", line 16, in <module>
    from .numpy_ops import NumpyOps
  File "thinc/backends/numpy_ops.pyx", line 1, in init thinc.backends.numpy_ops
ValueError: numpy.dtype size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject

===Add-ons (active)===
(add-on provided name [Add-on folder, installed at, version, is config changed])
AnkiMorphs ['472573498', 2024-09-20T21:27, 'None', mod]

===IDs of active AnkiWeb add-ons===
472573498

===Add-ons (inactive)===
(add-on provided name [Add-on folder, installed at, version, is config changed])

A quick google search suggests that downgrading numpy should fix this, but numpy is already on 1.26.4 in my venv:

ridan@luna ~/.local/share/Anki2/addons21 % source spacyenv/bin/activate
ridan@luna ~/.local/share/Anki2/addons21 % pip freeze | grep numpy
numpy==1.26.4
ridan@luna ~/.local/share/Anki2/addons21 % python --version
Python 3.12.6

Unlike last time (#239), I do not have python-spacy or python-thinc installed system-wide. The python version of the venv is the same as the one Anki is running under.

Any clues?

[mortii]

spacy has been struggling to fix numpy issues since the v2 release a while ago (i.e. issue 13528).

A quick google search suggests that downgrading numpy should fix this, but numpy is already on 1.26.4 in my venv

From what I understand you should do the opposite of that, since the current spacy 3.12 python version supports numpy 2, iirc.

Have you tried to do a clean install of the virtual environment?

[zeroeightysix]

From what I understand you should do the opposite of that, since the current spacy 3.12 python version supports numpy 2, iirc.

I tried doing this, but unfortunately I can't install numpy 2, as the oldest numpy2-compatible version of thinc (==8.3.0) is incompatible with the latest spaCy (thinc<8.3.0). This is kind of puzzling, as that spaCy release advertises numpy 2 support..

I guess this is the problem described in https://github.com/explosion/spaCy/issues/13607 .

Have you tried to do a clean install of the virtual environment?

I have, but unfortunately this still installs the old version of numpy.

I suppose we need to wait for a new spaCy release to resolve the version conflicts :/

[mortii]

It's quite a mess, I don't envy the explosion devs right now...

I suppose we need to wait for a new spaCy release to resolve the version conflicts :/

If you don't want to wait, you could downgrade the spacy version to something that is compatible with the other packages. So, maybe something like this?

python3.12 -m venv spacyenv
source spacyenv/bin/activate
python -m pip install --upgrade pip setuptools wheel
python -m pip install numpy==1.26
python -m pip install thinc==8.2.5
python -m pip install spacy==3.7.5
...

Haven't tested that particular permutation though.

[mortii]

@zeroeightysix does it work with the latest spacy release?

[zeroeightysix]

@zeroeightysix does it work with the latest spacy release?

Hi, yes, it does!

I recreated the venv, and it worked out of the box again.

[mortii]

Awesome! Let me know if you have any other problems :)