search

MaartenGr / BERTopic

Leveraging BERT and c-TF-IDF to create easily interpretable topics.
https://maartengr.github.io/BERTopic/
MIT License
6.21k stars 767 forks source link

Spacy,Numpy,Thinc incompatibility #2232

Open 0xtunafresh opened 2 days ago

0xtunafresh commented 2 days ago

Have you searched existing issues? 🔎

Desribe the bug

I want to use spacy.load('xx-ent-wiki-sm') but

I got error "File "thinc/backends/numpy_ops.pyx", line 1, in init thinc.backends.numpy_ops ValueError: numpy.dtype size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject"

pip list : thinc : 8.1.8 numpy :1.26.4 spacy:3.6.0

python : 3.9

Reproduction

nlp = spacy.load('xx-ent-wiki-sm')

BERTopic Version

jsjtadie commented 2 days ago

Compatibility issues among Spacy, Numpy, and Thinc can arise due to different version requirements.

Spacy depends on specific versions of Numpy and Thinc for proper functionality. For example, if an older version of Spacy is used with a newer version of Numpy that has introduced breaking changes in its API, it may lead to errors such as function signature mismatches or unexpected behavior in mathematical operations that Spacy relies on.

Similarly, Thinc has its own dependencies and compatibility constraints with the other two. If the versions are not in harmony, issues like module import failures, segmentation faults during runtime, or incorrect results in neural - network - related computations (if applicable) might occur. It's crucial to ensure that the versions of Spacy, Numpy, and Thinc are carefully selected to work well together, usually by referring to the official documentation of each library for their compatibility matrices.

MaartenGr commented 2 days ago

@jsjtadie Is that your response or that of an LLM? Could you explain in more detail what exactly the OP should do here?

@0xtunafresh The best thing to do in these situations is to start from a completely fresh environment and then install the required dependencies. Make sure to run pip install bertopic[spacy] as that will attempt to install them both together. If that does not work, then I would advise sharing this issue on the official spacy repository. Since your code doesn't seem to be using BERTopic but merely installing it, it might be better suited over there.