search

SuffolkLITLab / FormFyxer

A tool for learning about and pre-processing forms
MIT License
11 stars 1 forks source link

Error installing: `ValueError: numpy.dtype size changed` #135

Open BryceStevenWilley opened 1 month ago

BryceStevenWilley commented 1 month ago

Having a lot of trouble installing FormFyxer on a fresh virtual env. Below is the error

    File "/tmp/pip-build-env-jyabgwzo/overlay/lib/python3.10/site-packages/thinc/backends/__init__.py", line 17, in <module>
     from .cupy_ops import CupyOps
  File "/tmp/pip-build-env-jyabgwzo/overlay/lib/python3.10/site-packages/thinc/backends/cupy_ops.py", line 16, in <module>
     from .numpy_ops import NumpyOps
   File "thinc/backends/numpy_ops.pyx", line 1, in init thinc.backends.numpy_ops
 ValueError: numpy.dtype size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject

Related to https://github.com/explosion/spaCy/issues/13528, but I seem to be having the same error when I fix the version of numpy to below 2.0.0. Not sure why the obvious fix isn't working, but pip runs all of it in a temporary dir that gets deleted, so I can't see what actual code is being run there.

Workaround is just to remove the entire InstallSpacyModelCommand:

https://github.com/SuffolkLITLab/FormFyxer/blob/fce844760001c8998a3d40765bf894cf4f7d50cd/setup.py#L9-L14

That workaround won't fly for our situation, but I'll have to see if it breaks our GH actions as well, or if it's just on my machine.

BryceStevenWilley commented 1 month ago

An example of the build failure: https://github.com/SuffolkLITLab/FormFyxer/actions/runs/9540258846/job/26291740248?pr=136

BryceStevenWilley commented 1 month ago

Figured out a local fix: from the thinc docs, you can specify build constraints. I think since we're running spacy while building/installing formfyxer, the normal fix of "just use numpy<2.0.0" doesn't work because of how pip installs dependencies when building packages. Can't fully articulate, but if you run:

PIP_CONSTRAINT=build-constraints.txt pip install . --no-cache-dir

with build-constraints.txt as:

numpy==1.26.4

it works.

Happy to hear additional approaches, but I think the best way to fix this at the moment is to patch ALActions to have a similar build-constraints.txt in it, and then use it when building everything. It shouldn't matter for most repos, but I think this spacy / thinc / numpy issue will pop up in a few different places.

BryceStevenWilley commented 1 month ago

Okay, with https://github.com/SuffolkLITLab/ALActions/pull/26, this should be fixed on CI/CD. Not sure if we should add something to the docs saying that pip install . won't work though.

TBH, I'm sure this will start affecting a few people come this week, and spacy and thinc might come up with better workarounds, so I might keep this open just to wait for those.