Import error on fresh install

[b-grimaud]

As I was trying out some changes, none of them would take effect even though my pip install was editable, so I removed everything and did a fresh install from my fork which is up to date with the master branch.

pip installs just fine, but when I try to import HS2 in a Python shell :

Python 3.12.2 | packaged by Anaconda, Inc. | (main, Feb 27 2024, 17:35:02) [GCC 11.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import herdingspikes
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/user/Documents/GitHub/HS2/herdingspikes/__init__.py", line 4, in <module>
    from herdingspikes.hs2 import HSDetection, HSClustering, HSDetectionLightning
  File "/home/user/Documents/GitHub/HS2/herdingspikes/hs2.py", line 10, in <module>
    from .detection_localisation.detect import detectData
ImportError: /home/user/.conda/envs/spike/bin/../lib/libstdc++.so.6: version `GLIBCXX_3.4.32' not found (required by /home/user/Documents/GitHub/HS2/herdingspikes/detection_localisation/detect.cpython-312-x86_64-linux-gnu.so)
>>>

I also tried to recompile with the old way of installing HS2, through python setup.py install, which builds just fine, but same issue when I try to actually import it.

As far as I can tell, up until version 0.4.004 (this commit) it worked just fine.

Running Python 3.12.2 on Ubuntu 24.04. I checked and libstdc++6 is correctly installed and up to date on my machine.

[mhhennig]

Hmm, can you check if cython is re-run properly. You can maybe delete ‘detect.cpp’ in the ‘detection_lightning’ folder to make sure it is recreated.

[mhhennig]

Also, in the trace above it looks like the import is not from the pip installed version, but from the clone. This kind of error occurs when the C code is compiled the wrong way (I think).

[b-grimaud]

Hmm, can you check if cython is re-run properly.

I'm not too familiar with cython, should I look for something in the trace ?

You can maybe delete ‘detect.cpp’ in the ‘detection_lightning’ folder to make sure it is recreated.

Deleted it as well as the pycache folder, just in case. Re-ran pip install -e ., same error, then python setup.py clean and python setup.py install, no changes, then pip install -e . again for good measure but still no improvement.

Also, in the trace above it looks like the import is not from the pip installed version, but from the clone.

If I understand correctly, it's looking for the library in the environment folder instead of the system files ?

Maybe of note, in the build folder, I have a few different subfolders :

lib.linux-x86_64-cpython-310
lib.linux-x86_64-cpython-311
lib.linux-x86_64-cpython-312
temp.linux-x86_64-cpython-310
temp.linux-x86_64-cpython-311
temp.linux-x86_64-cpython-312

They're all dated from previous builds, but the folder dated from yesterday is bdist.linux-x86_64.

Maybe it's targeting the wrong build ?

[b-grimaud]

Small update :

I upgraded Cython to 3.0.11, removed everything again, uninstalled (twice) through pip then ran pip cache purge and python setup.py clean.

This time, building with python setup.py install fails with the following trace :

herdingspikes/detection_localisation/detect.cpp: In function ‘npy_intp __pyx_f_5numpy_5dtype_8itemsize_itemsize(PyArray_Descr*)’:
herdingspikes/detection_localisation/detect.cpp:4059:13: error: ‘PyDataType_ELSIZE’ was not declared in this scope; did you mean ‘PyDataType_ISUNSIZED’?
 4059 |   __pyx_r = PyDataType_ELSIZE(__pyx_v_self);
      |             ^~~~~~~~~~~~~~~~~
      |             PyDataType_ISUNSIZED
herdingspikes/detection_localisation/detect.cpp: In function ‘npy_intp __pyx_f_5numpy_5dtype_9alignment_alignment(PyArray_Descr*)’:
herdingspikes/detection_localisation/detect.cpp:4093:13: error: ‘PyDataType_ALIGNMENT’ was not declared in this scope; did you mean ‘PyDataType_ISSIGNED’?
 4093 |   __pyx_r = PyDataType_ALIGNMENT(__pyx_v_self);
      |             ^~~~~~~~~~~~~~~~~~~~
      |             PyDataType_ISSIGNED
herdingspikes/detection_localisation/detect.cpp: In function ‘PyObject* __pyx_f_5numpy_5dtype_6fields_fields(PyArray_Descr*)’:
herdingspikes/detection_localisation/detect.cpp:4131:15: error: ‘PyDataType_FIELDS’ was not declared in this scope; did you mean ‘PyDataType_HASFIELDS’?
 4131 |   __pyx_t_1 = PyDataType_FIELDS(__pyx_v_self);
      |               ^~~~~~~~~~~~~~~~~
      |               PyDataType_HASFIELDS
herdingspikes/detection_localisation/detect.cpp: In function ‘PyObject* __pyx_f_5numpy_5dtype_5names_names(PyArray_Descr*)’:
herdingspikes/detection_localisation/detect.cpp:4173:15: error: ‘PyDataType_NAMES’ was not declared in this scope; did you mean ‘PyDataType_ISNUMBER’?
 4173 |   __pyx_t_1 = PyDataType_NAMES(__pyx_v_self);
      |               ^~~~~~~~~~~~~~~~
      |               PyDataType_ISNUMBER
herdingspikes/detection_localisation/detect.cpp: In function ‘PyArray_ArrayDescr* __pyx_f_5numpy_5dtype_8subarray_subarray(PyArray_Descr*)’:
herdingspikes/detection_localisation/detect.cpp:4211:13: error: ‘PyDataType_SUBARRAY’ was not declared in this scope; did you mean ‘PyDataType_HASSUBARRAY’?
 4211 |   __pyx_r = PyDataType_SUBARRAY(__pyx_v_self);
      |             ^~~~~~~~~~~~~~~~~~~
      |             PyDataType_HASSUBARRAY
herdingspikes/detection_localisation/detect.cpp: In function ‘npy_uint64 __pyx_f_5numpy_5dtype_5flags_flags(PyArray_Descr*)’:
herdingspikes/detection_localisation/detect.cpp:4245:13: error: ‘PyDataType_FLAGS’ was not declared in this scope; did you mean ‘PyDataType_FLAGCHK’?
 4245 |   __pyx_r = PyDataType_FLAGS(__pyx_v_self);
      |             ^~~~~~~~~~~~~~~~
      |             PyDataType_FLAGCHK
herdingspikes/detection_localisation/detect.cpp: In function ‘int __pyx_f_5numpy_9broadcast_7numiter_numiter(PyArrayMultiIterObject*)’:
herdingspikes/detection_localisation/detect.cpp:4279:13: error: ‘PyArray_MultiIter_NUMITER’ was not declared in this scope; did you mean ‘PyArray_MultiIter_NEXTi’?
 4279 |   __pyx_r = PyArray_MultiIter_NUMITER(__pyx_v_self);
      |             ^~~~~~~~~~~~~~~~~~~~~~~~~
      |             PyArray_MultiIter_NEXTi
herdingspikes/detection_localisation/detect.cpp: In function ‘npy_intp __pyx_f_5numpy_9broadcast_4size_size(PyArrayMultiIterObject*)’:
herdingspikes/detection_localisation/detect.cpp:4313:13: error: ‘PyArray_MultiIter_SIZE’ was not declared in this scope; did you mean ‘PyArray_MultiIter_RESET’?
 4313 |   __pyx_r = PyArray_MultiIter_SIZE(__pyx_v_self);
      |             ^~~~~~~~~~~~~~~~~~~~~~
      |             PyArray_MultiIter_RESET
herdingspikes/detection_localisation/detect.cpp: In function ‘npy_intp __pyx_f_5numpy_9broadcast_5index_index(PyArrayMultiIterObject*)’:
herdingspikes/detection_localisation/detect.cpp:4347:13: error: ‘PyArray_MultiIter_INDEX’ was not declared in this scope; did you mean ‘PyArray_MultiIter_NEXT’?
 4347 |   __pyx_r = PyArray_MultiIter_INDEX(__pyx_v_self);
      |             ^~~~~~~~~~~~~~~~~~~~~~~
      |             PyArray_MultiIter_NEXT
herdingspikes/detection_localisation/detect.cpp: In function ‘int __pyx_f_5numpy_9broadcast_2nd_nd(PyArrayMultiIterObject*)’:
herdingspikes/detection_localisation/detect.cpp:4381:13: error: ‘PyArray_MultiIter_NDIM’ was not declared in this scope; did you mean ‘PyArray_MultiIter_NEXT’?
 4381 |   __pyx_r = PyArray_MultiIter_NDIM(__pyx_v_self);
      |             ^~~~~~~~~~~~~~~~~~~~~~
      |             PyArray_MultiIter_NEXT
herdingspikes/detection_localisation/detect.cpp: In function ‘npy_intp* __pyx_f_5numpy_9broadcast_10dimensions_dimensions(PyArrayMultiIterObject*)’:
herdingspikes/detection_localisation/detect.cpp:4415:13: error: ‘PyArray_MultiIter_DIMS’ was not declared in this scope; did you mean ‘PyArray_MultiIter_DATA’?
 4415 |   __pyx_r = PyArray_MultiIter_DIMS(__pyx_v_self);
      |             ^~~~~~~~~~~~~~~~~~~~~~
      |             PyArray_MultiIter_DATA
herdingspikes/detection_localisation/detect.cpp: In function ‘void** __pyx_f_5numpy_9broadcast_5iters_iters(PyArrayMultiIterObject*)’:
herdingspikes/detection_localisation/detect.cpp:4449:13: error: ‘PyArray_MultiIter_ITERS’ was not declared in this scope; did you mean ‘PyArray_MultiIter_NEXT’?
 4449 |   __pyx_r = PyArray_MultiIter_ITERS(__pyx_v_self);
      |             ^~~~~~~~~~~~~~~~~~~~~~~
      |             PyArray_MultiIter_NEXT
error: command '/usr/bin/gcc' failed with exit code 1

The pip editable install still builds just fine, but the import bug is still there.

[mhhennig]

Can you try deleting herdingspikes/detection_localisation/detect.cpp and then try to build. This is due to a version issue with numpy I think, and it should go away.

[b-grimaud]

Tried it, no changes. I'm running numpy 1.26.4, which seems to be the latest version that scipy supports.

Yet another layer to the issue : if I try to build HS2 from pip in non-editable mode, i.e. without the -e flag, the build fails with the following trace :

Building wheels for collected packages: herdingspikes
  Building wheel for herdingspikes (pyproject.toml) ... done
  Created wheel for herdingspikes: filename=herdingspikes-0.4.5-cp312-cp312-linux_x86_64.whl size=335861 sha256=3c255018c6e9bd45bb1c781634537d356715379b67f4741646d58a75d4dff0c8
  Stored in directory: /tmp/pip-ephem-wheel-cache-5mohvu_u/wheels/5a/78/5f/efd68ccd8a89f65e88ee3cfdc8675527583431a40d5052d26b
  WARNING: Built wheel for herdingspikes is invalid: Wheel has unexpected file name: expected '0.4.5+git.38a55483', got '0.4.5'
Failed to build herdingspikes
ERROR: Could not build wheels for herdingspikes, which is required to install pyproject.toml-based projects