HIVtrace numpy array error

[lentivirus]

ValueError: numpy.ndarray size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject

[stevenweaver]

Dear @lentivirus,

Thank you for reporting this issue. Can you please provide the output of your environment? pip freeze.

Best, Steven

[lentivirus]

(hiv) hivdr@DR:~$ pip freeze bioext==0.20.4 biopython==1.81 Cython==0.29.34 fakemp==0.9.1 gcc7==0.0.7 hivclustering==1.6.3 hivtrace==0.6.2 hppy==0.9.9 hyphy-python==0.1.10 joblib==1.2.0 numpy==1.20.0 pysam==0.21.0 scipy==1.10.1 six==1.16.0 tn93==1.2.0 tornado==6.3.1

[lentivirus]

I have even tried to build a similar environment using a yml file from other computer where hivtrace is working fine.

Still getting the numpy array error

Please suggest a way out

[lentivirus]

Hi there any updates regarding my last issue?

[stevenweaver]

Dear @lentivirus,

Which version of python are you using? From there, I should be able to replicate the issue.

Best, Steven

[lentivirus]

Hi steven, My Python version is 3.9.16

[stevenweaver]

Dear @lentivirus,

I've updated hivtrace to be compatible with Python version 3.9. Please try again by installing version 0.6.3.

Best, Steven

[lentivirus]

Tried installing as suggested, now got the following error

collect2: error: ld returned 1 exit status error: command '/usr/bin/g++' failed with exit code 1 [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip. error: legacy-install-failure

× Encountered error while trying to install package. ╰─> hyphy-python

note: This is an issue with the package mentioned above, not pip. hint: See above for output from the failure

[FynnFreyer]

@stevenweaver @lentivirus

I got somewhat similar results, trying to install hivtrace on Python 3.11, so I think the following is related

The Issue with hyphy-python seems to have been fixed by your commit, but now the BioExt package is acting up.

A little digging leads me to believe that the reason for this is that the Python C API changed after 3.8, which is why the Extensions for BioExt can't find "longintrepr.h", and fail to compile. I think this might be related to BioExt#26.

Another issue is that NumPy 1.20.3 (which is EOL) just doesn't support 3.11.

When cloning the BioExt Repo, commenting out the extensions (except tn93_extension), bumping the numpy version to 1.22 (also EOL, but supports Python 3.11) and then installing that, everything installs fine. Afterward, I can also install hivtrace cleanly on 3.11 (which probably needs those extensions to be available, so this is not a solution, but hopefully enough information to fix the problem). When not commenting them out, it fails.

Cython had a similar issue, but fixed it (which is why the cythonized tn93_extension still compiles cleanly even though it uses "longintrepr.h"). Here's the related discussion on the Python bugtracker

I will crosspost on the BioExt Issue as well, but in the meantime the workaround for people that need hivtrace would be to use something lower, I guess (Python 3.8 works on my machine™ at least).

Also, here's a minimal git patch to ease verification for you (GitHub unfortunately doesn't allow attaching these as a file).

diff --git a/setup.py b/setup.py
index 45a3fd2..eb33561 100644
--- a/setup.py
+++ b/setup.py
@@ -22,32 +22,6 @@ tn93_extension = cythonize([
         )])

 ext_modules = [
-    Extension(
-        'BioExt.align._align',
-        sources=[
-            os.path.join('BioExt', 'align', '_align.c'),
-            os.path.join('BioExt', 'align', 'alignment.c')
-            ],
-        include_dirs=np_inc,
-        libraries=['m'],
-        extra_compile_args=['-O3', '-I.']
-        ),
-    Extension(
-        'BioExt.merge._merge',
-        sources=[
-            os.path.join('BioExt', 'merge', '_merge.c'),
-            os.path.join('BioExt', 'merge', 'merge.cpp')
-            ],
-        extra_compile_args=['-O3', '-I.']
-        ),
-    Extension(
-        'BioExt.rateclass._rateclass',
-        sources=[
-            os.path.join('BioExt', 'rateclass', '_rateclass.cpp'),
-            os.path.join('BioExt', 'rateclass', 'rateclass.cpp')
-            ],
-        extra_compile_args=['-O3', '-I.']
-        )
     ] + tn93_extension

@@ -134,7 +108,7 @@ setup(
     ext_modules=ext_modules,
     install_requires=[
         'biopython >=1.78',
-        'numpy ==1.20.3',
+        'numpy ==1.22.4',
         'scipy >=0.15',
         'pysam >=0.17',
         'joblib',

[stevenweaver]

Fixed in 0.7.0