robbmcleod / pyfastnoisesimd

Python module wrapping C++ FastNoiseSIMD
BSD 3-Clause "New" or "Revised" License
39 stars 6 forks source link

Broken build on non-x86_64 systems #7

Closed QuLogic closed 6 years ago

QuLogic commented 6 years ago

On 32-bit systems, the build completes, but attempting to run the tests fails due to a missing symbol. https://kojipkgs.fedoraproject.org//work/tasks/5861/23965861/build.log

+ /usr/bin/python3 -c 'import sys; import pyfastnoisesimd; sys.exit(0 if pyfastnoisesimd.test().wasSuccessful() else 1)'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/builddir/build/BUILDROOT/python-pyfastnoisesimd-0.2.1-1.fc28.i386/usr/lib/python3.6/site-packages/pyfastnoisesimd/__init__.py", line 14, in <module>
    from pyfastnoisesimd.helpers import (
  File "/builddir/build/BUILDROOT/python-pyfastnoisesimd-0.2.1-1.fc28.i386/usr/lib/python3.6/site-packages/pyfastnoisesimd/helpers.py", line 1, in <module>
    import pyfastnoisesimd.extension as ext
ImportError: /builddir/build/BUILDROOT/python-pyfastnoisesimd-0.2.1-1.fc28.i386/usr/lib/python3.6/site-packages/pyfastnoisesimd/extension.cpython-36m-i386-linux-gnu.so: undefined symbol: _ZN22FastNoiseSIMD_internal16FastNoiseSIMD_L0C1Ei

L0 is the No-SIMD fallback, so I think this is a problem with the arch detection.

On all other arches, the build does not complete at all due to a problem in cpu detection:

Traceback (most recent call last):
  File "setup.py", line 126, in <module>
    if cpu_info['arch'] == 'ARM_8': # Detected by compiler flags
TypeError: 'NoneType' object is not subscriptable
QuLogic commented 6 years ago

I think the best way to fix this is to always build all options and let FastNoiseSIMD do feature detection. Unfortunately, I have yet to find a good example of someone passing per-file compilation options to a C extension.

QuLogic commented 6 years ago

I have a work-in-progress to attempt the above change. Unfortunately, 32-bit still fails for some reason. The other arches also fail due to problems in FastNoiseSIMD that I'll have to take upstream.

robbmcleod commented 6 years ago

Oh, I did that with python-blosc. You build build_clib first and then package them into the extension.

See here:

https://github.com/Blosc/python-blosc/pull/129/files

Diff line 119 of setup.py, I added separate flags to the Snappy codec.

QuLogic commented 6 years ago

Ohhh, thanks, that looks much simpler. Cursed by the lack of setuptools documentation again...

robbmcleod commented 6 years ago

I added the latest version of cpuinfo.py from the source in 36fcc785243cb0a9668408cf8794b2091de7a090, from the repo at:

https://github.com/workhorsy/py-cpuinfo

That might help with returning flags correctly, but it kind of sounds like the problem is in the no SIMD branch of the macros.