search

numba / llvmlite

A lightweight LLVM python binding for writing JIT compilers
https://llvmlite.pydata.org/
BSD 2-Clause "Simplified" License
1.9k stars 315 forks source link

OSError on in test_binding and test_ir #75

Closed ml31415 closed 9 years ago

ml31415 commented 9 years ago

I stumbled over this issue when recently updating numba. For llvmlite 0.5.0 and 0.5.1 I get the following error when running the test suite. This error is also raised, whenever I try to import numba. There already was a bugreport against numba for this ( https://github.com/numba/numba/issues/1150 ), but unfortunately it seems to be a bit lost.

This is the test log:

======================================================================
ERROR: llvmlite.tests.test_binding (unittest.loader.ModuleImportFailure)
----------------------------------------------------------------------
ImportError: Failed to import test module: llvmlite.tests.test_binding
Traceback (most recent call last):
  File "/usr/lib/python2.7/unittest/loader.py", line 254, in _find_tests
    module = self._get_module_from_name(name)
  File "/usr/lib/python2.7/unittest/loader.py", line 232, in _get_module_from_name
    __import__(name)
  File "/home/michael/workspace/llvmlite/llvmlite/tests/test_binding.py", line 12, in <module>
    from llvmlite import binding as llvm
  File "/home/michael/workspace/llvmlite/llvmlite/binding/__init__.py", line 6, in <module>
    from .dylib import *
  File "/home/michael/workspace/llvmlite/llvmlite/binding/dylib.py", line 4, in <module>
    from . import ffi
  File "/home/michael/workspace/llvmlite/llvmlite/binding/ffi.py", line 36, in <module>
    lib = ctypes.CDLL(os.path.join(_lib_dir, get_library_name()))
  File "/usr/lib/python2.7/ctypes/__init__.py", line 365, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: /home/michael/workspace/llvmlite/llvmlite/binding/libllvmlite.so: undefined symbol: _ZSt24__throw_out_of_range_fmtPKcz

======================================================================
ERROR: llvmlite.tests.test_ir (unittest.loader.ModuleImportFailure)
----------------------------------------------------------------------
ImportError: Failed to import test module: llvmlite.tests.test_ir
Traceback (most recent call last):
  File "/usr/lib/python2.7/unittest/loader.py", line 254, in _find_tests
    module = self._get_module_from_name(name)
  File "/usr/lib/python2.7/unittest/loader.py", line 232, in _get_module_from_name
    __import__(name)
  File "/home/michael/workspace/llvmlite/llvmlite/tests/test_ir.py", line 16, in <module>
    from llvmlite import binding as llvm
  File "/home/michael/workspace/llvmlite/llvmlite/binding/__init__.py", line 6, in <module>
    from .dylib import *
  File "/home/michael/workspace/llvmlite/llvmlite/binding/dylib.py", line 4, in <module>
    from . import ffi
  File "/home/michael/workspace/llvmlite/llvmlite/binding/ffi.py", line 36, in <module>
    lib = ctypes.CDLL(os.path.join(_lib_dir, get_library_name()))
  File "/usr/lib/python2.7/ctypes/__init__.py", line 365, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: /home/michael/workspace/llvmlite/llvmlite/binding/libllvmlite.so: undefined symbol: _ZSt24__throw_out_of_range_fmtPKcz
llc --version
LLVM (http://llvm.org/):
  LLVM version 3.5.0

  Optimized build.
  Built Oct  6 2014 (05:33:01).
  Default target: x86_64-pc-linux-gnu
  Host CPU: x86-64

  Registered Targets:
    aarch64    - AArch64 (little endian)
    aarch64_be - AArch64 (big endian)
    arm        - ARM
    arm64      - AArch64 (little endian)
    arm64_be   - AArch64 (big endian)
    armeb      - ARM (big endian)
    cpp        - C++ backend
    hexagon    - Hexagon
    mips       - Mips
    mips64     - Mips64 [experimental]
    mips64el   - Mips64el [experimental]
    mipsel     - Mipsel
    msp430     - MSP430 [experimental]
    nvptx      - NVIDIA PTX 32-bit
    nvptx64    - NVIDIA PTX 64-bit
    ppc32      - PowerPC 32
    ppc64      - PowerPC 64
    ppc64le    - PowerPC 64 LE
    r600       - AMD GPUs HD2XXX-HD6XXX
    sparc      - Sparc
    sparcv9    - Sparc V9
    systemz    - SystemZ
    thumb      - Thumb
    thumbeb    - Thumb (big endian)
    x86        - 32-bit X86: Pentium-Pro and above
    x86-64     - 64-bit X86: EM64T and AMD64
    xcore      - XCore

Any help would be greatly appreciated. Reinstalling llvmlite and numba was of no help, as well as switching back to 0.4.0. If I can help with providing any further information, please let me know.

pitrou commented 9 years ago

Which Ubuntu version do you have? Apparently you need to install llvm-dev: http://askubuntu.com/questions/583441/difference-between-llvm-dev-and-llvm-3-5-dev (note: this was found by running a Google search on the error message)

ml31415 commented 9 years ago

I'm using Ubuntu 14.04 and LLVM from source. The package is also already installed.

michael@nyx:~$ dpkg -l | grep llvm
ii  libllvm3.4:amd64                                      1:3.4.2-3ubuntu2~xedgers                               amd64        Modular compiler and toolchain technologies, runtime library
ii  libllvm3.5:amd64                                      1:3.5-4ubuntu2                                         amd64        Modular compiler and toolchain technologies, runtime library
ii  libllvm3.5:i386                                       1:3.5-4ubuntu2                                         i386         Modular compiler and toolchain technologies, runtime library
ii  libllvm3.6:amd64                                      1:3.6.2~svn238746-1~exp1                               amd64        Modular compiler and toolchain technologies, runtime library
ii  llvm                                                  1:3.4-0ubuntu1                                         amd64        Low-Level Virtual Machine (LLVM)
ii  llvm-3.4                                              1:3.4.2-3ubuntu2~xedgers                               amd64        Modular compiler and toolchain technologies
ii  llvm-3.4-dev                                          1:3.4.2-3ubuntu2~xedgers                               amd64        Modular compiler and toolchain technologies, libraries and headers
ii  llvm-3.4-runtime                                      1:3.4.2-3ubuntu2~xedgers                               amd64        Modular compiler and toolchain technologies, IR interpreter
ii  llvm-3.5                                              1:3.5-4ubuntu2                                         amd64        Modular compiler and toolchain technologies
ii  llvm-3.5-dev                                          1:3.5-4ubuntu2                                         amd64        Modular compiler and toolchain technologies, libraries and headers
ii  llvm-3.5-runtime                                      1:3.5-4ubuntu2                                         amd64        Modular compiler and toolchain technologies, IR interpreter
ii  llvm-3.6                                              1:3.6.2~svn238746-1~exp1                               amd64        Modular compiler and toolchain technologies
ii  llvm-3.6-dev                                          1:3.6.2~svn238746-1~exp1                               amd64        Modular compiler and toolchain technologies, libraries and headers
ii  llvm-3.6-runtime                                      1:3.6.2~svn238746-1~exp1                               amd64        Modular compiler and toolchain technologies, IR interpreter
ii  llvm-3.6-tools                                        1:3.6.2~svn238746-1~exp1                               amd64        Modular compiler and toolchain technologies, tools
ii  llvm-dev                                              1:3.4-0ubuntu1                                         amd64        Low-Level Virtual Machine (LLVM), libraries and headers
ii  llvm-runtime                                          1:3.4-0ubuntu1                                         amd64        Low-Level Virtual Machine (LLVM), bytecode interpreter

The symlinks should also be correct:

michael@nyx:~$ ll /etc/alternatives/llvm*
lrwxrwxrwx 1 root root 20 Mär  7 18:31 /etc/alternatives/llvm-as -> /usr/bin/llvm-as-3.5
lrwxrwxrwx 1 root root 24 Mär  7 18:31 /etc/alternatives/llvm-config -> /usr/bin/llvm-config-3.5
lrwxrwxrwx 1 root root 21 Mär  7 18:31 /etc/alternatives/llvm-dis -> /usr/bin/llvm-dis-3.5
lrwxrwxrwx 1 root root 22 Mär  7 18:31 /etc/alternatives/llvm-link -> /usr/bin/llvm-link-3.5
lrwxrwxrwx 1 root root 16 Mär  7 18:31 /etc/alternatives/llvm-llc -> /usr/bin/llc-3.5
lrwxrwxrwx 1 root root 16 Mär  7 18:31 /etc/alternatives/llvm-lli -> /usr/bin/lli-3.5
lrwxrwxrwx 1 root root 16 Mär  7 18:31 /etc/alternatives/llvm-opt -> /usr/bin/opt-3.5

Is there anything else I should check?

pitrou commented 9 years ago

With "LLVM from source", you mean you have your own compiled copy of LLVM? If so, you will have to point LLVM_CONFIG to its full path: http://llvmlite.pydata.org/en/latest/install/index.html#compiling

ml31415 commented 9 years ago

Found the problem. The symlinks in /usr/bin weren't pointing to the alternatives anymore, but instead to the 3.4 installation directly. I suppose any llvm package lately overwrote them. Thanks for the hints!

jkschin commented 8 years ago

I am currently facing this issue. I ran python runtests.py and it showed me a similar error as stated above.

The symbolic links point to /usr/lib/llvm-3.7/bin/llvm_config

@ml31415 how did you solve it?

ml31415 commented 8 years ago

Did you check the links in /usr/bin? They usually should point to /etc/alternatives. For me, they were pointing to some other versions directly, and I just hat to point them back to /etc/alternatives. If you want to make sure, that the right version is used, you can also set LLVM_CONFIG in the shell, before you start the installation, as pitrou mentioned already above.

jkschin commented 8 years ago

Yes I checked the links in /usr/bin

lrwxrwxrwx 1 root root 28 Oct 16 2015 bugpoint-3.7 -> ../lib/llvm-3.7/bin/bugpoint lrwxrwxrwx 1 root root 23 Oct 16 2015 llc-3.7 -> ../lib/llvm-3.7/bin/llc lrwxrwxrwx 1 root root 23 Oct 16 2015 lli-3.7 -> ../lib/llvm-3.7/bin/lli lrwxrwxrwx 1 root root 36 Oct 16 2015 lli-child-target-3.7 -> ../lib/llvm-3.7/bin/lli-child-target lrwxrwxrwx 1 root root 27 Oct 16 2015 llvm-ar-3.7 -> ../lib/llvm-3.7/bin/llvm-ar lrwxrwxrwx 1 root root 27 Oct 16 2015 llvm-as-3.7 -> ../lib/llvm-3.7/bin/llvm-as lrwxrwxrwx 1 root root 35 Oct 16 2015 llvm-bcanalyzer-3.7 -> ../lib/llvm-3.7/bin/llvm-bcanalyzer lrwxrwxrwx 1 root root 31 Oct 16 2015 llvm-config-3.7 -> ../lib/llvm-3.7/bin/llvm-config lrwxrwxrwx 1 root root 28 Oct 16 2015 llvm-cov-3.7 -> ../lib/llvm-3.7/bin/llvm-cov lrwxrwxrwx 1 root root 32 Oct 16 2015 llvm-cxxdump-3.7 -> ../lib/llvm-3.7/bin/llvm-cxxdump lrwxrwxrwx 1 root root 29 Oct 16 2015 llvm-diff-3.7 -> ../lib/llvm-3.7/bin/llvm-diff lrwxrwxrwx 1 root root 28 Oct 16 2015 llvm-dis-3.7 -> ../lib/llvm-3.7/bin/llvm-dis lrwxrwxrwx 1 root root 33 Oct 16 2015 llvm-dsymutil-3.7 -> ../lib/llvm-3.7/bin/llvm-dsymutil lrwxrwxrwx 1 root root 34 Oct 16 2015 llvm-dwarfdump-3.7 -> ../lib/llvm-3.7/bin/llvm-dwarfdump lrwxrwxrwx 1 root root 32 Oct 16 2015 llvm-extract-3.7 -> ../lib/llvm-3.7/bin/llvm-extract lrwxrwxrwx 1 root root 29 Oct 16 2015 llvm-link-3.7 -> ../lib/llvm-3.7/bin/llvm-link lrwxrwxrwx 1 root root 27 Oct 16 2015 llvm-mc-3.7 -> ../lib/llvm-3.7/bin/llvm-mc lrwxrwxrwx 1 root root 33 Oct 16 2015 llvm-mcmarkup-3.7 -> ../lib/llvm-3.7/bin/llvm-mcmarkup lrwxrwxrwx 1 root root 27 Oct 16 2015 llvm-nm-3.7 -> ../lib/llvm-3.7/bin/llvm-nm lrwxrwxrwx 1 root root 32 Oct 16 2015 llvm-objdump-3.7 -> ../lib/llvm-3.7/bin/llvm-objdump lrwxrwxrwx 1 root root 32 Oct 16 2015 llvm-pdbdump-3.7 -> ../lib/llvm-3.7/bin/llvm-pdbdump lrwxrwxrwx 1 root root 33 Oct 16 2015 llvm-profdata-3.7 -> ../lib/llvm-3.7/bin/llvm-profdata lrwxrwxrwx 1 root root 31 Oct 16 2015 llvm-ranlib-3.7 -> ../lib/llvm-3.7/bin/llvm-ranlib lrwxrwxrwx 1 root root 32 Oct 16 2015 llvm-readobj-3.7 -> ../lib/llvm-3.7/bin/llvm-readobj lrwxrwxrwx 1 root root 31 Oct 16 2015 llvm-rtdyld-3.7 -> ../lib/llvm-3.7/bin/llvm-rtdyld lrwxrwxrwx 1 root root 29 Oct 16 2015 llvm-size-3.7 -> ../lib/llvm-3.7/bin/llvm-size lrwxrwxrwx 1 root root 31 Oct 16 2015 llvm-stress-3.7 -> ../lib/llvm-3.7/bin/llvm-stress lrwxrwxrwx 1 root root 35 Oct 16 2015 llvm-symbolizer-3.7 -> ../lib/llvm-3.7/bin/llvm-symbolizer lrwxrwxrwx 1 root root 31 Oct 16 2015 llvm-tblgen-3.7 -> ../lib/llvm-3.7/bin/llvm-tblgen lrwxrwxrwx 1 root root 30 Oct 16 2015 macho-dump-3.7 -> ../lib/llvm-3.7/bin/macho-dump lrwxrwxrwx 1 root root 28 Oct 16 2015 obj2yaml-3.7 -> ../lib/llvm-3.7/bin/obj2yaml lrwxrwxrwx 1 root root 23 Oct 16 2015 opt-3.7 -> ../lib/llvm-3.7/bin/opt lrwxrwxrwx 1 root root 39 Oct 16 2015 verify-uselistorder-3.7 -> ../lib/llvm-3.7/bin/verify-uselistorder lrwxrwxrwx 1 root root 28 Oct 16 2015 yaml2obj-3.7 -> ../lib/llvm-3.7/bin/yaml2obj

I don't have any llvm type files in /etc/alternatives

LLVM_CONFIG=/usr/lib/llvm-3.7/bin/llvm-config

ml31415 commented 8 years ago

If you're not using the LLVM_CONFIG, you'd need any /usr/bin/llvm-config pointing to your 3.7 installed llvm-config link/binary. As this isn't there, and your llvm-config still seems to be somewhere in the system path, I suppose, you may have some custom install in /usr/local/bin or somewhere else in your path.

jkschin commented 8 years ago

I don't have a custom install of llvm-config in /usr/local/bin

bottle.py easy_install easy_install-2.7 f2py miniterm.py miniterm.pyc numba opencv_annotation opencv_createsamples opencv_traincascade opencv_visualisation opencv_waldboost_detector pio pip pip2 pip2.7 platformio pycc scons scons-2.5.0 sconsign sconsign-2.5.0 scons-time scons-time-2.5.0 wheel

Any other suggestions?

pitrou commented 8 years ago

If you want to debug this, try which llvm-config and llvm-config --version. Basically, if you want to compile llvmlite yourself, you have to have at least some system administrations and/or development skills.