JonathanSalwan / Triton

Triton is a dynamic binary analysis library. Build your own program analysis tools, automate your reverse engineering, perform software verification or just emulate code.
https://triton-library.github.io
Apache License 2.0
3.39k stars 524 forks source link

[OSX ERROR] SystemError: initialization of triton did not return an extension module #1308

Closed patrafter1999 closed 3 months ago

patrafter1999 commented 4 months ago

Hi Guys,

I have spent half a day trying to install this very promising package on my Mac OSX (Sonoma 14.3.1), but it ended with this weird error, which I can't debug any more. Your help will be greatly appreciated.

Regards,


(ai) ~/Downloads/Triton/dist:$ python
Python 3.10.8 (main, Nov 24 2022, 08:08:27) [Clang 14.0.6 ] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import triton
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
SystemError: initialization of triton did not return an extension module
>>>
(ai) ~/Downloads/Triton:$ python -m build --wheel
* Creating venv isolated environment...
* Installing packages in isolated environment... (setuptools >= 40.8.0, wheel)
* Getting build dependencies for wheel...
running egg_info
writing triton_library.egg-info/PKG-INFO
writing dependency_links to triton_library.egg-info/dependency_links.txt
writing top-level names to triton_library.egg-info/top_level.txt
reading manifest file 'triton_library.egg-info/SOURCES.txt'
adding license file 'LICENSE.txt'
writing manifest file 'triton_library.egg-info/SOURCES.txt'
* Installing packages in isolated environment... (wheel)
* Building wheel...
running bdist_wheel
running build
running build_ext
-- The C compiler identification is AppleClang 15.0.0.15000100
-- The CXX compiler identification is AppleClang 15.0.0.15000100
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /Library/Developer/CommandLineTools/usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /Library/Developer/CommandLineTools/usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Compiling with Python bindings
-- Found Python3: /private/var/folders/l7/wnf9k67s0w33rvzchxqh6_gw0000gn/T/build-env-yu5af59l/bin/python (found version "3.10.8") found components: Interpreter Development Development.Module Development.Embed
-- Python3 includes: /Users/jsmith/opt/anaconda3/envs/ai/include/python3.10
-- Python3 libraries: /Users/jsmith/opt/anaconda3/envs/ai/lib/libpython3.10.dylib
-- Compiling with Z3 SMT solver
-- Found Z3 include directory: /opt/homebrew/include
-- Found Z3 library: /opt/homebrew/lib/libz3.dylib
-- Z3 version: 4.12.5.0
-- Compiling with Capstone
-- Found CAPSTONE include directory: /opt/homebrew/include
-- Found CAPSTONE library: /opt/homebrew/lib/libcapstone.dylib
-- CAPSTONE version: 5.0.1
-- Compiling with wide-integer headers
-- Compiling with Python autocomplete stub
-- Configuring done (1.0s)
-- Generating done (0.1s)
-- Build files have been written to: /Users/jsmith/Downloads/Triton/build/temp.macosx-11.1-arm64-cpython-310
[  2%] Building CXX object src/libtriton/CMakeFiles/triton.dir/arch/arm/aarch64/aarch64Semantics.cpp.o
[  4%] Building CXX object src/libtriton/CMakeFiles/triton.dir/arch/architecture.cpp.o
[  4%] Building CXX object src/libtriton/CMakeFiles/triton.dir/arch/arm/aarch64/aarch64Specifications.cpp.o
[  4%] Building CXX object src/libtriton/CMakeFiles/triton.dir/arch/arm/aarch64/aarch64Cpu.cpp.o
[  5%] Building CXX object src/libtriton/CMakeFiles/triton.dir/arch/arm/arm32/arm32Cpu.cpp.o
[  6%] Building CXX object src/libtriton/CMakeFiles/triton.dir/arch/arm/arm32/arm32Semantics.cpp.o
[  8%] Building CXX object src/libtriton/CMakeFiles/triton.dir/arch/arm/arm32/arm32Specifications.cpp.o
[  9%] Building CXX object src/libtriton/CMakeFiles/triton.dir/arch/arm/armOperandProperties.cpp.o
[ 10%] Building CXX object src/libtriton/CMakeFiles/triton.dir/arch/basicBlock.cpp.o
[ 11%] Building CXX object src/libtriton/CMakeFiles/triton.dir/arch/bitsVector.cpp.o
[ 12%] Building CXX object src/libtriton/CMakeFiles/triton.dir/arch/immediate.cpp.o
[ 13%] Building CXX object src/libtriton/CMakeFiles/triton.dir/arch/instruction.cpp.o
[ 13%] Building CXX object src/libtriton/CMakeFiles/triton.dir/arch/irBuilder.cpp.o
[ 14%] Building CXX object src/libtriton/CMakeFiles/triton.dir/arch/memoryAccess.cpp.o
[ 16%] Building CXX object src/libtriton/CMakeFiles/triton.dir/arch/operandWrapper.cpp.o
[ 17%] Building CXX object src/libtriton/CMakeFiles/triton.dir/arch/register.cpp.o
[ 18%] Building CXX object src/libtriton/CMakeFiles/triton.dir/arch/x86/x8664Cpu.cpp.o
[ 19%] Building CXX object src/libtriton/CMakeFiles/triton.dir/arch/x86/x86Cpu.cpp.o
[ 20%] Building CXX object src/libtriton/CMakeFiles/triton.dir/arch/x86/x86Semantics.cpp.o
[ 21%] Building CXX object src/libtriton/CMakeFiles/triton.dir/arch/x86/x86Specifications.cpp.o
[ 24%] Building CXX object src/libtriton/CMakeFiles/triton.dir/ast/astContext.cpp.o
[ 24%] Building CXX object src/libtriton/CMakeFiles/triton.dir/ast/ast.cpp.o
[ 25%] Building CXX object src/libtriton/CMakeFiles/triton.dir/ast/representations/astPcodeRepresentation.cpp.o
[ 26%] Building CXX object src/libtriton/CMakeFiles/triton.dir/ast/representations/astPythonRepresentation.cpp.o
[ 27%] Building CXX object src/libtriton/CMakeFiles/triton.dir/ast/representations/astRepresentation.cpp.o
[ 28%] Building CXX object src/libtriton/CMakeFiles/triton.dir/ast/representations/astSmtRepresentation.cpp.o
[ 29%] Building CXX object src/libtriton/CMakeFiles/triton.dir/callbacks/callbacks.cpp.o
[ 31%] Building CXX object src/libtriton/CMakeFiles/triton.dir/context/context.cpp.o
[ 32%] Building CXX object src/libtriton/CMakeFiles/triton.dir/engines/lifters/liftingToDot.cpp.o
[ 33%] Building CXX object src/libtriton/CMakeFiles/triton.dir/engines/lifters/liftingToPython.cpp.o
[ 34%] Building CXX object src/libtriton/CMakeFiles/triton.dir/engines/lifters/liftingToSMT.cpp.o
[ 35%] Building CXX object src/libtriton/CMakeFiles/triton.dir/engines/solver/solverEngine.cpp.o
[ 36%] Building CXX object src/libtriton/CMakeFiles/triton.dir/engines/solver/solverModel.cpp.o
[ 37%] Building CXX object src/libtriton/CMakeFiles/triton.dir/engines/symbolic/pathConstraint.cpp.o
[ 39%] Building CXX object src/libtriton/CMakeFiles/triton.dir/engines/symbolic/pathManager.cpp.o
[ 40%] Building CXX object src/libtriton/CMakeFiles/triton.dir/engines/symbolic/symbolicEngine.cpp.o
[ 41%] Building CXX object src/libtriton/CMakeFiles/triton.dir/engines/symbolic/symbolicExpression.cpp.o
[ 42%] Building CXX object src/libtriton/CMakeFiles/triton.dir/engines/symbolic/symbolicSimplification.cpp.o
[ 42%] Building CXX object src/libtriton/CMakeFiles/triton.dir/engines/symbolic/symbolicVariable.cpp.o
[ 43%] Building CXX object src/libtriton/CMakeFiles/triton.dir/engines/synthesis/oracleTable.cpp.o
[ 44%] Building CXX object src/libtriton/CMakeFiles/triton.dir/engines/synthesis/synthesisResult.cpp.o
[ 45%] Building CXX object src/libtriton/CMakeFiles/triton.dir/engines/synthesis/synthesizer.cpp.o
[ 47%] Building CXX object src/libtriton/CMakeFiles/triton.dir/engines/taint/taintEngine.cpp.o
[ 48%] Building CXX object src/libtriton/CMakeFiles/triton.dir/modes/modes.cpp.o
[ 49%] Building CXX object src/libtriton/CMakeFiles/triton.dir/stubs/aarch64-libc.cpp.o
[ 50%] Building CXX object src/libtriton/CMakeFiles/triton.dir/stubs/i386-systemv-libc.cpp.o
[ 51%] Building CXX object src/libtriton/CMakeFiles/triton.dir/stubs/x8664-ms-libc.cpp.o
[ 52%] Building CXX object src/libtriton/CMakeFiles/triton.dir/stubs/x8664-systemv-libc.cpp.o
[ 54%] Building CXX object src/libtriton/CMakeFiles/triton.dir/utils/coreUtils.cpp.o
[ 55%] Building CXX object src/libtriton/CMakeFiles/triton.dir/ast/z3/tritonToZ3.cpp.o
[ 56%] Building CXX object src/libtriton/CMakeFiles/triton.dir/ast/z3/z3ToTriton.cpp.o
[ 57%] Building CXX object src/libtriton/CMakeFiles/triton.dir/engines/solver/z3/z3Solver.cpp.o
[ 58%] Building CXX object src/libtriton/CMakeFiles/triton.dir/bindings/python/init.cpp.o
[ 59%] Building CXX object src/libtriton/CMakeFiles/triton.dir/bindings/python/modules/tritonCallbacks.cpp.o
[ 60%] Building CXX object src/libtriton/CMakeFiles/triton.dir/bindings/python/namespaces/initArchNamespace.cpp.o
[ 62%] Building CXX object src/libtriton/CMakeFiles/triton.dir/bindings/python/namespaces/initAstNodeNamespace.cpp.o
[ 63%] Building CXX object src/libtriton/CMakeFiles/triton.dir/bindings/python/namespaces/initAstRepresentationNamespace.cpp.o
[ 64%] Building CXX object src/libtriton/CMakeFiles/triton.dir/bindings/python/namespaces/initCallbackNamespace.cpp.o
[ 65%] Building CXX object src/libtriton/CMakeFiles/triton.dir/bindings/python/namespaces/initConditionsNamespace.cpp.o
[ 66%] Building CXX object src/libtriton/CMakeFiles/triton.dir/bindings/python/namespaces/initCpuSizeNamespace.cpp.o
[ 67%] Building CXX object src/libtriton/CMakeFiles/triton.dir/bindings/python/namespaces/initExceptionNamespace.cpp.o
[ 68%] Building CXX object src/libtriton/CMakeFiles/triton.dir/bindings/python/namespaces/initExtendNamespace.cpp.o
[ 70%] Building CXX object src/libtriton/CMakeFiles/triton.dir/bindings/python/namespaces/initModeNamespace.cpp.o
[ 71%] Building CXX object src/libtriton/CMakeFiles/triton.dir/bindings/python/namespaces/initOpcodesNamespace.cpp.o
[ 71%] Building CXX object src/libtriton/CMakeFiles/triton.dir/bindings/python/namespaces/initOperandNamespace.cpp.o
[ 72%] Building CXX object src/libtriton/CMakeFiles/triton.dir/bindings/python/namespaces/initPrefixesNamespace.cpp.o
[ 73%] Building CXX object src/libtriton/CMakeFiles/triton.dir/bindings/python/namespaces/initRegNamespace.cpp.o
[ 74%] Building CXX object src/libtriton/CMakeFiles/triton.dir/bindings/python/namespaces/initShiftsNamespace.cpp.o
[ 75%] Building CXX object src/libtriton/CMakeFiles/triton.dir/bindings/python/namespaces/initSolverNamespace.cpp.o
[ 77%] Building CXX object src/libtriton/CMakeFiles/triton.dir/bindings/python/namespaces/initSolverStateNamespace.cpp.o
[ 78%] Building CXX object src/libtriton/CMakeFiles/triton.dir/bindings/python/namespaces/initStubsNamespace.cpp.o
[ 79%] Building CXX object src/libtriton/CMakeFiles/triton.dir/bindings/python/namespaces/initSymbolicNamespace.cpp.o
[ 80%] Building CXX object src/libtriton/CMakeFiles/triton.dir/bindings/python/namespaces/initVASNamespace.cpp.o
[ 81%] Building CXX object src/libtriton/CMakeFiles/triton.dir/bindings/python/namespaces/initVersionNamespace.cpp.o
[ 82%] Building CXX object src/libtriton/CMakeFiles/triton.dir/bindings/python/objects/pyAstContext.cpp.o
[ 83%] Building CXX object src/libtriton/CMakeFiles/triton.dir/bindings/python/objects/pyAstNode.cpp.o
[ 86%] Building CXX object src/libtriton/CMakeFiles/triton.dir/bindings/python/objects/pyBitsVector.cpp.o
[ 86%] Building CXX object src/libtriton/CMakeFiles/triton.dir/bindings/python/objects/pyBasicBlock.cpp.o
[ 87%] Building CXX object src/libtriton/CMakeFiles/triton.dir/bindings/python/objects/pyImmediate.cpp.o
[ 88%] Building CXX object src/libtriton/CMakeFiles/triton.dir/bindings/python/objects/pyInstruction.cpp.o
[ 89%] Building CXX object src/libtriton/CMakeFiles/triton.dir/bindings/python/objects/pyMemoryAccess.cpp.o
[ 90%] Building CXX object src/libtriton/CMakeFiles/triton.dir/bindings/python/objects/pyPathConstraint.cpp.o
[ 91%] Building CXX object src/libtriton/CMakeFiles/triton.dir/bindings/python/objects/pyRegister.cpp.o
[ 93%] Building CXX object src/libtriton/CMakeFiles/triton.dir/bindings/python/objects/pySolverModel.cpp.o
[ 94%] Building CXX object src/libtriton/CMakeFiles/triton.dir/bindings/python/objects/pySymbolicExpression.cpp.o
[ 95%] Building CXX object src/libtriton/CMakeFiles/triton.dir/bindings/python/objects/pySymbolicVariable.cpp.o
[ 96%] Building CXX object src/libtriton/CMakeFiles/triton.dir/bindings/python/objects/pyTritonContext.cpp.o
[ 97%] Building CXX object src/libtriton/CMakeFiles/triton.dir/bindings/python/pyXFunctions.cpp.o
[ 98%] Building CXX object src/libtriton/CMakeFiles/triton.dir/bindings/python/utils.cpp.o
[100%] Linking CXX shared library libtriton.dylib
[100%] Built target triton
[100%] Built target python-triton
Built target gen_arm32_doc_from_spec
Built target gen_x86_doc_from_spec
Built target gen_aarch64_doc_from_spec
Generating without bitwuzla
warning: empty namespace STUBS
autocomplete generation done
Built target python_autocomplete
copying build/temp.macosx-11.1-arm64-cpython-310/src/libtriton/libtriton.dylib -> build/lib.macosx-11.1-arm64-cpython-310/triton.cpython-310-darwin.so
copying build/temp.macosx-11.1-arm64-cpython-310/doc/triton_autocomplete/triton.pyi -> build/lib.macosx-11.1-arm64-cpython-310
installing to build/bdist.macosx-11.1-arm64/wheel
running install
running install_lib
creating build/bdist.macosx-11.1-arm64
creating build/bdist.macosx-11.1-arm64/wheel
copying build/lib.macosx-11.1-arm64-cpython-310/triton.pyi -> build/bdist.macosx-11.1-arm64/wheel
copying build/lib.macosx-11.1-arm64-cpython-310/triton.cpython-310-darwin.so -> build/bdist.macosx-11.1-arm64/wheel
running install_egg_info
running egg_info
writing triton_library.egg-info/PKG-INFO
writing dependency_links to triton_library.egg-info/dependency_links.txt
writing top-level names to triton_library.egg-info/top_level.txt
reading manifest file 'triton_library.egg-info/SOURCES.txt'
adding license file 'LICENSE.txt'
writing manifest file 'triton_library.egg-info/SOURCES.txt'
Copying triton_library.egg-info to build/bdist.macosx-11.1-arm64/wheel/triton_library-1.0.0rc2-py3.10.egg-info
running install_scripts
[WARNING] This wheel needs a higher macOS version than the version your Python interpreter is compiled against.  To silence this warning, set MACOSX_DEPLOYMENT_TARGET to at least 14_0 or recreate these files with lower MACOSX_DEPLOYMENT_TARGET:
build/bdist.macosx-11.1-arm64/wheel/triton.cpython-310-darwin.so[WARNING] This wheel needs a higher macOS version than the version your Python interpreter is compiled against.  To silence this warning, set MACOSX_DEPLOYMENT_TARGET to at least 14_0 or recreate these files with lower MACOSX_DEPLOYMENT_TARGET:
build/bdist.macosx-11.1-arm64/wheel/triton.cpython-310-darwin.socreating build/bdist.macosx-11.1-arm64/wheel/triton_library-1.0.0rc2.dist-info/WHEEL
creating '/Users/jsmith/Downloads/Triton/dist/.tmp-i2ehwfyu/triton_library-1.0.0rc2-cp310-cp310-macosx_14_0_arm64.whl' and adding 'build/bdist.macosx-11.1-arm64/wheel' to it
adding 'triton.cpython-310-darwin.so'
adding 'triton.pyi'
adding 'triton_library-1.0.0rc2.dist-info/LICENSE.txt'
adding 'triton_library-1.0.0rc2.dist-info/METADATA'
adding 'triton_library-1.0.0rc2.dist-info/WHEEL'
adding 'triton_library-1.0.0rc2.dist-info/top_level.txt'
adding 'triton_library-1.0.0rc2.dist-info/RECORD'
removing build/bdist.macosx-11.1-arm64/wheel
Successfully built triton_library-1.0.0rc2-cp310-cp310-macosx_14_0_arm64.whl
(ai) ~/Downloads/Triton:$ cd dist
(ai) ~/Downloads/Triton/dist:$ pip install triton_library-1.0.0rc2-cp310-cp310-macosx_14_0_arm64.whl
Processing ./triton_library-1.0.0rc2-cp310-cp310-macosx_14_0_arm64.whl
Installing collected packages: triton-library
Successfully installed triton-library-1.0.0rc2
patrafter1999 commented 4 months ago

FYI, I followed the install instructions, which all went successfully.

$ git clone https://github.com/JonathanSalwan/Triton
$ cd Triton
$ mkdir build ; cd build
$ cmake ..
$ make -j3
JonathanSalwan commented 4 months ago

FYI, I followed the install instructions, which all went successfully.

$ git clone https://github.com/JonathanSalwan/Triton
$ cd Triton
$ mkdir build ; cd build
$ cmake ..
$ make -j3

So is it working? Once you did make -j3, you can then use make install and the python package will be pushed into your site-packages. (no need to do python -m build --wheel)

I'm also using MacOS with custom dependencies and below my compile:

cmake -DCAPSTONE_INCLUDE_DIRS=/Users/jonathan/Works/Tools/capstone-5.0.1/include/ \
      -DCAPSTONE_LIBRARIES=/Users/jonathan/Works/Tools/capstone-5.0.1/libcapstone.a \
      -DCAPSTONE_VERSION_HEADER=/Users/jonathan/Works/Tools/capstone-5.0.1/include/capstone/capstone.h \
      -DCMAKE_INSTALL_PREFIX=/opt/homebrew/ \
      -DPYTHON_EXECUTABLE=/opt/homebrew/bin/python3 \
      -DPYTHON_LIBRARIES=/opt/homebrew/Cellar/python@3.11/3.11.6_1/Frameworks/Python.framework/Versions/3.11/lib/libpython3.11.dylib \
      -DPYTHON_INCLUDE_DIRS=/opt/homebrew/opt/python@3.11/Frameworks/Python.framework/Versions/3.11/include/python3.11/ \
      -DLLVM_INTERFACE=ON \
      -DCMAKE_PREFIX_PATH=/Users/jonathan/Works/Tools/llvm-project-14.0.0/build \
      -DBITWUZLA_INTERFACE=ON \
      -DBITWUZLA_INCLUDE_DIRS=/usr/local/include/ \
      -DBITWUZLA_LIBRARIES="/usr/local/lib/libbitwuzla.dylib" \
      -DBOOST_INTERFACE=OFF \
      ..
patrafter1999 commented 4 months ago

Thanks a lot for your reply, @JonathanSalwan.

Everything went well until I've got that error message with 'import triton' :(. I use anaconda env with python 3.10. I had trouble with LLVM and BITWULZA, so I tried without them.

I will give it another go with your cmake. Thanks.

Cheers,

JonathanSalwan commented 4 months ago

SystemError: initialization of triton did not return an extension module

From experiences, it can comes from:

patrafter1999 commented 4 months ago

That's interesting.

I can dig in more on the first point. What do you suggest me to check or correct for your second point on libtriton.dylib?

JonathanSalwan commented 4 months ago

Make sure you have only one libtriton installed on your system.


$ sudo /usr/libexec/locate.updatedb

$ locate triton.so
/Users/jonathan/Works/Tools/Triton/build/src/libtriton/triton.so
/opt/homebrew/lib/python3.11/site-packages/triton.so

$ otool -L /opt/homebrew/lib/python3.11/site-packages/triton.so
/opt/homebrew/lib/python3.11/site-packages/triton.so:
    @rpath/libtriton.dylib (compatibility version 0.0.0, current version 0.0.0)
    /opt/homebrew/opt/z3/lib/libz3.4.12.dylib (compatibility version 4.12.0, current version 4.12.2)
    @rpath/libLTO.dylib (compatibility version 1.0.0, current version 14.0.0)
    @rpath/libRemarks.dylib (compatibility version 1.0.0, current version 14.0.0)
    /usr/local/lib/libbitwuzla.0.dylib (compatibility version 0.0.0, current version 0.0.0)
    /opt/homebrew/opt/python@3.11/Frameworks/Python.framework/Versions/3.11/Python (compatibility version 3.11.0, current version 3.11.0)
    /usr/lib/libedit.3.dylib (compatibility version 2.0.0, current version 3.0.0)
    /usr/lib/libxml2.2.dylib (compatibility version 10.0.0, current version 10.9.0)
    /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1336.61.1)
    /usr/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.12)
    /usr/lib/libncurses.5.4.dylib (compatibility version 5.4.0, current version 5.4.0)
    /usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 1600.157.0)
patrafter1999 commented 4 months ago

I am not familiar with all those tools, but I will figure it out. Thanks heaps, @JonathanSalwan !