taichi-dev / taichi

Productive, portable, and performant GPU programming in Python.
https://taichi-lang.org
Apache License 2.0
25.35k stars 2.27k forks source link

Can't build with the doc. #3183

Open Renkai opened 2 years ago

Renkai commented 2 years ago

When I follow this doc to build Taichi https://docs.taichi.graphics/lang/articles/contribution/dev_install#installing-dependencies under macOS 11.6 and pre-prebuilt llvm 10.0.0

(taichi-dev) ➜  taichi git:(master) export TAICHI_CMAKE_ARGS="-DTI_WITH_CUDA=OFF -DTI_WITH_OPENGL=OFF -DTI_WITH_CC=OFF"
(taichi-dev) ➜  taichi git:(master) python3 setup.py develop --user  # Optionally add DEBUG=1 to keep debug information.
[]
['taichi', 'taichi.misc', 'taichi.ui', 'taichi.tools', 'taichi.core', 'taichi.linalg', 'taichi.tests', 'taichi.snode', 'taichi.lang', 'taichi.type', 'taichi.profiler', 'taichi.aot', 'taichi.lang.ast']
running develop
running egg_info
writing python/taichi.egg-info/PKG-INFO
writing dependency_links to python/taichi.egg-info/dependency_links.txt
writing entry points to python/taichi.egg-info/entry_points.txt
writing requirements to python/taichi.egg-info/requires.txt
writing top-level names to python/taichi.egg-info/top_level.txt
reading manifest file 'python/taichi.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'python/taichi/*.md'
warning: no files found matching 'python/taichi/lib/*.so'
warning: no files found matching 'python/taichi/lib/*.pyd'
warning: no files found matching 'python/taichi/lib/*.bc'
warning: no previously-included files matching '*.pyc' found anywhere in distribution
warning: no previously-included files matching '*.pyo' found anywhere in distribution
warning: no previously-included files matching 'ffmpeg' found anywhere in distribution
adding license file 'LICENSE'
writing manifest file 'python/taichi.egg-info/SOURCES.txt'
running build_ext
---------- Running CMake prepare ----------------------------------------
Taichi Version 0.8.2
       Commit 581fdac86fa3b337fee5cfec45c2a643426b20ca
Using /Users/renkaige/miniconda/envs/taichi-dev/bin/python3 as python executable.
Python 3.8.12
    version: 3.8
    include: /Users/renkaige/miniconda/envs/taichi-dev/include/python3.8
    library: /Users/renkaige/miniconda/envs/taichi-dev/lib/libpython3.8.dylib
    gitpython version: 3.1.24
    numpy include: /Users/renkaige/miniconda/envs/taichi-dev/lib/python3.8/site-packages/numpy/core/include
    pybind11 include: /Users/renkaige/miniconda/envs/taichi-dev/lib/python3.8/site-packages/pybind11/include;/Users/renkaige/miniconda/envs/taichi-dev/lib/python3.8/site-packages/pybind11/include
Using C++ compiler: /Library/Developer/CommandLineTools/usr/bin/c++
Clang compiler detected. Using std=c++17.
Building for processor x86_64
Setting -march=nehalem for x86_64 processors
Using float32 (single) precision as real
-- Found LLVM 10.0.0
-- Using LLVMConfig.cmake in: /usr/local/taichi-llvm-10.0.0-macos/lib/cmake/llvm
LLVM include dirs /usr/local/taichi-llvm-10.0.0-macos/include
LLVM library dirs /usr/local/taichi-llvm-10.0.0-macos/lib
-- TI_WITH_CUDA_TOOLKIT = OFF
-- SPIRV-Cross: Finding Git version for SPIRV-Cross.
-- SPIRV-Cross: Git hash: 97a438d2
SPIRV-Cross:  Testing will be disabled for SPIRV-Cross. Could not find glslang or SPIRV-Tools build under external/. To enable testing, run ./checkout_glslang_spirv_tools.sh and ./build_glslang_spirv_tools.sh first.
PYTHON_LIBRARIES: /Users/renkaige/miniconda/envs/taichi-dev/lib/libpython3.8.dylib
C++ Flags:  -DTI_ISE_NONE -std=c++17 -fsized-deallocation -Wall  -march=nehalem -DTI_ARCH_x64 -DTI_PASS_EXCEPTION_TO_PYTHON -DTI_INCLUDED
Build type: Release
python=/Users/renkaige/miniconda/envs/taichi-dev/bin/python3
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/renkaige/renkai-lab/taichi/build
---------- Building extensions ----------------------------------------
Consolidate compiler generated dependencies of target spirv-cross-core
Consolidate compiler generated dependencies of target spirv-cross-reflect
[  3%] Built target spirv-cross-core
[  4%] Built target spirv-cross-reflect
Consolidate compiler generated dependencies of target spirv-cross-glsl
Consolidate compiler generated dependencies of target spirv-cross-util
[  5%] Built target spirv-cross-glsl
[  6%] Built target spirv-cross-util
Consolidate compiler generated dependencies of target spirv-cross-msl
Consolidate compiler generated dependencies of target spirv-cross-hlsl
Consolidate compiler generated dependencies of target spirv-cross-cpp
[  7%] Built target spirv-cross-msl
[  8%] Built target spirv-cross-hlsl
[  9%] Built target spirv-cross-cpp
Consolidate compiler generated dependencies of target spirv-cross
Consolidate compiler generated dependencies of target spirv-cross-c
Building commit 581fdac86fa3b337fee5cfec45c2a643426b20ca
[  9%] Building CXX object external/SPIRV-Cross/CMakeFiles/spirv-cross.dir/main.cpp.o
[ 10%] Building CXX object external/SPIRV-Cross/CMakeFiles/spirv-cross-c.dir/spirv_cross_c.cpp.o
[ 10%] Built target generate_commit_hash
/usr/local/taichi-llvm-10.0.0-macos/bin/llvm-as: runtime_x64.ll:2393:74: error: expected ')' at end of argument list
define void @Element_get_pcoord(%struct.PhysicalCoordinates* noalias sret(%struct.PhysicalCoordinates) align 4 %0, %struct.Element* %1) #0 {
                                                                         ^
make[2]: *** [CMakeFiles/generate_llvm_runtime_x64] Error 1
make[1]: *** [CMakeFiles/generate_llvm_runtime_x64.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
[ 10%] Linking CXX static library libspirv-cross-c.a
[ 10%] Built target spirv-cross-c
[ 10%] Linking CXX executable ../../spirv-cross
[ 10%] Built target spirv-cross
make: *** [all] Error 2
Traceback (most recent call last):
  File "setup.py", line 223, in <module>
    setup(name=project_name,
  File "/Users/renkaige/miniconda/envs/taichi-dev/lib/python3.8/site-packages/setuptools/__init__.py", line 153, in setup
    return distutils.core.setup(**attrs)
  File "/Users/renkaige/miniconda/envs/taichi-dev/lib/python3.8/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/Users/renkaige/miniconda/envs/taichi-dev/lib/python3.8/distutils/dist.py", line 966, in run_commands
    self.run_command(cmd)
  File "/Users/renkaige/miniconda/envs/taichi-dev/lib/python3.8/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/Users/renkaige/miniconda/envs/taichi-dev/lib/python3.8/site-packages/setuptools/command/develop.py", line 34, in run
    self.install_for_development()
  File "/Users/renkaige/miniconda/envs/taichi-dev/lib/python3.8/site-packages/setuptools/command/develop.py", line 114, in install_for_development
    self.run_command('build_ext')
  File "/Users/renkaige/miniconda/envs/taichi-dev/lib/python3.8/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/Users/renkaige/miniconda/envs/taichi-dev/lib/python3.8/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "setup.py", line 159, in run
    subprocess.check_call(cmake_cmd, cwd=self.build_temp)
  File "/Users/renkaige/miniconda/envs/taichi-dev/lib/python3.8/subprocess.py", line 364, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['cmake', '--build', '.', '--config', 'Release', '--', '-j16']' returned non-zero exit status 2.
k-ye commented 2 years ago

Thanks! I wonder what's your clang version? E.g. clang --version

Renkai commented 2 years ago
(taichi-dev) ➜  taichi git:(master) clang --version
Apple clang version 13.0.0 (clang-1300.0.29.3)
Target: x86_64-apple-darwin20.6.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin

I believe it's the builtin clang of macOS if my memory is not wrong.

k-ye commented 2 years ago

Yeah, this is possibly a compatibility issue between LLVM-10 (kind of old) and clang-13. Will look into it

Zheaoli commented 2 years ago

Seems clang-13 with llvm-10 on ubuntu has a different error

/usr/bin/ld: CMakeFiles/spirv-cross.dir/main.cpp.o: in function `main':
main.cpp:(.text+0xc): undefined reference to `__cxa_begin_catch'
/usr/bin/ld: main.cpp:(.text+0x38): undefined reference to `__cxa_end_catch'
/usr/bin/ld: CMakeFiles/spirv-cross.dir/main.cpp.o: in function `main_inner(int, char**)':
main.cpp:(.text+0x2bba): undefined reference to `operator new(unsigned long)'
/usr/bin/ld: main.cpp:(.text+0x2c90): undefined reference to `operator delete(void*)'
/usr/bin/ld: main.cpp:(.text+0x2d63): undefined reference to `operator delete(void*)'
/usr/bin/ld: main.cpp:(.text+0x2ff7): undefined reference to `operator new(unsigned long)'
/usr/bin/ld: main.cpp:(.text+0x314a): undefined reference to `operator delete(void*)'
/usr/bin/ld: main.cpp:(.text+0x315c): undefined reference to `operator delete(void*)'
/usr/bin/ld: main.cpp:(.text+0x3309): undefined reference to `operator delete(void*)'
/usr/bin/ld: main.cpp:(.text+0x331b): undefined reference to `operator delete(void*)'
/usr/bin/ld: main.cpp:(.text+0x337c): undefined reference to `operator delete(void*)'
/usr/bin/ld: CMakeFiles/spirv-cross.dir/main.cpp.o:main.cpp:(.text+0x3393): more undefined references to `operator delete(void*)' follow
/usr/bin/ld: CMakeFiles/spirv-cross.dir/main.cpp.o: in function `main_inner(int, char**)':
main.cpp:(.text+0x3431): undefined reference to `std::__throw_bad_alloc()'
/usr/bin/ld: main.cpp:(.text+0x343d): undefined reference to `std::__throw_length_error(char const*)'
/usr/bin/ld: main.cpp:(.text+0x34c3): undefined reference to `operator delete(void*)'
/usr/bin/ld: main.cpp:(.text+0x34e1): undefined reference to `operator delete(void*)'
/usr/bin/ld: main.cpp:(.text+0x4b25): undefined reference to `operator delete(void*)'
/usr/bin/ld: CMakeFiles/spirv-cross.dir/main.cpp.o: in function `compile_iteration[abi:cxx11](CLIArguments const&, std::vector<unsigned int, std::allocator<unsigned int> >)':
main.cpp:(.text+0x4c94): undefined reference to `operator delete(void*)'
/usr/bin/ld: main.cpp:(.text+0x4cbe): undefined reference to `operator new(unsigned long)'
/usr/bin/ld: main.cpp:(.text+0x4dbe): undefined reference to `std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_create(unsigned long&, unsigned long)'
/usr/bin/ld: main.cpp:(.text+0x4e0c): undefined reference to `operator new(unsigned long)'
/usr/bin/ld: main.cpp:(.text+0x5163): undefined reference to `operator new(unsigned long)'
/usr/bin/ld: main.cpp:(.text+0x53c9): undefined reference to `operator delete(void*)'
/usr/bin/ld: main.cpp:(.text+0x5592): undefined reference to `std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_create(unsigned long&, unsigned long)'
/usr/bin/ld: main.cpp:(.text+0x5787): undefined reference to `std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_assign(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
/usr/bin/ld: main.cpp:(.text+0x5c54): undefined reference to `operator delete(void*)'
/usr/bin/ld: main.cpp:(.text+0x5c90): undefined reference to `operator delete(void*)'
/usr/bin/ld: main.cpp:(.text+0x5cb4): undefined reference to `operator delete(void*)'
/usr/bin/ld: main.cpp:(.text+0x5ce7): undefined reference to `operator delete(void*)'
/usr/bin/ld: main.cpp:(.text+0x5e59): undefined reference to `operator delete(void*)'
/usr/bin/ld: CMakeFiles/spirv-cross.dir/main.cpp.o:main.cpp:(.text+0x5e81): more undefined references to `operator delete(void*)' follow
/usr/bin/ld: CMakeFiles/spirv-cross.dir/main.cpp.o: in function `compile_iteration[abi:cxx11](CLIArguments const&, std::vector<unsigned int, std::allocator<unsigned int> >)':
main.cpp:(.text+0x6d66): undefined reference to `operator new(unsigned long)'
/usr/bin/ld: main.cpp:(.text+0x6df5): undefined reference to `std::terminate()'
/usr/bin/ld: main.cpp:(.text+0x6e92): undefined reference to `operator delete(void*, unsigned long)'
sjwsl commented 2 years ago

This is probably because LLVM IR is not guaranteed to be forward compatible. So old llvm-as may not be able to parse the IR generated by new clang. Seems llvm 12 break the compatibility. Since clang on MacOS strongly relies on the system tool-chain, we may not be able to provide a single binary now (the binary from the LLVM official site may not work too). Also found a similar issue here.

The current recommended solution is to downgrade clang to a version no greater than 11 (10 is better) using brew/MacPorts or maintain multiple xcode tool-chain.

Will update doc and add version checks in cmake.

XYZ0901 commented 2 years ago

have the same problem with clang10 and llvm10

ubuntu@ubuntu:~/taichi$ clang --version
clang version 10.0.0-4ubuntu1 
Target: aarch64-unknown-linux-gnu
Thread model: posix
InstalledDir: /usr/bin
sjwsl commented 2 years ago

have the same problem with clang10 and llvm10

ubuntu@ubuntu:~/taichi$ clang --version
clang version 10.0.0-4ubuntu1 
Target: aarch64-unknown-linux-gnu
Thread model: posix
InstalledDir: /usr/bin

Could you provide the complete commands and errors in your building?