Closed HyacinthJingjing closed 1 year ago
From your posted logs
// Version of the compiler, e.g., 5.4.0
static constexpr const char *kGCCVersion = "7.3.1";
...
../../../lib/libk2fsa.so: undefined reference to std::__throw_out_of_range_fmt(char const*, ...)@GLIBCXX_3.4.20' ../../../lib/libk2fsa.so: undefined reference to std::runtime_error::runtime_error(char const)@GLIBCXX_3.4.21'
...
Exception: Failed to build k2
gcc version 8.3.1 20190311 (Red Hat 8.3.1-3) (GCC)
It looks to me there are two different versions of GCC, i.e., GCC 7.3.1 and GCC 8.3.1.
Please follow the following doc https://k2-fsa.github.io/k2/installation/faqs.html#error-c-14-or-later-compatible-compiler-is-required-to-use-aten to setup the correct environment variables so that cmake uses the correct gcc to compile k2.
By the way, instead of compiling k2 from source, I suggest that you use the following method https://k2-fsa.github.io/k2/installation/from_wheels.html to install k2.
It is much much faster and there is no chance that you get an error with this method.
Thank you for reply. I have tried the install from the wheel "k2-1.24.3.dev20230524+cuda11.3.torch1.11.0-cp39-cp39-linux_x86_64.whl", but I encounter the error "/lib64/libm.so.6: version `GLIBC_2.27' not found" when I import k2 from python.Maybe the wheel is compiled on ubuntu,my stystem is linux?
I have tried the install from the wheel "k2-1.24.3.dev20230524+cuda11.3.torch1.11.0-cp39-cp39-linux_x86_64.whl",
Please try a wheel built in July or after that.
The wheel built in May is not a manylinux wheel.
I have tried the install from the wheel "k2-1.24.3.dev20230524+cuda11.3.torch1.11.0-cp39-cp39-linux_x86_64.whl"
Please follow https://k2-fsa.github.io/k2/installation/from_wheels.html#linux-cuda-example
Since you want to use cuda11.3+torch1.11.0, please use
pip install k2==1.24.3.dev20230725+cuda11.3.torch1.11.0 -f https://k2-fsa.github.io/k2/cuda.html
@HyacinthJingjing
Have you fixed it?
Thank you. I have successfully installed k2.
running install /opt/conda/envs/torch1.11/lib/python3.9/site-packages/setuptools/_distutils/cmd.py:66: SetuptoolsDeprecationWarning: setup.py install is deprecated. !!
!! self.initialize_options() /opt/conda/envs/torch1.11/lib/python3.9/site-packages/setuptools/_distutils/cmd.py:66: EasyInstallDeprecationWarning: easy_install command is deprecated. !!
!! self.initialize_options() running bdist_egg running egg_info writing k2.egg-info/PKG-INFO writing dependency_links to k2.egg-info/dependency_links.txt writing requirements to k2.egg-info/requires.txt writing top-level names to k2.egg-info/top_level.txt reading manifest file 'k2.egg-info/SOURCES.txt' reading manifest template 'MANIFEST.in' adding license file 'LICENSE' writing manifest file 'k2.egg-info/SOURCES.txt' installing library code to build/bdist.linux-x86_64/egg running install_lib running build_py copying k2/python/k2/init.py -> build/lib.linux-x86_64-cpython-39/k2 running build_ext cmake_path: /usr/local/cmake/bin/cmake Setting PYTHON_EXECUTABLE to /opt/conda/envs/torch1.11/bin/python3 build command is:
-- CMAKE_VERSION: 3.20.20210602-g2104cfe-dirty -- Enabled languages: CXX;CUDA -- K2_OS: -- C++ Standard version: 14 -- Autodetected CUDA architecture(s): 7.0 7.0 7.0 7.0 7.0 7.0 7.0 7.0 -- K2_COMPUTE_ARCH_FLAGS: -gencode;arch=compute_70,code=sm_70 -- K2_COMPUTE_ARCH_CANDIDATES 35;50;60;61;70;75;80;86 -- Skipping arch 35 -- Skipping arch 50 -- Skipping arch 60 -- Skipping arch 61 -- Adding arch 70 -- Skipping arch 75 -- Skipping arch 80 -- Skipping arch 86 -- K2_COMPUTE_ARCHS: 70 -- Found Valgrind: /opt/rh/devtoolset-8/root/usr/bin
-- Found Valgrind: /opt/rh/devtoolset-8/root/usr/bin/valgrind -- To check memory, run
ctest -R <NAME> -D ExperimentalMemCheck
-- Downloading pybind11 from https://github.com/pybind/pybind11/archive/5bc0943ed96836f46489f53961f6c438d2935357.zip -- pybind11 is downloaded to /opt/k2/build/temp.linux-x86_64-cpython-39/_deps/pybind11-src -- pybind11 v2.11.0 dev1 -- Python executable: /opt/conda/envs/torch1.11/bin/python3 -- Caffe2: CUDA detected: 11.3 -- Caffe2: CUDA nvcc is: /usr/local/cuda/bin/nvcc -- Caffe2: CUDA toolkit directory: /usr/local/cuda -- Caffe2: Header version is: 11.3 -- Found cuDNN: v8.2.0 (include: /usr/local/cuda/include, library: /usr/local/cuda/lib64/libcudnn.so) -- /usr/local/cuda/lib64/libnvrtc.so shorthash is 1ea278b5 -- Autodetected CUDA architecture(s): 7.0 7.0 7.0 7.0 7.0 7.0 7.0 7.0 -- Added CUDA NVCC flags for: -gencode;arch=compute_70,code=sm_70 CMake Warning at /opt/conda/envs/torch1.11/lib/python3.9/site-packages/torch/share/cmake/Torch/TorchConfig.cmake:22 (message): static library kineto_LIBRARY-NOTFOUND not found. Call Stack (most recent call first): /opt/conda/envs/torch1.11/lib/python3.9/site-packages/torch/share/cmake/Torch/TorchConfig.cmake:127 (append_torchlib_if_found) cmake/torch.cmake:11 (find_package) CMakeLists.txt:306 (include)-- K2_TORCH_VERSION: 1.11 -- PyTorch version: 1.11.0+cu113 -- PyTorch cuda version: 11.3 -- Generated /opt/k2/build/temp.linux-x86_64-cpython-39/torch_version.py -- Downloading moderngpu from https://github.com/moderngpu/moderngpu/archive/8ec9ac0de8672de7217d014917eedec5317f75f3.zip -- moderngpu is downloaded to /opt/k2/build/temp.linux-x86_64-cpython-39/_deps/moderngpu-src -- CMAKE_CUDA_FLAGS: -Wno-deprecated-gpu-targets -lineinfo --expt-extended-lambda -use_fast_math -Xptxas=-w --expt-extended-lambda -gencode arch=compute_70,code=sm_70 -DONNX_NAMESPACE=onnx_c2 -gencode arch=compute_70,code=sm_70 -Xcudafe --diag_suppress=cc_clobber_ignored,--diag_suppress=integer_sign_change,--diag_suppress=useless_using_declaration,--diag_suppress=set_but_not_used,--diag_suppress=field_without_dll_interface,--diag_suppress=base_class_has_different_dll_interface,--diag_suppress=dll_interface_conflict_none_assumed,--diag_suppress=dll_interface_conflict_dllexport_assumed,--diag_suppress=implicit_return_from_non_void_function,--diag_suppress=unsigned_compare_with_zero,--diag_suppress=declared_but_not_referenced,--diag_suppress=bad_friend_decl --expt-relaxed-constexpr --expt-extended-lambda -D_GLIBCXX_USE_CXX11_ABI=0 --compiler-options -Wall --compiler-options -Wno-strict-overflow --compiler-options -Wno-unknown-pragmas -- CMAKE_CXX_FLAGS: -D_GLIBCXX_USE_CXX11_ABI=0 -Wno-unused-variable -Wno-strict-overflow -- CMAKE_CUDA_FLAGS: -Wno-deprecated-gpu-targets -lineinfo --expt-extended-lambda -use_fast_math -Xptxas=-w --expt-extended-lambda -gencode arch=compute_70,code=sm_70 -DONNX_NAMESPACE=onnx_c2 -gencode arch=compute_70,code=sm_70 -Xcudafe --diag_suppress=cc_clobber_ignored,--diag_suppress=integer_sign_change,--diag_suppress=useless_using_declaration,--diag_suppress=set_but_not_used,--diag_suppress=field_without_dll_interface,--diag_suppress=base_class_has_different_dll_interface,--diag_suppress=dll_interface_conflict_none_assumed,--diag_suppress=dll_interface_conflict_dllexport_assumed,--diag_suppress=implicit_return_from_non_void_function,--diag_suppress=unsigned_compare_with_zero,--diag_suppress=declared_but_not_referenced,--diag_suppress=bad_friend_decl --expt-relaxed-constexpr --expt-extended-lambda -D_GLIBCXX_USE_CXX11_ABI=0 --compiler-options -Wall --compiler-options -Wno-strict-overflow --compiler-options -Wno-unknown-pragmas -- Generated /opt/k2/build/temp.linux-x86_64-cpython-39/k2/csrc/version.h -- PYTHON_LIBRARY: /opt/conda/envs/torch1.11/lib/libpython3.9.so -- Including k2/torch. K2_TORCH_VERSION is 1.11 -- Downloading kaldifeat from https://github.com/csukuangfj/kaldifeat/archive/refs/tags/v1.20.tar.gz -- kaldifeat is downloaded to /opt/k2/build/temp.linux-x86_64-cpython-39/_deps/kaldifeat-src -- kaldifeat's binary dir is /opt/k2/build/temp.linux-x86_64-cpython-39/_deps/kaldifeat-build -- C++ Standard version: 14 -- Python executable: /opt/conda/envs/torch1.11/bin/python3 CMake Warning at /opt/conda/envs/torch1.11/lib/python3.9/site-packages/torch/share/cmake/Torch/TorchConfig.cmake:22 (message): static library kineto_LIBRARY-NOTFOUND not found. Call Stack (most recent call first): /opt/conda/envs/torch1.11/lib/python3.9/site-packages/torch/share/cmake/Torch/TorchConfig.cmake:127 (append_torchlib_if_found) cmake/torch.cmake:11 (find_package) build/temp.linux-x86_64-cpython-39/_deps/kaldifeat-src/CMakeLists.txt:55 (include)
-- K2_TORCH_VERSION: 1.11 -- PyTorch version: 1.11.0+cu113 -- PyTorch cuda version: 11.3 -- CMAKE_CXX_FLAGS: -D_GLIBCXX_USE_CXX11_ABI=0 -Wno-unused-variable -Wno-strict-overflow -D_GLIBCXX_USE_CXX11_ABI=0 -- CMAKE_INSTALL_PREFIX: /opt/k2/build/lib.linux-x86_64-cpython-39/k2 -- All headers: /opt/k2/build/temp.linux-x86_64-cpython-39/_deps/kaldifeat-src/kaldifeat/csrc/feature-common-inl.h;/opt/k2/build/temp.linux-x86_64-cpython-39/_deps/kaldifeat-src/kaldifeat/csrc/feature-common.h;/opt/k2/build/temp.linux-x86_64-cpython-39/_deps/kaldifeat-src/kaldifeat/csrc/feature-fbank.h;/opt/k2/build/temp.linux-x86_64-cpython-39/_deps/kaldifeat-src/kaldifeat/csrc/feature-functions.h;/opt/k2/build/temp.linux-x86_64-cpython-39/_deps/kaldifeat-src/kaldifeat/csrc/feature-mfcc.h;/opt/k2/build/temp.linux-x86_64-cpython-39/_deps/kaldifeat-src/kaldifeat/csrc/feature-plp.h;/opt/k2/build/temp.linux-x86_64-cpython-39/_deps/kaldifeat-src/kaldifeat/csrc/feature-spectrogram.h;/opt/k2/build/temp.linux-x86_64-cpython-39/_deps/kaldifeat-src/kaldifeat/csrc/feature-window.h;/opt/k2/build/temp.linux-x86_64-cpython-39/_deps/kaldifeat-src/kaldifeat/csrc/log.h;/opt/k2/build/temp.linux-x86_64-cpython-39/_deps/kaldifeat-src/kaldifeat/csrc/matrix-functions.h;/opt/k2/build/temp.linux-x86_64-cpython-39/_deps/kaldifeat-src/kaldifeat/csrc/mel-computations.h;/opt/k2/build/temp.linux-x86_64-cpython-39/_deps/kaldifeat-src/kaldifeat/csrc/online-feature-itf.h;/opt/k2/build/temp.linux-x86_64-cpython-39/_deps/kaldifeat-src/kaldifeat/csrc/online-feature.h;/opt/k2/build/temp.linux-x86_64-cpython-39/_deps/kaldifeat-src/kaldifeat/csrc/pitch-functions.h -- Configuring done -- Generating done -- Build files have been written to: /opt/k2/build/temp.linux-x86_64-cpython-39 /**
ifndef K2_CSRC_VERSIONH
define K2_CSRC_VERSIONH
namespace k2 {
// Version of k2 in "major.minor.patch" format static constexpr const char *kVersion = "1.24.3";
// The commit used to build k2 static constexpr const char *kGitSha1 = "b835546b6005d243865e0acc3d29bd9c51670b1e";
// Date of the commit used to build k2 static constexpr const char *kGitDate = "Wed Jul 26 19:29:59 2023";
// Version of CUDA used to build k2. // Its format is "major.minor", e.g., 10.1 static constexpr const char *kCudaVersion = "11.3";
// cuDNN version, e.g., 8.0.2 static constexpr const char *kCudnnVersion = "8.2.0";
// clang-format off // Version of Python used to build k2 Python bindings. static constexpr const char *kPythonVersion = "3.9";
// clang-format on
// CMake build type, e.g., Release or Debug. static constexpr const char *kBuildType = "Release";
// The operating system that is used to build k2, e.g., Ubuntu 16.04 LTS static constexpr const char *kOS = R"os()os";
// e.g., 3.18.0 static constexpr const char *kCMakeVersion = "3.20.20210602-g2104cfe-dirty";
// Version of the compiler, e.g., 5.4.0 static constexpr const char *kGCCVersion = "7.3.1";
// CUDA flags used to compile k2 static constexpr const char *kCMakeCudaFlags = R"cuda_flags( -Wno-deprecated-gpu-targets -lineinfo --expt-extended-lambda -use_fast_math -Xptxas=-w --expt-extended-lambda -gencode arch=compute_70,code=sm_70 -DONNX_NAMESPACE=onnx_c2 -gencode arch=compute_70,code=sm_70 -Xcudafe --diag_suppress=cc_clobber_ignored,--diag_suppress=integer_sign_change,--diag_suppress=useless_using_declaration,--diag_suppress=set_but_not_used,--diag_suppress=field_without_dll_interface,--diag_suppress=base_class_has_different_dll_interface,--diag_suppress=dll_interface_conflict_none_assumed,--diag_suppress=dll_interface_conflict_dllexport_assumed,--diag_suppress=implicit_return_from_non_void_function,--diag_suppress=unsigned_compare_with_zero,--diag_suppress=declared_but_not_referenced,--diag_suppress=bad_friend_decl --expt-relaxed-constexpr --expt-extended-lambda -D_GLIBCXX_USE_CXX11_ABI=0 --compiler-options -Wall --compiler-options -Wno-strict-overflow --compiler-options -Wno-unknown-pragmas )cuda_flags";
// CXX flags used to compile k2 static constexpr const char *kCMakeCxxFlags = R"cxx_flags( -D_GLIBCXX_USE_CXX11_ABI=0 -Wno-unused-variable -Wno-strict-overflow )cxx_flags";
// Which PyTorch version k2 is using, e.g., 1.6.0+cu101 static constexpr const char *kTorchVersion = "1.11.0+cu113";
// Which CUDA version PyTorch is using, e.g., 10.1 static constexpr const char *kTorchCudaVersion = "11.3";
ifndef K2_WITH_CUDA
define K2_WITH_CUDA
endif
ifdef K2_WITH_CUDA
static constexpr bool kWithCuda = true;
else
static constexpr bool kWithCuda = false;
endif
// Indicate whether NVTX is enabled or not
ifndef K2_ENABLE_NVTX
define K2_ENABLE_NVTX
endif
ifdef K2_ENABLE_NVTX
static constexpr bool kEnableNvtx = true;
else
static constexpr bool kEnableNvtx = false;
endif
} // namespace k2
endif // K2_CSRC_VERSIONH
[ 1%] Building CUDA object k2/csrc/CMakeFiles/k2_log.dir/log.cu.o Consolidate compiler generated dependencies of target kaldifeat_core [ 9%] Built target kaldifeat_core [ 9%] Building CXX object k2/torch/csrc/CMakeFiles/k2_fbank.dir/features.cc.o [ 10%] Linking CUDA shared library ../../lib/libk2_log.so [ 10%] Built target k2_log [ 12%] Building CXX object k2/csrc/host/CMakeFiles/fsa.dir/arcsort.cc.o [ 13%] Building CXX object k2/csrc/host/CMakeFiles/fsa.dir/aux_labels.cc.o [ 13%] Building CXX object k2/csrc/host/CMakeFiles/fsa.dir/connect.cc.o [ 14%] Building CXX object k2/csrc/host/CMakeFiles/fsa.dir/determinize.cc.o [ 15%] Building CXX object k2/csrc/host/CMakeFiles/fsa.dir/determinize_pruned.cc.o [ 16%] Building CXX object k2/csrc/host/CMakeFiles/fsa.dir/fsa.cc.o [ 16%] Building CXX object k2/csrc/host/CMakeFiles/fsa.dir/fsa_equivalent.cc.o [ 17%] Building CXX object k2/csrc/host/CMakeFiles/fsa.dir/fsa_renderer.cc.o [ 18%] Building CXX object k2/csrc/host/CMakeFiles/fsa.dir/fsa_util.cc.o [ 19%] Building CXX object k2/csrc/host/CMakeFiles/fsa.dir/intersect.cc.o [ 19%] Building CXX object k2/csrc/host/CMakeFiles/fsa.dir/properties.cc.o [ 20%] Building CXX object k2/csrc/host/CMakeFiles/fsa.dir/rmepsilon_pruned.cc.o [ 21%] Building CXX object k2/csrc/host/CMakeFiles/fsa.dir/topsort.cc.o [ 23%] Building CXX object k2/csrc/host/CMakeFiles/fsa.dir/util.cc.o [ 23%] Building CXX object k2/csrc/host/CMakeFiles/fsa.dir/weights.cc.o [ 24%] Linking CXX shared library ../../../lib/libk2fsa.so [ 24%] Built target fsa [ 27%] Building CUDA object k2/csrc/CMakeFiles/context.dir/array_ops.cu.o [ 28%] Building CUDA object k2/csrc/CMakeFiles/context.dir/algorithms.cu.o [ 28%] Building CUDA object k2/csrc/CMakeFiles/context.dir/connect.cu.o [ 28%] Building CUDA object k2/csrc/CMakeFiles/context.dir/array_of_ragged.cu.o [ 28%] Building CUDA object k2/csrc/CMakeFiles/context.dir/context.cu.o [ 29%] Building CUDA object k2/csrc/CMakeFiles/context.dir/dtype.cu.o [ 29%] Building CUDA object k2/csrc/CMakeFiles/context.dir/fsa.cu.o [ 30%] Building CUDA object k2/csrc/CMakeFiles/context.dir/fsa_algo.cu.o [ 31%] Linking CXX shared library ../../../lib/libk2_fbank.so [ 32%] Building CUDA object k2/csrc/CMakeFiles/context.dir/fsa_utils.cu.o [ 32%] Built target k2_fbank [ 34%] Building CUDA object k2/csrc/CMakeFiles/context.dir/hash.cu.o [ 34%] Building CUDA object k2/csrc/CMakeFiles/context.dir/host_shim.cu.o [ 35%] Building CUDA object k2/csrc/CMakeFiles/context.dir/intersect.cu.o [ 36%] Building CUDA object k2/csrc/CMakeFiles/context.dir/intersect_dense.cu.o [ 37%] Building CUDA object k2/csrc/CMakeFiles/context.dir/intersect_dense_pruned.cu.o [ 37%] Building CUDA object k2/csrc/CMakeFiles/context.dir/math.cu.o [ 38%] Building CUDA object k2/csrc/CMakeFiles/context.dir/moderngpu_allocator.cu.o [ 39%] Building CUDA object k2/csrc/CMakeFiles/context.dir/pinned_context.cu.o [ 40%] Building CUDA object k2/csrc/CMakeFiles/context.dir/ragged.cu.o [ 40%] Building CUDA object k2/csrc/CMakeFiles/context.dir/ragged_ops.cu.o [ 41%] Building CUDA object k2/csrc/CMakeFiles/context.dir/ragged_utils.cu.o [ 42%] Building CUDA object k2/csrc/CMakeFiles/context.dir/rand.cu.o [ 43%] Building CUDA object k2/csrc/CMakeFiles/context.dir/reverse.cu.o [ 45%] Building CUDA object k2/csrc/CMakeFiles/context.dir/rm_epsilon.cu.o [ 45%] Building CUDA object k2/csrc/CMakeFiles/context.dir/rnnt_decode.cu.o [ 46%] Building CUDA object k2/csrc/CMakeFiles/context.dir/tensor.cu.o [ 47%] Building CUDA object k2/csrc/CMakeFiles/context.dir/tensor_ops.cu.o [ 48%] Building CUDA object k2/csrc/CMakeFiles/context.dir/thread_pool.cu.o [ 48%] Building CUDA object k2/csrc/CMakeFiles/context.dir/timer.cu.o [ 49%] Building CUDA object k2/csrc/CMakeFiles/context.dir/top_sort.cu.o [ 50%] Building CUDA object k2/csrc/CMakeFiles/context.dir/torch_util.cu.o [ 51%] Building CUDA object k2/csrc/CMakeFiles/context.dir/utils.cu.o [ 51%] Building CUDA object k2/csrc/CMakeFiles/context.dir/nbest.cu.o [ 52%] Building CUDA object k2/csrc/CMakeFiles/context.dir/pytorch_context.cu.o /opt/k2/k2/csrc/ragged_ops_inl.h: In function ‘k2::Renumbering k2::PruneRaggedAxis1(k2::Ragged&, T, int32_t) [with T = double]’: /opt/k2/k2/csrc/ragged_ops_inl.h:870:16: warning: ‘order_map_data’ may be used uninitialized in this function [-Wmaybe-uninitialized] const int32_t *order_map_data; ^, std::allocator<std::vector > > k2::ModifiedBeamSearch(const torch::jit::Module&, const at::Tensor&, const at::Tensor&, int32_t)’:
/opt/k2/k2/torch/csrc/beam_search.cu:272:20: warning: comparison of integer expressions of different signedness: ‘int32_t’ {aka ‘int’} and ‘std::vector::size_type’ {aka ‘long unsigned int’} [-Wsign-compare]
if (cur_batch_size < cur.size()) {
~~~~~ [ 53%] Building CUDA object k2/csrc/CMakeFiles/context.dir/cudpp/cudpp.cu.o [ 54%] Linking CUDA device code CMakeFiles/context.dir/cmake_device_link.o [ 54%] Linking CUDA shared library ../../lib/libk2context.so [ 54%] Built target context [ 54%] Building CUDA object k2/python/csrc/CMakeFiles/_k2.dir/k2.cu.o [ 59%] Building CUDA object k2/python/csrc/CMakeFiles/_k2.dir/version.cu.o [ 59%] Building CUDA object k2/torch/csrc/CMakeFiles/k2_torch.dir/decode.cu.o [ 58%] Building CUDA object k2/python/csrc/CMakeFiles/_k2.dir/torch.cu.o [ 59%] Building CUDA object k2/torch/csrc/CMakeFiles/k2_torch.dir/beam_search.cu.o [ 59%] Building CUDA object k2/python/csrc/CMakeFiles/_k2.dir/torch/arc.cu.o [ 59%] Building CUDA object k2/python/csrc/CMakeFiles/_k2.dir/torch/fsa.cu.o /opt/k2/k2/torch/csrc/beam_search.cu: In function ‘std::vector<std::vector