k2-fsa / k2

FSA/FST algorithms, differentiable, with PyTorch compatibility.
https://k2-fsa.github.io/k2
Apache License 2.0
1.1k stars 214 forks source link

Install k2 problem #1238

Closed HyacinthJingjing closed 1 year ago

HyacinthJingjing commented 1 year ago

running install /opt/conda/envs/torch1.11/lib/python3.9/site-packages/setuptools/_distutils/cmd.py:66: SetuptoolsDeprecationWarning: setup.py install is deprecated. !!

    ********************************************************************************
    Please avoid running ``setup.py`` directly.
    Instead, use pypa/build, pypa/installer or other
    standards-based tools.

    See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details.
    ********************************************************************************

!! self.initialize_options() /opt/conda/envs/torch1.11/lib/python3.9/site-packages/setuptools/_distutils/cmd.py:66: EasyInstallDeprecationWarning: easy_install command is deprecated. !!

    ********************************************************************************
    Please avoid running ``setup.py`` and ``easy_install``.
    Instead, use pypa/build, pypa/installer or other
    standards-based tools.

    See https://github.com/pypa/setuptools/issues/917 for details.
    ********************************************************************************

!! self.initialize_options() running bdist_egg running egg_info writing k2.egg-info/PKG-INFO writing dependency_links to k2.egg-info/dependency_links.txt writing requirements to k2.egg-info/requires.txt writing top-level names to k2.egg-info/top_level.txt reading manifest file 'k2.egg-info/SOURCES.txt' reading manifest template 'MANIFEST.in' adding license file 'LICENSE' writing manifest file 'k2.egg-info/SOURCES.txt' installing library code to build/bdist.linux-x86_64/egg running install_lib running build_py copying k2/python/k2/init.py -> build/lib.linux-x86_64-cpython-39/k2 running build_ext cmake_path: /usr/local/cmake/bin/cmake Setting PYTHON_EXECUTABLE to /opt/conda/envs/torch1.11/bin/python3 build command is:

            cd build/temp.linux-x86_64-cpython-39

            cmake -DCMAKE_BUILD_TYPE=Release -DPYTHON_EXECUTABLE=/opt/conda/envs/torch1.11/bin/python3 -DK2_ENABLE_BENCHMARK=OFF  -DK2_ENABLE_TESTS=OFF  -DCMAKE_INSTALL_PREFIX=/opt/k2/build/lib.linux-x86_64-cpython-39/k2  /opt/k2

            cat k2/csrc/version.h

            make -j6 install

-- CMAKE_VERSION: 3.20.20210602-g2104cfe-dirty -- Enabled languages: CXX;CUDA -- K2_OS: -- C++ Standard version: 14 -- Autodetected CUDA architecture(s): 7.0 7.0 7.0 7.0 7.0 7.0 7.0 7.0 -- K2_COMPUTE_ARCH_FLAGS: -gencode;arch=compute_70,code=sm_70 -- K2_COMPUTE_ARCH_CANDIDATES 35;50;60;61;70;75;80;86 -- Skipping arch 35 -- Skipping arch 50 -- Skipping arch 60 -- Skipping arch 61 -- Adding arch 70 -- Skipping arch 75 -- Skipping arch 80 -- Skipping arch 86 -- K2_COMPUTE_ARCHS: 70 -- Found Valgrind: /opt/rh/devtoolset-8/root/usr/bin
-- Found Valgrind: /opt/rh/devtoolset-8/root/usr/bin/valgrind -- To check memory, run ctest -R <NAME> -D ExperimentalMemCheck -- Downloading pybind11 from https://github.com/pybind/pybind11/archive/5bc0943ed96836f46489f53961f6c438d2935357.zip -- pybind11 is downloaded to /opt/k2/build/temp.linux-x86_64-cpython-39/_deps/pybind11-src -- pybind11 v2.11.0 dev1 -- Python executable: /opt/conda/envs/torch1.11/bin/python3 -- Caffe2: CUDA detected: 11.3 -- Caffe2: CUDA nvcc is: /usr/local/cuda/bin/nvcc -- Caffe2: CUDA toolkit directory: /usr/local/cuda -- Caffe2: Header version is: 11.3 -- Found cuDNN: v8.2.0 (include: /usr/local/cuda/include, library: /usr/local/cuda/lib64/libcudnn.so) -- /usr/local/cuda/lib64/libnvrtc.so shorthash is 1ea278b5 -- Autodetected CUDA architecture(s): 7.0 7.0 7.0 7.0 7.0 7.0 7.0 7.0 -- Added CUDA NVCC flags for: -gencode;arch=compute_70,code=sm_70 CMake Warning at /opt/conda/envs/torch1.11/lib/python3.9/site-packages/torch/share/cmake/Torch/TorchConfig.cmake:22 (message): static library kineto_LIBRARY-NOTFOUND not found. Call Stack (most recent call first): /opt/conda/envs/torch1.11/lib/python3.9/site-packages/torch/share/cmake/Torch/TorchConfig.cmake:127 (append_torchlib_if_found) cmake/torch.cmake:11 (find_package) CMakeLists.txt:306 (include)

-- K2_TORCH_VERSION: 1.11 -- PyTorch version: 1.11.0+cu113 -- PyTorch cuda version: 11.3 -- Generated /opt/k2/build/temp.linux-x86_64-cpython-39/torch_version.py -- Downloading moderngpu from https://github.com/moderngpu/moderngpu/archive/8ec9ac0de8672de7217d014917eedec5317f75f3.zip -- moderngpu is downloaded to /opt/k2/build/temp.linux-x86_64-cpython-39/_deps/moderngpu-src -- CMAKE_CUDA_FLAGS: -Wno-deprecated-gpu-targets -lineinfo --expt-extended-lambda -use_fast_math -Xptxas=-w --expt-extended-lambda -gencode arch=compute_70,code=sm_70 -DONNX_NAMESPACE=onnx_c2 -gencode arch=compute_70,code=sm_70 -Xcudafe --diag_suppress=cc_clobber_ignored,--diag_suppress=integer_sign_change,--diag_suppress=useless_using_declaration,--diag_suppress=set_but_not_used,--diag_suppress=field_without_dll_interface,--diag_suppress=base_class_has_different_dll_interface,--diag_suppress=dll_interface_conflict_none_assumed,--diag_suppress=dll_interface_conflict_dllexport_assumed,--diag_suppress=implicit_return_from_non_void_function,--diag_suppress=unsigned_compare_with_zero,--diag_suppress=declared_but_not_referenced,--diag_suppress=bad_friend_decl --expt-relaxed-constexpr --expt-extended-lambda -D_GLIBCXX_USE_CXX11_ABI=0 --compiler-options -Wall --compiler-options -Wno-strict-overflow --compiler-options -Wno-unknown-pragmas -- CMAKE_CXX_FLAGS: -D_GLIBCXX_USE_CXX11_ABI=0 -Wno-unused-variable -Wno-strict-overflow -- CMAKE_CUDA_FLAGS: -Wno-deprecated-gpu-targets -lineinfo --expt-extended-lambda -use_fast_math -Xptxas=-w --expt-extended-lambda -gencode arch=compute_70,code=sm_70 -DONNX_NAMESPACE=onnx_c2 -gencode arch=compute_70,code=sm_70 -Xcudafe --diag_suppress=cc_clobber_ignored,--diag_suppress=integer_sign_change,--diag_suppress=useless_using_declaration,--diag_suppress=set_but_not_used,--diag_suppress=field_without_dll_interface,--diag_suppress=base_class_has_different_dll_interface,--diag_suppress=dll_interface_conflict_none_assumed,--diag_suppress=dll_interface_conflict_dllexport_assumed,--diag_suppress=implicit_return_from_non_void_function,--diag_suppress=unsigned_compare_with_zero,--diag_suppress=declared_but_not_referenced,--diag_suppress=bad_friend_decl --expt-relaxed-constexpr --expt-extended-lambda -D_GLIBCXX_USE_CXX11_ABI=0 --compiler-options -Wall --compiler-options -Wno-strict-overflow --compiler-options -Wno-unknown-pragmas -- Generated /opt/k2/build/temp.linux-x86_64-cpython-39/k2/csrc/version.h -- PYTHON_LIBRARY: /opt/conda/envs/torch1.11/lib/libpython3.9.so -- Including k2/torch. K2_TORCH_VERSION is 1.11 -- Downloading kaldifeat from https://github.com/csukuangfj/kaldifeat/archive/refs/tags/v1.20.tar.gz -- kaldifeat is downloaded to /opt/k2/build/temp.linux-x86_64-cpython-39/_deps/kaldifeat-src -- kaldifeat's binary dir is /opt/k2/build/temp.linux-x86_64-cpython-39/_deps/kaldifeat-build -- C++ Standard version: 14 -- Python executable: /opt/conda/envs/torch1.11/bin/python3 CMake Warning at /opt/conda/envs/torch1.11/lib/python3.9/site-packages/torch/share/cmake/Torch/TorchConfig.cmake:22 (message): static library kineto_LIBRARY-NOTFOUND not found. Call Stack (most recent call first): /opt/conda/envs/torch1.11/lib/python3.9/site-packages/torch/share/cmake/Torch/TorchConfig.cmake:127 (append_torchlib_if_found) cmake/torch.cmake:11 (find_package) build/temp.linux-x86_64-cpython-39/_deps/kaldifeat-src/CMakeLists.txt:55 (include)

-- K2_TORCH_VERSION: 1.11 -- PyTorch version: 1.11.0+cu113 -- PyTorch cuda version: 11.3 -- CMAKE_CXX_FLAGS: -D_GLIBCXX_USE_CXX11_ABI=0 -Wno-unused-variable -Wno-strict-overflow -D_GLIBCXX_USE_CXX11_ABI=0 -- CMAKE_INSTALL_PREFIX: /opt/k2/build/lib.linux-x86_64-cpython-39/k2 -- All headers: /opt/k2/build/temp.linux-x86_64-cpython-39/_deps/kaldifeat-src/kaldifeat/csrc/feature-common-inl.h;/opt/k2/build/temp.linux-x86_64-cpython-39/_deps/kaldifeat-src/kaldifeat/csrc/feature-common.h;/opt/k2/build/temp.linux-x86_64-cpython-39/_deps/kaldifeat-src/kaldifeat/csrc/feature-fbank.h;/opt/k2/build/temp.linux-x86_64-cpython-39/_deps/kaldifeat-src/kaldifeat/csrc/feature-functions.h;/opt/k2/build/temp.linux-x86_64-cpython-39/_deps/kaldifeat-src/kaldifeat/csrc/feature-mfcc.h;/opt/k2/build/temp.linux-x86_64-cpython-39/_deps/kaldifeat-src/kaldifeat/csrc/feature-plp.h;/opt/k2/build/temp.linux-x86_64-cpython-39/_deps/kaldifeat-src/kaldifeat/csrc/feature-spectrogram.h;/opt/k2/build/temp.linux-x86_64-cpython-39/_deps/kaldifeat-src/kaldifeat/csrc/feature-window.h;/opt/k2/build/temp.linux-x86_64-cpython-39/_deps/kaldifeat-src/kaldifeat/csrc/log.h;/opt/k2/build/temp.linux-x86_64-cpython-39/_deps/kaldifeat-src/kaldifeat/csrc/matrix-functions.h;/opt/k2/build/temp.linux-x86_64-cpython-39/_deps/kaldifeat-src/kaldifeat/csrc/mel-computations.h;/opt/k2/build/temp.linux-x86_64-cpython-39/_deps/kaldifeat-src/kaldifeat/csrc/online-feature-itf.h;/opt/k2/build/temp.linux-x86_64-cpython-39/_deps/kaldifeat-src/kaldifeat/csrc/online-feature.h;/opt/k2/build/temp.linux-x86_64-cpython-39/_deps/kaldifeat-src/kaldifeat/csrc/pitch-functions.h -- Configuring done -- Generating done -- Build files have been written to: /opt/k2/build/temp.linux-x86_64-cpython-39 /**

ifndef K2_CSRC_VERSIONH

define K2_CSRC_VERSIONH

namespace k2 {

// Version of k2 in "major.minor.patch" format static constexpr const char *kVersion = "1.24.3";

// The commit used to build k2 static constexpr const char *kGitSha1 = "b835546b6005d243865e0acc3d29bd9c51670b1e";

// Date of the commit used to build k2 static constexpr const char *kGitDate = "Wed Jul 26 19:29:59 2023";

// Version of CUDA used to build k2. // Its format is "major.minor", e.g., 10.1 static constexpr const char *kCudaVersion = "11.3";

// cuDNN version, e.g., 8.0.2 static constexpr const char *kCudnnVersion = "8.2.0";

// clang-format off // Version of Python used to build k2 Python bindings. static constexpr const char *kPythonVersion = "3.9";

// clang-format on

// CMake build type, e.g., Release or Debug. static constexpr const char *kBuildType = "Release";

// The operating system that is used to build k2, e.g., Ubuntu 16.04 LTS static constexpr const char *kOS = R"os()os";

// e.g., 3.18.0 static constexpr const char *kCMakeVersion = "3.20.20210602-g2104cfe-dirty";

// Version of the compiler, e.g., 5.4.0 static constexpr const char *kGCCVersion = "7.3.1";

// CUDA flags used to compile k2 static constexpr const char *kCMakeCudaFlags = R"cuda_flags( -Wno-deprecated-gpu-targets -lineinfo --expt-extended-lambda -use_fast_math -Xptxas=-w --expt-extended-lambda -gencode arch=compute_70,code=sm_70 -DONNX_NAMESPACE=onnx_c2 -gencode arch=compute_70,code=sm_70 -Xcudafe --diag_suppress=cc_clobber_ignored,--diag_suppress=integer_sign_change,--diag_suppress=useless_using_declaration,--diag_suppress=set_but_not_used,--diag_suppress=field_without_dll_interface,--diag_suppress=base_class_has_different_dll_interface,--diag_suppress=dll_interface_conflict_none_assumed,--diag_suppress=dll_interface_conflict_dllexport_assumed,--diag_suppress=implicit_return_from_non_void_function,--diag_suppress=unsigned_compare_with_zero,--diag_suppress=declared_but_not_referenced,--diag_suppress=bad_friend_decl --expt-relaxed-constexpr --expt-extended-lambda -D_GLIBCXX_USE_CXX11_ABI=0 --compiler-options -Wall --compiler-options -Wno-strict-overflow --compiler-options -Wno-unknown-pragmas )cuda_flags";

// CXX flags used to compile k2 static constexpr const char *kCMakeCxxFlags = R"cxx_flags( -D_GLIBCXX_USE_CXX11_ABI=0 -Wno-unused-variable -Wno-strict-overflow )cxx_flags";

// Which PyTorch version k2 is using, e.g., 1.6.0+cu101 static constexpr const char *kTorchVersion = "1.11.0+cu113";

// Which CUDA version PyTorch is using, e.g., 10.1 static constexpr const char *kTorchCudaVersion = "11.3";

ifndef K2_WITH_CUDA

define K2_WITH_CUDA

endif

ifdef K2_WITH_CUDA

static constexpr bool kWithCuda = true;

else

static constexpr bool kWithCuda = false;

endif

// Indicate whether NVTX is enabled or not

ifndef K2_ENABLE_NVTX

define K2_ENABLE_NVTX

endif

ifdef K2_ENABLE_NVTX

static constexpr bool kEnableNvtx = true;

else

static constexpr bool kEnableNvtx = false;

endif

} // namespace k2

endif // K2_CSRC_VERSIONH

[ 1%] Building CUDA object k2/csrc/CMakeFiles/k2_log.dir/log.cu.o Consolidate compiler generated dependencies of target kaldifeat_core [ 9%] Built target kaldifeat_core [ 9%] Building CXX object k2/torch/csrc/CMakeFiles/k2_fbank.dir/features.cc.o [ 10%] Linking CUDA shared library ../../lib/libk2_log.so [ 10%] Built target k2_log [ 12%] Building CXX object k2/csrc/host/CMakeFiles/fsa.dir/arcsort.cc.o [ 13%] Building CXX object k2/csrc/host/CMakeFiles/fsa.dir/aux_labels.cc.o [ 13%] Building CXX object k2/csrc/host/CMakeFiles/fsa.dir/connect.cc.o [ 14%] Building CXX object k2/csrc/host/CMakeFiles/fsa.dir/determinize.cc.o [ 15%] Building CXX object k2/csrc/host/CMakeFiles/fsa.dir/determinize_pruned.cc.o [ 16%] Building CXX object k2/csrc/host/CMakeFiles/fsa.dir/fsa.cc.o [ 16%] Building CXX object k2/csrc/host/CMakeFiles/fsa.dir/fsa_equivalent.cc.o [ 17%] Building CXX object k2/csrc/host/CMakeFiles/fsa.dir/fsa_renderer.cc.o [ 18%] Building CXX object k2/csrc/host/CMakeFiles/fsa.dir/fsa_util.cc.o [ 19%] Building CXX object k2/csrc/host/CMakeFiles/fsa.dir/intersect.cc.o [ 19%] Building CXX object k2/csrc/host/CMakeFiles/fsa.dir/properties.cc.o [ 20%] Building CXX object k2/csrc/host/CMakeFiles/fsa.dir/rmepsilon_pruned.cc.o [ 21%] Building CXX object k2/csrc/host/CMakeFiles/fsa.dir/topsort.cc.o [ 23%] Building CXX object k2/csrc/host/CMakeFiles/fsa.dir/util.cc.o [ 23%] Building CXX object k2/csrc/host/CMakeFiles/fsa.dir/weights.cc.o [ 24%] Linking CXX shared library ../../../lib/libk2fsa.so [ 24%] Built target fsa [ 27%] Building CUDA object k2/csrc/CMakeFiles/context.dir/array_ops.cu.o [ 28%] Building CUDA object k2/csrc/CMakeFiles/context.dir/algorithms.cu.o [ 28%] Building CUDA object k2/csrc/CMakeFiles/context.dir/connect.cu.o [ 28%] Building CUDA object k2/csrc/CMakeFiles/context.dir/array_of_ragged.cu.o [ 28%] Building CUDA object k2/csrc/CMakeFiles/context.dir/context.cu.o [ 29%] Building CUDA object k2/csrc/CMakeFiles/context.dir/dtype.cu.o [ 29%] Building CUDA object k2/csrc/CMakeFiles/context.dir/fsa.cu.o [ 30%] Building CUDA object k2/csrc/CMakeFiles/context.dir/fsa_algo.cu.o [ 31%] Linking CXX shared library ../../../lib/libk2_fbank.so [ 32%] Building CUDA object k2/csrc/CMakeFiles/context.dir/fsa_utils.cu.o [ 32%] Built target k2_fbank [ 34%] Building CUDA object k2/csrc/CMakeFiles/context.dir/hash.cu.o [ 34%] Building CUDA object k2/csrc/CMakeFiles/context.dir/host_shim.cu.o [ 35%] Building CUDA object k2/csrc/CMakeFiles/context.dir/intersect.cu.o [ 36%] Building CUDA object k2/csrc/CMakeFiles/context.dir/intersect_dense.cu.o [ 37%] Building CUDA object k2/csrc/CMakeFiles/context.dir/intersect_dense_pruned.cu.o [ 37%] Building CUDA object k2/csrc/CMakeFiles/context.dir/math.cu.o [ 38%] Building CUDA object k2/csrc/CMakeFiles/context.dir/moderngpu_allocator.cu.o [ 39%] Building CUDA object k2/csrc/CMakeFiles/context.dir/pinned_context.cu.o [ 40%] Building CUDA object k2/csrc/CMakeFiles/context.dir/ragged.cu.o [ 40%] Building CUDA object k2/csrc/CMakeFiles/context.dir/ragged_ops.cu.o [ 41%] Building CUDA object k2/csrc/CMakeFiles/context.dir/ragged_utils.cu.o [ 42%] Building CUDA object k2/csrc/CMakeFiles/context.dir/rand.cu.o [ 43%] Building CUDA object k2/csrc/CMakeFiles/context.dir/reverse.cu.o [ 45%] Building CUDA object k2/csrc/CMakeFiles/context.dir/rm_epsilon.cu.o [ 45%] Building CUDA object k2/csrc/CMakeFiles/context.dir/rnnt_decode.cu.o [ 46%] Building CUDA object k2/csrc/CMakeFiles/context.dir/tensor.cu.o [ 47%] Building CUDA object k2/csrc/CMakeFiles/context.dir/tensor_ops.cu.o [ 48%] Building CUDA object k2/csrc/CMakeFiles/context.dir/thread_pool.cu.o [ 48%] Building CUDA object k2/csrc/CMakeFiles/context.dir/timer.cu.o [ 49%] Building CUDA object k2/csrc/CMakeFiles/context.dir/top_sort.cu.o [ 50%] Building CUDA object k2/csrc/CMakeFiles/context.dir/torch_util.cu.o [ 51%] Building CUDA object k2/csrc/CMakeFiles/context.dir/utils.cu.o [ 51%] Building CUDA object k2/csrc/CMakeFiles/context.dir/nbest.cu.o [ 52%] Building CUDA object k2/csrc/CMakeFiles/context.dir/pytorch_context.cu.o /opt/k2/k2/csrc/ragged_ops_inl.h: In function ‘k2::Renumbering k2::PruneRaggedAxis1(k2::Ragged&, T, int32_t) [with T = double]’: /opt/k2/k2/csrc/ragged_ops_inl.h:870:16: warning: ‘order_map_data’ may be used uninitialized in this function [-Wmaybe-uninitialized] const int32_t *order_map_data; ^~~~~~ [ 53%] Building CUDA object k2/csrc/CMakeFiles/context.dir/cudpp/cudpp.cu.o [ 54%] Linking CUDA device code CMakeFiles/context.dir/cmake_device_link.o [ 54%] Linking CUDA shared library ../../lib/libk2context.so [ 54%] Built target context [ 54%] Building CUDA object k2/python/csrc/CMakeFiles/_k2.dir/k2.cu.o [ 59%] Building CUDA object k2/python/csrc/CMakeFiles/_k2.dir/version.cu.o [ 59%] Building CUDA object k2/torch/csrc/CMakeFiles/k2_torch.dir/decode.cu.o [ 58%] Building CUDA object k2/python/csrc/CMakeFiles/_k2.dir/torch.cu.o [ 59%] Building CUDA object k2/torch/csrc/CMakeFiles/k2_torch.dir/beam_search.cu.o [ 59%] Building CUDA object k2/python/csrc/CMakeFiles/_k2.dir/torch/arc.cu.o [ 59%] Building CUDA object k2/python/csrc/CMakeFiles/_k2.dir/torch/fsa.cu.o /opt/k2/k2/torch/csrc/beam_search.cu: In function ‘std::vector<std::vector, std::allocator<std::vector > > k2::ModifiedBeamSearch(const torch::jit::Module&, const at::Tensor&, const at::Tensor&, int32_t)’: /opt/k2/k2/torch/csrc/beam_search.cu:272:20: warning: comparison of integer expressions of different signedness: ‘int32_t’ {aka ‘int’} and ‘std::vector::size_type’ {aka ‘long unsigned int’} [-Wsign-compare] if (cur_batch_size < cur.size()) {


/opt/k2/k2/torch/csrc/beam_search.cu:298:23: warning: comparison of integer expressions of different signedness: ‘int32_t’ {aka ‘int’} and ‘std::vector<k2::Hypothesis>::size_type’ {aka ‘long unsigned int’} [-Wsign-compare]
     for (int32_t k = 0; k != prev.size(); ++k) {
                     ~~^~~~~~~~~~~~~~
[ 62%] Building CUDA object k2/torch/csrc/CMakeFiles/k2_torch.dir/dense_fsa_vec.cu.o
[ 62%] Building CUDA object k2/python/csrc/CMakeFiles/_k2.dir/torch/fsa_algo.cu.o
[ 62%] Building CUDA object k2/torch/csrc/CMakeFiles/k2_torch.dir/deserialization.cu.o
[ 63%] Building CUDA object k2/python/csrc/CMakeFiles/_k2.dir/torch/index_add.cu.o
[ 64%] Building CUDA object k2/torch/csrc/CMakeFiles/k2_torch.dir/fsa_algo.cu.o
[ 64%] Building CUDA object k2/torch/csrc/CMakeFiles/k2_torch.dir/fsa_class.cu.o
/opt/k2/k2/torch/csrc/deserialization.cu: In function ‘void k2::_GLOBAL__N__50_tmpxft_000078be_00000000_7_deserialization_cpp1_ii_4aae5ec1::restoreAccurateTypeTags(const c10::IValue&, const TypePtr&)’:
/opt/k2/k2/torch/csrc/deserialization.cu:101:8: warning: enumeration value ‘StorageType’ not handled in switch [-Wswitch]
     switch (w.static_type->kind()) {
        ^
/opt/k2/k2/torch/csrc/deserialization.cu:101:8: warning: enumeration value ‘ComplexType’ not handled in switch [-Wswitch]
/opt/k2/k2/torch/csrc/deserialization.cu:101:8: warning: enumeration value ‘StreamObjType’ not handled in switch [-Wswitch]
/opt/k2/k2/torch/csrc/deserialization.cu:101:8: warning: enumeration value ‘UnionType’ not handled in switch [-Wswitch]
/opt/k2/k2/torch/csrc/deserialization.cu:101:8: warning: enumeration value ‘DynamicType’ not handled in switch [-Wswitch]
[ 65%] Building CUDA object k2/python/csrc/CMakeFiles/_k2.dir/torch/index_select.cu.o
[ 65%] Building CUDA object k2/python/csrc/CMakeFiles/_k2.dir/torch/mutual_information.cu.o
[ 68%] Building CUDA object k2/torch/csrc/CMakeFiles/k2_torch.dir/hypothesis.cu.o
[ 68%] Building CUDA object k2/python/csrc/CMakeFiles/_k2.dir/torch/mutual_information_cpu.cu.o
[ 69%] Building CUDA object k2/python/csrc/CMakeFiles/_k2.dir/torch/nbest.cu.o
[ 72%] Building CUDA object k2/python/csrc/CMakeFiles/_k2.dir/torch/pruned_ranges_to_lattice.cu.o
[ 72%] Building CUDA object k2/torch/csrc/CMakeFiles/k2_torch.dir/parse_options.cu.o
[ 72%] Building CUDA object k2/torch/csrc/CMakeFiles/k2_torch.dir/nbest.cu.o
/opt/k2/k2/torch/csrc/parse_options.cu: In member function ‘int32_t k2::ParseOptions::ToInt(const string&) const’:
/opt/k2/k2/torch/csrc/parse_options.cu:705:8: warning: ‘ret’ may be used uninitialized in this function [-Wmaybe-uninitialized]
   return ret;
        ^~~
/opt/k2/k2/torch/csrc/parse_options.cu: In member function ‘uint32_t k2::ParseOptions::ToUint(const string&) const’:
/opt/k2/k2/torch/csrc/parse_options.cu:712:8: warning: ‘ret’ may be used uninitialized in this function [-Wmaybe-uninitialized]
   return ret;
        ^~~
[ 72%] Building CUDA object k2/torch/csrc/CMakeFiles/k2_torch.dir/symbol_table.cu.o
[ 73%] Building CUDA object k2/python/csrc/CMakeFiles/_k2.dir/torch/ragged.cu.o
[ 73%] Building CUDA object k2/torch/csrc/CMakeFiles/k2_torch.dir/utils.cu.o
[ 74%] Building CUDA object k2/python/csrc/CMakeFiles/_k2.dir/torch/ragged_ops.cu.o
[ 75%] Building CUDA object k2/torch/csrc/CMakeFiles/k2_torch.dir/wave_reader.cu.o
[ 76%] Building CUDA object k2/python/csrc/CMakeFiles/_k2.dir/torch/rnnt_decode.cu.o
[ 78%] Building CUDA object k2/python/csrc/CMakeFiles/_k2.dir/torch/v2/any.cu.o
[ 79%] Linking CUDA shared library ../../../lib/libk2_torch.so
[ 79%] Built target k2_torch
[ 79%] Building CUDA object k2/python/csrc/CMakeFiles/_k2.dir/torch/v2/autograd/swoosh.cu.o
[ 79%] Building CUDA object k2/torch/csrc/CMakeFiles/k2_torch_api.dir/torch_api.cu.o
[ 80%] Building CUDA object k2/python/csrc/CMakeFiles/_k2.dir/torch/v2/doc/doc.cu.o
[ 81%] Building CUDA object k2/torch/bin/CMakeFiles/ctc_decode.dir/ctc_decode.cu.o
[ 82%] Building CUDA object k2/torch/bin/CMakeFiles/hlg_decode.dir/hlg_decode.cu.o
[ 83%] Linking CUDA shared library ../../../lib/libk2_torch_api.so
[ 83%] Built target k2_torch_api
[ 84%] Building CUDA object k2/python/csrc/CMakeFiles/_k2.dir/torch/v2/k2.cu.o
[ 85%] Building CUDA object k2/torch/bin/CMakeFiles/ngram_lm_rescore.dir/ngram_lm_rescore.cu.o
[ 86%] Linking CUDA executable ../../../bin/ctc_decode
../../../lib/libk2fsa.so: undefined reference to `std::__throw_out_of_range_fmt(char const*, ...)@GLIBCXX_3.4.20'
../../../lib/libk2fsa.so: undefined reference to `std::runtime_error::runtime_error(char const*)@GLIBCXX_3.4.21'
../../../lib/libk2fsa.so: undefined reference to `operator delete(void*, unsigned long)@CXXABI_1.3.9'
collect2: error: ld returned 1 exit status
make[2]: *** [k2/torch/bin/CMakeFiles/ctc_decode.dir/build.make:122: bin/ctc_decode] Error 1
make[1]: *** [CMakeFiles/Makefile2:1393: k2/torch/bin/CMakeFiles/ctc_decode.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
[ 89%] Building CUDA object k2/python/csrc/CMakeFiles/_k2.dir/torch/mutual_information_cuda.cu.o
[ 89%] Building CUDA object k2/python/csrc/CMakeFiles/_k2.dir/torch/v2/ragged_shape.cu.o
[ 89%] Building CUDA object k2/python/csrc/CMakeFiles/_k2.dir/torch/v2/ragged_any.cu.o
[ 90%] Linking CUDA executable ../../../bin/hlg_decode
../../../lib/libk2fsa.so: undefined reference to `std::__throw_out_of_range_fmt(char const*, ...)@GLIBCXX_3.4.20'
../../../lib/libk2fsa.so: undefined reference to `std::runtime_error::runtime_error(char const*)@GLIBCXX_3.4.21'
../../../lib/libk2fsa.so: undefined reference to `operator delete(void*, unsigned long)@CXXABI_1.3.9'
collect2: error: ld returned 1 exit status
make[2]: *** [k2/torch/bin/CMakeFiles/hlg_decode.dir/build.make:122: bin/hlg_decode] Error 1
make[1]: *** [CMakeFiles/Makefile2:1424: k2/torch/bin/CMakeFiles/hlg_decode.dir/all] Error 2
[ 90%] Linking CUDA executable ../../../bin/ngram_lm_rescore
../../../lib/libk2fsa.so: undefined reference to `std::__throw_out_of_range_fmt(char const*, ...)@GLIBCXX_3.4.20'
../../../lib/libk2fsa.so: undefined reference to `std::runtime_error::runtime_error(char const*)@GLIBCXX_3.4.21'
../../../lib/libk2fsa.so: undefined reference to `operator delete(void*, unsigned long)@CXXABI_1.3.9'
collect2: error: ld returned 1 exit status
make[2]: *** [k2/torch/bin/CMakeFiles/ngram_lm_rescore.dir/build.make:122: bin/ngram_lm_rescore] Error 1
make[1]: *** [CMakeFiles/Makefile2:1455: k2/torch/bin/CMakeFiles/ngram_lm_rescore.dir/all] Error 2
[ 91%] Linking CUDA device code CMakeFiles/_k2.dir/cmake_device_link.o
[ 92%] Linking CUDA shared module ../../../lib/_k2.cpython-39-x86_64-linux-gnu.so
lto1: fatal error: LTO_tags out of range: Range is 0 to 365, value is 3073
compilation terminated.
lto-wrapper: /usr/bin/g++ returned 1 exit status
/opt/rh/devtoolset-8/root/usr/bin/ld: error: lto-wrapper failed
collect2: error: ld returned 1 exit status
make[2]: *** [k2/python/csrc/CMakeFiles/_k2.dir/build.make:531: lib/_k2.cpython-39-x86_64-linux-gnu.so] Error 1
make[1]: *** [CMakeFiles/Makefile2:1201: k2/python/csrc/CMakeFiles/_k2.dir/all] Error 2
make: *** [Makefile:146: all] Error 2
Traceback (most recent call last):
  File "/opt/k2/setup.py", line 234, in <module>
    setuptools.setup(
  File "/opt/conda/envs/torch1.11/lib/python3.9/site-packages/setuptools/__init__.py", line 107, in setup
    return distutils.core.setup(**attrs)
  File "/opt/conda/envs/torch1.11/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 185, in setup
    return run_commands(dist)
  File "/opt/conda/envs/torch1.11/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
    dist.run_commands()
  File "/opt/conda/envs/torch1.11/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
    self.run_command(cmd)
  File "/opt/conda/envs/torch1.11/lib/python3.9/site-packages/setuptools/dist.py", line 1234, in run_command
    super().run_command(command)
  File "/opt/conda/envs/torch1.11/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
    cmd_obj.run()
  File "/opt/conda/envs/torch1.11/lib/python3.9/site-packages/setuptools/command/install.py", line 80, in run
    self.do_egg_install()
  File "/opt/conda/envs/torch1.11/lib/python3.9/site-packages/setuptools/command/install.py", line 129, in do_egg_install
    self.run_command('bdist_egg')
  File "/opt/conda/envs/torch1.11/lib/python3.9/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
    self.distribution.run_command(command)
  File "/opt/conda/envs/torch1.11/lib/python3.9/site-packages/setuptools/dist.py", line 1234, in run_command
    super().run_command(command)
  File "/opt/conda/envs/torch1.11/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
    cmd_obj.run()
  File "/opt/conda/envs/torch1.11/lib/python3.9/site-packages/setuptools/command/bdist_egg.py", line 164, in run
    cmd = self.call_command('install_lib', warn_dir=0)
  File "/opt/conda/envs/torch1.11/lib/python3.9/site-packages/setuptools/command/bdist_egg.py", line 150, in call_command
    self.run_command(cmdname)
  File "/opt/conda/envs/torch1.11/lib/python3.9/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
    self.distribution.run_command(command)
  File "/opt/conda/envs/torch1.11/lib/python3.9/site-packages/setuptools/dist.py", line 1234, in run_command
    super().run_command(command)
  File "/opt/conda/envs/torch1.11/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
    cmd_obj.run()
  File "/opt/conda/envs/torch1.11/lib/python3.9/site-packages/setuptools/command/install_lib.py", line 11, in run
    self.build()
  File "/opt/conda/envs/torch1.11/lib/python3.9/site-packages/setuptools/_distutils/command/install_lib.py", line 111, in build
    self.run_command('build_ext')
  File "/opt/conda/envs/torch1.11/lib/python3.9/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
    self.distribution.run_command(command)
  File "/opt/conda/envs/torch1.11/lib/python3.9/site-packages/setuptools/dist.py", line 1234, in run_command
    super().run_command(command)
  File "/opt/conda/envs/torch1.11/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
    cmd_obj.run()
  File "/opt/conda/envs/torch1.11/lib/python3.9/site-packages/setuptools/command/build_ext.py", line 84, in run
    _build_ext.run(self)
  File "/opt/conda/envs/torch1.11/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 345, in run
    self.build_extensions()
  File "/opt/conda/envs/torch1.11/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 467, in build_extensions
    self._build_extensions_serial()
  File "/opt/conda/envs/torch1.11/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 493, in _build_extensions_serial
    self.build_extension(ext)
  File "/opt/k2/setup.py", line 206, in build_extension
    raise Exception("Failed to build k2")
Exception: Failed to build k2
gcc version 8.3.1 20190311 (Red Hat 8.3.1-3) (GCC)
csukuangfj commented 1 year ago

From your posted logs

// Version of the compiler, e.g., 5.4.0
static constexpr const char *kGCCVersion = "7.3.1";

...

../../../lib/libk2fsa.so: undefined reference to std::__throw_out_of_range_fmt(char const*, ...)@GLIBCXX_3.4.20' ../../../lib/libk2fsa.so: undefined reference to std::runtime_error::runtime_error(char const)@GLIBCXX_3.4.21'

...

Exception: Failed to build k2
gcc version 8.3.1 20190311 (Red Hat 8.3.1-3) (GCC)

It looks to me there are two different versions of GCC, i.e., GCC 7.3.1 and GCC 8.3.1.

Please follow the following doc https://k2-fsa.github.io/k2/installation/faqs.html#error-c-14-or-later-compatible-compiler-is-required-to-use-aten to setup the correct environment variables so that cmake uses the correct gcc to compile k2.


By the way, instead of compiling k2 from source, I suggest that you use the following method https://k2-fsa.github.io/k2/installation/from_wheels.html to install k2.

It is much much faster and there is no chance that you get an error with this method.

HyacinthJingjing commented 1 year ago

Thank you for reply. I have tried the install from the wheel "k2-1.24.3.dev20230524+cuda11.3.torch1.11.0-cp39-cp39-linux_x86_64.whl", but I encounter the error "/lib64/libm.so.6: version `GLIBC_2.27' not found" when I import k2 from python.Maybe the wheel is compiled on ubuntu,my stystem is linux?

csukuangfj commented 1 year ago

I have tried the install from the wheel "k2-1.24.3.dev20230524+cuda11.3.torch1.11.0-cp39-cp39-linux_x86_64.whl",

Please try a wheel built in July or after that.

The wheel built in May is not a manylinux wheel.

csukuangfj commented 1 year ago

I have tried the install from the wheel "k2-1.24.3.dev20230524+cuda11.3.torch1.11.0-cp39-cp39-linux_x86_64.whl"

Please follow https://k2-fsa.github.io/k2/installation/from_wheels.html#linux-cuda-example

Since you want to use cuda11.3+torch1.11.0, please use

pip install k2==1.24.3.dev20230725+cuda11.3.torch1.11.0 -f https://k2-fsa.github.io/k2/cuda.html
csukuangfj commented 1 year ago

@HyacinthJingjing

Have you fixed it?

HyacinthJingjing commented 1 year ago

Thank you. I have successfully installed k2.