Runtime Error - Githubissues

Describe the bug I always meet following error when compiling the code

Please complete all sections of this template if applicable. For installation, you must report the environment. Otherwise, your issue will be closed automatically.

To Reproduce

torch: 1.10.1
cuda: 11.3
gcc: 7.5.0
hardware: Nvidia A100 80G

Command: pip install -U git+https://github.com/NVIDIA/MinkowskiEngine -v --no-deps --install-option="--blas_include_dirs=${CONDA_PREFIX}/include" --install-option="--blas=openblas"

Expected behavior A clear and concise description of what you expected to happen.

Desktop (please complete the following information):

OS: Linux version 3.10.0-957.el7.x86_64
Python version: 3.7.13
Pytorch version:1.10.1
CUDA version: 11.3
NVIDIA Driver version: [e.g. 450.11]
Minkowski Engine version: latest
Output of the following command. (If you installed the latest MinkowskiEngine, paste the output of python -c "import MinkowskiEngine as ME; ME.print_diagnostics()". Otherwise, paste the output of the following command.)

[20/21] /mnt/cache/share/cuda-11.3/bin/nvcc  -I/mnt/cache/liuyuan/miniconda3/envs/test/lib/python3.7/site-packages/torch/include -I/mnt/cache/liuyuan/miniconda3/envs/test/lib/python3.7/
site-packages/torch/include/torch/csrc/api/include -I/mnt/cache/liuyuan/miniconda3/envs/test/lib/python3.7/site-packages/torch/include/TH -I/mnt/cache/liuyuan/miniconda3/envs/test/lib/pytho
n3.7/site-packages/torch/include/THC -I/mnt/cache/share/cuda-11.3/include -I/tmp/pip-req-build-dfaet08_/src -I/tmp/pip-req-build-dfaet08_/src/3rdparty -I/mnt/cache/liuyuan/miniconda3/envs/t
est/include -I/mnt/cache/liuyuan/miniconda3/envs/test/include/python3.7m -c -c /tmp/pip-req-build-dfaet08_/src/spmm.cu -o /tmp/pip-req-build-dfaet08_/build/temp.linux-x86_64-3.7/tmp/pip-req
-build-dfaet08_/src/spmm.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-o
ptions ''"'"'-fPIC'"'"'' --expt-relaxed-constexpr --expt-extended-lambda -O3 -Xcompiler=-fno-gnu-unique -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB=
"_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_80,code=compute_80 -gencode=arch=compute_80,code=sm_80 -std=c++
14
    FAILED: /tmp/pip-req-build-dfaet08_/build/temp.linux-x86_64-3.7/tmp/pip-req-build-dfaet08_/src/spmm.o
    /mnt/cache/share/cuda-11.3/bin/nvcc  -I/mnt/cache/liuyuan/miniconda3/envs/test/lib/python3.7/site-packages/torch/include -I/mnt/cache/liuyuan/miniconda3/envs/test/lib/python3.7/site-pac
kages/torch/include/torch/csrc/api/include -I/mnt/cache/liuyuan/miniconda3/envs/test/lib/python3.7/site-packages/torch/include/TH -I/mnt/cache/liuyuan/miniconda3/envs/test/lib/python3.7/sit
e-packages/torch/include/THC -I/mnt/cache/share/cuda-11.3/include -I/tmp/pip-req-build-dfaet08_/src -I/tmp/pip-req-build-dfaet08_/src/3rdparty -I/mnt/cache/liuyuan/miniconda3/envs/test/incl
ude -I/mnt/cache/liuyuan/miniconda3/envs/test/include/python3.7m -c -c /tmp/pip-req-build-dfaet08_/src/spmm.cu -o /tmp/pip-req-build-dfaet08_/build/temp.linux-x86_64-3.7/tmp/pip-req-build-d
faet08_/src/spmm.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options '
'"'"'-fPIC'"'"'' --expt-relaxed-constexpr --expt-extended-lambda -O3 -Xcompiler=-fno-gnu-unique -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstd
cpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_80,code=compute_80 -gencode=arch=compute_80,code=sm_80 -std=c++14
    /tmp/pip-req-build-dfaet08_/src/spmm.cu(93): warning: variable "is_int64" was declared but never referenced

    /tmp/pip-req-build-dfaet08_/src/spmm.cu(355): warning: variable "is_int64" was declared but never referenced
   /tmp/pip-req-build-dfaet08_/src/spmm.cu(355): warning: variable "is_int64" was declared but never referenced

    /tmp/pip-req-build-dfaet08_/src/spmm.cu(464): warning: variable "num_unique_keys" was set but never used
              detected during instantiation of "std::vector<at::Tensor, std::allocator<at::Tensor>> minkowski::coo_spmm_average<th_int_type>(const at::Tensor &, const at::Tensor &, int64_t,
 int64_t, const at::Tensor &, int64_t) [with th_int_type=int32_t]"
    (592): here

    /tmp/pip-req-build-dfaet08_/src/spmm.cu(464): warning: variable "num_unique_keys" was set but never used
              detected during instantiation of "std::vector<at::Tensor, std::allocator<at::Tensor>> minkowski::coo_spmm_average<th_int_type>(const at::Tensor &, const at::Tensor &, int64_t,
 int64_t, const at::Tensor &, int64_t) [with th_int_type=int32_t]"
    (592): here

    /tmp/pip-req-build-dfaet08_/src/spmm.cu: In instantiation of ‘minkowski::coo_spmm(const at::Tensor&, const at::Tensor&, const at::Tensor&, int64_t, int64_t, const at::Tensor&, int64_t, 
bool)::<lambda()>::<lambda()> [with th_int_type = int]’:
    /tmp/pip-req-build-dfaet08_/src/spmm.cu:203:0:   required from ‘struct minkowski::coo_spmm(const at::Tensor&, const at::Tensor&, const at::Tensor&, int64_t, int64_t, const at::Tensor&, 
int64_t, bool)::<lambda()> [with th_int_type = int]::<lambda()>’
    /tmp/pip-req-build-dfaet08_/src/spmm.cu:203:0:   required from ‘minkowski::coo_spmm(const at::Tensor&, const at::Tensor&, const at::Tensor&, int64_t, int64_t, const at::Tensor&, int64_t
, bool)::<lambda()> [with th_int_type = int]’
    /tmp/pip-req-build-dfaet08_/src/spmm.cu:203:0:   required from ‘struct minkowski::coo_spmm(const at::Tensor&, const at::Tensor&, const at::Tensor&, int64_t, int64_t, const at::Tensor&, 
int64_t, bool) [with th_int_type = int; int64_t = long int]::<lambda()>’
    /tmp/pip-req-build-dfaet08_/src/spmm.cu:203:0:   required from ‘at::Tensor minkowski::coo_spmm(const at::Tensor&, const at::Tensor&, const at::Tensor&, int64_t, int64_t, const at::Tenso
r&, int64_t, bool) [with th_int_type = int; int64_t = long int]’
    /tmp/pip-req-build-dfaet08_/src/spmm.cu:335:232:   required from here
    /tmp/pip-req-build-dfaet08_/src/spmm.cu:203:1272: internal compiler error: in maybe_undo_parenthesized_ref, at cp/semantics.c:1740
       AT_DISPATCH_FLOATING_TYPES(vals.scalar_type(), "coo_spmm", [&] {
0x6c12d9 maybe_undo_parenthesized_ref(tree_node*)
        ../../gcc/cp/semantics.c:1739
    0x706578 cp_fold
        ../../gcc/cp/cp-gimplify.c:2180
    0x7073dc cp_fold_maybe_rvalue
        ../../gcc/cp/cp-gimplify.c:2042
    0x706286 cp_fold
        ../../gcc/cp/cp-gimplify.c:2149
    0x69dadf cp_build_binary_op(unsigned int, tree_code, tree_node*, tree_node*, int)
        ../../gcc/cp/typeck.c:5260
    0x5e47dc build_new_op_1
        ../../gcc/cp/call.c:5978
    0x5e52be build_new_op(unsigned int, tree_code, int, tree_node*, tree_node*, tree_node*, tree_node**, int)
        ../../gcc/cp/call.c:6022
    0x695952 build_x_binary_op(unsigned int, tree_code, tree_node*, tree_code, tree_node*, tree_code, tree_node**, int)
        ../../gcc/cp/typeck.c:3941
    0x61e43f tsubst_copy_and_build(tree_node*, tree_node*, int, tree_node*, bool, bool)
        ../../gcc/cp/pt.c:17001
    0x61e3e8 tsubst_copy_and_build(tree_node*, tree_node*, int, tree_node*, bool, bool)
        ../../gcc/cp/pt.c:16998
    0x61f5b6 tsubst_copy_and_build(tree_node*, tree_node*, int, tree_node*, bool, bool)
        ../../gcc/cp/pt.c:17312
    0x61fef0 tsubst_copy_and_build(tree_node*, tree_node*, int, tree_node*, bool, bool)
        ../../gcc/cp/pt.c:17660
    0x61f715 tsubst_copy_and_build(tree_node*, tree_node*, int, tree_node*, bool, bool)
        ../../gcc/cp/pt.c:16897
    0x61e1a9 tsubst_copy_and_build(tree_node*, tree_node*, int, tree_node*, bool, bool)
[21/21] /mnt/cache/share/cuda-11.3/bin/nvcc  -I/mnt/cache/liuyuan/miniconda3/envs/test/lib/python3.7/site-packages/torch/include -I/mnt/cache/liuyuan/miniconda3/envs/test/lib/python3.7/
site-packages/torch/include/torch/csrc/api/include -I/mnt/cache/liuyuan/miniconda3/envs/test/lib/python3.7/site-packages/torch/include/TH -I/mnt/cache/liuyuan/miniconda3/envs/test/lib/pytho
n3.7/site-packages/torch/include/THC -I/mnt/cache/share/cuda-11.3/include -I/tmp/pip-req-build-dfaet08_/src -I/tmp/pip-req-build-dfaet08_/src/3rdparty -I/mnt/cache/liuyuan/miniconda3/envs/t
est/include -I/mnt/cache/liuyuan/miniconda3/envs/test/include/python3.7m -c -c /tmp/pip-req-build-dfaet08_/pybind/minkowski.cu -o /tmp/pip-req-build-dfaet08_/build/temp.linux-x86_64-3.7/pyb
ind/minkowski.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'
"'-fPIC'"'"'' --expt-relaxed-constexpr --expt-extended-lambda -O3 -Xcompiler=-fno-gnu-unique -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp
"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_80,code=compute_80 -gencode=arch=compute_80,code=sm_80 -std=c++14
    ninja: build stopped: subcommand failed.
    Traceback (most recent call last):
      File "/mnt/cache/liuyuan/miniconda3/envs/test/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1746, in _run_ninja_build
        env=env)
      File "/mnt/cache/liuyuan/miniconda3/envs/test/lib/python3.7/subprocess.py", line 512, in run
        output=stdout, stderr=stderr)
    subprocess.CalledProcessError: Command '['ninja', '-v', '-j', '12']' returned non-zero exit status 1.

    The above exception was the direct cause of the following exception:

    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-req-build-dfaet08_/setup.py", line 364, in <module>
        python_requires=">=3.6",
      File "/mnt/cache/liuyuan/miniconda3/envs/test/lib/python3.7/site-packages/setuptools/__init__.py", line 153, in setup
        return distutils.core.setup(**attrs)
      File "/mnt/cache/liuyuan/miniconda3/envs/test/lib/python3.7/distutils/core.py", line 148, in setup
File "/mnt/cache/liuyuan/miniconda3/envs/test/lib/python3.7/distutils/core.py", line 148, in setup
        dist.run_commands()
      File "/mnt/cache/liuyuan/miniconda3/envs/test/lib/python3.7/distutils/dist.py", line 966, in run_commands
        self.run_command(cmd)
      File "/mnt/cache/liuyuan/miniconda3/envs/test/lib/python3.7/distutils/dist.py", line 985, in run_command
        cmd_obj.run()
      File "/mnt/cache/liuyuan/miniconda3/envs/test/lib/python3.7/site-packages/setuptools/command/install.py", line 61, in run
        return orig.install.run(self)
      File "/mnt/cache/liuyuan/miniconda3/envs/test/lib/python3.7/distutils/command/install.py", line 545, in run
        self.run_command('build')
      File "/mnt/cache/liuyuan/miniconda3/envs/test/lib/python3.7/distutils/cmd.py", line 313, in run_command
        self.distribution.run_command(command)
      File "/mnt/cache/liuyuan/miniconda3/envs/test/lib/python3.7/distutils/dist.py", line 985, in run_command
        cmd_obj.run()
      File "/mnt/cache/liuyuan/miniconda3/envs/test/lib/python3.7/distutils/command/build.py", line 135, in run
        self.run_command(cmd_name)
      File "/mnt/cache/liuyuan/miniconda3/envs/test/lib/python3.7/distutils/cmd.py", line 313, in run_command
        self.distribution.run_command(command)
      File "/mnt/cache/liuyuan/miniconda3/envs/test/lib/python3.7/distutils/dist.py", line 985, in run_command
        cmd_obj.run()
      File "/mnt/cache/liuyuan/miniconda3/envs/test/lib/python3.7/site-packages/setuptools/command/build_ext.py", line 79, in run
        _build_ext.run(self)
      File "/mnt/cache/liuyuan/miniconda3/envs/test/lib/python3.7/distutils/command/build_ext.py", line 340, in run
        self.build_extensions()
      File "/mnt/cache/liuyuan/miniconda3/envs/test/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 741, in build_extensions
        build_ext.build_extensions(self)
File "/mnt/cache/liuyuan/miniconda3/envs/test/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 741, in build_extensions
        build_ext.build_extensions(self)
      File "/mnt/cache/liuyuan/miniconda3/envs/test/lib/python3.7/distutils/command/build_ext.py", line 449, in build_extensions
        self._build_extensions_serial()
      File "/mnt/cache/liuyuan/miniconda3/envs/test/lib/python3.7/distutils/command/build_ext.py", line 474, in _build_extensions_serial
        self.build_extension(ext)
      File "/mnt/cache/liuyuan/miniconda3/envs/test/lib/python3.7/site-packages/setuptools/command/build_ext.py", line 202, in build_extension
        _build_ext.build_extension(self, ext)
      File "/mnt/cache/liuyuan/miniconda3/envs/test/lib/python3.7/distutils/command/build_ext.py", line 534, in build_extension
        depends=ext.depends)
      File "/mnt/cache/liuyuan/miniconda3/envs/test/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 571, in unix_wrap_ninja_compile
        with_cuda=with_cuda)
      File "/mnt/cache/liuyuan/miniconda3/envs/test/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1424, in _write_ninja_file_and_compile_objects
        error_prefix='Error compiling objects for extension')
      File "/mnt/cache/liuyuan/miniconda3/envs/test/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1756, in _run_ninja_build
        raise RuntimeError(message) from e
    RuntimeError: Error compiling objects for extension
    Running setup.py install for MinkowskiEngine ... error
ERROR: Command errored out with exit status 1: /mnt/cache/liuyuan/miniconda3/envs/test/bin/python -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-req-build-dfae
t08_/setup.py'"'"'; __file__='"'"'/tmp/pip-req-build-dfaet08_/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setu
ptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-r_ag34n0/i
nstall-record.txt --single-version-externally-managed --compile --install-headers /mnt/cache/liuyuan/miniconda3/envs/test/include/python3.7m/MinkowskiEngine --blas_include_dirs=/mnt/cache/l
iuyuan/miniconda3/envs/test/include --blas=openblas Check the logs for full command output.

Additional context Add any other context about the problem here.

NVIDIA / MinkowskiEngine

Runtime Error #523