Installation fails with `incomplete class type "at::Tensor"`

Describe the bug I was trying to pip install Minkowski Engine on Ubuntu, but I received the following error originating from a pytorch header:

[...]     
  [13/21] /usr/local/cuda/bin/nvcc  -I/home/myuser/.local/lib/python3.8/site-packages/torch/include -I/home/myuser/.local/lib/python3.8/site-packages/torch/include/torch/csrc/api/include -I/home/myuser/.local/lib/python3.8/site-packages/torch/include/TH -I/home/myuser/.local/lib/python3.8/site-packages/torch/include/THC -I/usr/local/cuda/include -I/home/myuser/MinkowskiEngine/src -I/home/myuser/MinkowskiEngine/src/3rdparty -I/usr/include/python3.8 -c -c /home/myuser/MinkowskiEngine/src/spmm.cu -o /home/myuser/MinkowskiEngine/build/temp.linux-x86_64-cpython-38/home/myuser/MinkowskiEngine/src/spmm.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' --expt-relaxed-constexpr --expt-extended-lambda -O3 -Xcompiler=-fno-gnu-unique -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1013"' -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_61,code=compute_61 -gencode=arch=compute_61,code=sm_61 -std=c++14
  FAILED: /home/myuser/MinkowskiEngine/build/temp.linux-x86_64-cpython-38/home/myuser/MinkowskiEngine/src/spmm.o
  /usr/local/cuda/bin/nvcc  -I/home/myuser/.local/lib/python3.8/site-packages/torch/include -I/home/myuser/.local/lib/python3.8/site-packages/torch/include/torch/csrc/api/include -I/home/myuser/.local/lib/python3.8/site-packages/torch/include/TH -I/home/myuser/.local/lib/python3.8/site-packages/torch/include/THC -I/usr/local/cuda/include -I/home/myuser/MinkowskiEngine/src -I/home/myuser/MinkowskiEngine/src/3rdparty -I/usr/include/python3.8 -c -c /home/myuser/MinkowskiEngine/src/spmm.cu -o /home/myuser/MinkowskiEngine/build/temp.linux-x86_64-cpython-38/home/myuser/MinkowskiEngine/src/spmm.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' --expt-relaxed-constexpr --expt-extended-lambda -O3 -Xcompiler=-fno-gnu-unique -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1013"' -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_61,code=compute_61 -gencode=arch=compute_61,code=sm_61 -std=c++14
  /home/myuser/.local/lib/python3.8/site-packages/torch/include/c10/core/SymInt.h(84): warning #68-D: integer conversion resulted in a change of sign

  /home/myuser/.local/lib/python3.8/site-packages/torch/include/c10/util/ArrayRef.h(140): error: expression must be a pointer to a complete object type
            detected during instantiation of "c10::ArrayRef<T>::iterator c10::ArrayRef<T>::end() const [with T=at::Tensor]"
  /home/myuser/.local/lib/python3.8/site-packages/torch/include/ATen/cuda/CUDAUtils.h(14): here

  /home/myuser/.local/lib/python3.8/site-packages/torch/include/ATen/cuda/CUDAUtils.h(15): error: incomplete class type "at::Tensor" is not allowed

  /home/myuser/MinkowskiEngine/src/spmm.cu(102): warning #177-D: variable "is_int64" was declared but never referenced

  /home/myuser/MinkowskiEngine/src/spmm.cu(364): warning #177-D: variable "is_int64" was declared but never referenced

  2 errors detected in the compilation of "/home/myuser/MinkowskiEngine/src/spmm.cu".
[...]

I can fix this error by editing the file /home/myuser/.local/lib/python3.8/site-packages/torch/include/ATen/cuda/CUDAUtils.h to include this header:

#include <ATen/core/Tensor.h>

It compiles nicely and I tried to run one of the examples, which seems to work.

Not sure if this is actually an error on the pytorch-side or actually a client-side issue with MinkowskiEngine (or actually a user error with my installation?). I think maybe you can judge this better.

To Reproduce

These are the steps how this can be reproduced on my system (iirc):

Install CUDA 11.6, set CUDA_HOME and extend PATH
Install libopenblas-dev
Install pytorch for CUDA 11.6 (python3 -m pip install torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/cu116)
Pip-Install Minkowski Engine (in my case: git clone and python3 -m pip install .)

Expected behavior Installation without an error.

Desktop (please complete the following information):

torch/utils/collect_env.py:

PyTorch version: 1.12.0+cu116
Is debug build: False
CUDA used to build PyTorch: 11.6
ROCM used to build PyTorch: N/A

OS: Ubuntu 20.04.4 LTS (x86_64)
GCC version: (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0
Clang version: Could not collect
CMake version: version 3.16.3
Libc version: glibc-2.31

Python version: 3.8.10 (default, Mar 15 2022, 12:22:08)  [GCC 9.4.0] (64-bit runtime)
Python platform: Linux-5.13.0-52-generic-x86_64-with-glibc2.29
Is CUDA available: True
CUDA runtime version: 11.6.124
GPU models and configuration: GPU 0: NVIDIA GeForce MX150
Nvidia driver version: 510.47.03
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] numpy==1.23.0
[pip3] torch==1.12.0+cu116
[pip3] torchaudio==0.12.0+cu116
[pip3] torchvision==0.13.0+cu116
[conda] Could not collect

MinkowskiEngine/diagnostics.py:

==========System==========
Linux-5.13.0-52-generic-x86_64-with-glibc2.29
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=20.04
DISTRIB_CODENAME=focal
DISTRIB_DESCRIPTION="Ubuntu 20.04.4 LTS"
3.8.10 (default, Mar 15 2022, 12:22:08) 
[GCC 9.4.0]
==========Pytorch==========
1.12.0+cu116
torch.cuda.is_available(): True
==========NVIDIA-SMI==========
/usr/bin/nvidia-smi
Driver Version 510.47.03
CUDA Version 11.6
VBIOS Version 86.08.28.00.59
Image Version N/A
GSP Firmware Version N/A
==========NVCC==========
/usr/local/cuda/bin/nvcc
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2022 NVIDIA Corporation
Built on Tue_Mar__8_18:18:20_PST_2022
Cuda compilation tools, release 11.6, V11.6.124
Build cuda_11.6.r11.6/compiler.31057947_0
==========CC==========
/usr/bin/c++
c++ (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0
Copyright (C) 2019 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
==========MinkowskiEngine==========
0.5.4
MinkowskiEngine compiled with CUDA Support: True
NVCC version MinkowskiEngine is compiled: 11060
CUDART version MinkowskiEngine is compiled: 11060

I installed the current master version of MinkowskiEngine (2f31bc51e0abdf89ed20730e531480df1b2cc64a).

Additional context

NVIDIA / MinkowskiEngine

Installation fails with `incomplete class type "at::Tensor"` #485