search

NVIDIA / MinkowskiEngine

Minkowski Engine is an auto-diff neural network library for high-dimensional sparse tensors
https://nvidia.github.io/MinkowskiEngine
Other
2.5k stars 369 forks source link

Failed to install the latest MinkowskiEngine (Dec 2023) #573

Open bring-nirachornkul opened 11 months ago

bring-nirachornkul commented 11 months ago

Failed to install the latest MinkowskiEngine (Dec 2023)

I used this method to install MinkowskiEngine.

CUDA 11.X

We recommend python>=3.6 for installation. First, follow the anaconda documentation to install anaconda on your computer.


conda create -n py3-mink python=3.8
conda activate py3-mink

conda install openblas-devel -c anaconda
conda install pytorch=1.9.0 torchvision cudatoolkit=11.1 -c pytorch -c nvidia

# Install MinkowskiEngine

# Uncomment the following line to specify the cuda home. Make sure `$CUDA_HOME/nvcc --version` is 11.X
# export CUDA_HOME=/usr/local/cuda-11.1
pip install -U git+https://github.com/NVIDIA/MinkowskiEngine -v --no-deps --install-option="--blas_include_dirs=${CONDA_PREFIX}/include" --install-option="--blas=openblas"

# Or if you want local MinkowskiEngine
git clone https://github.com/NVIDIA/MinkowskiEngine.git
cd MinkowskiEngine
python setup.py install --blas_include_dirs=${CONDA_PREFIX}/include --blas=openblas

My PC spec :

Linux Version : 

lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 23.04
Release:    23.04
Codename:   lunar

CPU : 

Vendor ID:               AuthenticAMD
  Model name:            AMD Ryzen 9 5900X 12-Core Processor

Ram : 64 GBs

also have the same problem

Here is the error : 

/home/blinkdrive/anaconda3/envs/py3-mink/lib/python3.8/site-packages/torch/include/ATen/core/jit_type.h(615): error: no operator "=" matches these operands
                  operand types are: c10::optional<__nv_bool> = __nv_bool

      /home/blinkdrive/anaconda3/envs/py3-mink/lib/python3.8/site-packages/torch/include/ATen/core/jit_type.h(1060): error: no instance of constructor "c10::NamedType::NamedType" matches the argument list
                  argument types are: (c10::TypeKind, c10::QualifiedName)

      /home/blinkdrive/anaconda3/envs/py3-mink/lib/python3.8/site-packages/torch/include/ATen/core/jit_type.h(1503): error: no instance of overloaded function "c10::TensorType::create" matches the argument list
                  argument types are: (const c10::nullopt_t, const c10::DeviceType, {...}, const c10::nullopt_t)

      /home/blinkdrive/anaconda3/envs/py3-mink/lib/python3.8/site-packages/torch/include/ATen/core/jit_type.h(1513): error: no suitable constructor exists to convert from "c10::ScalarType" to "c10::optional<c10::ScalarType>"

      /home/blinkdrive/anaconda3/envs/py3-mink/lib/python3.8/site-packages/torch/include/ATen/core/jit_type.h(1515): error: no suitable constructor exists to convert from "c10::ScalarType" to "c10::optional<c10::ScalarType>"

      /home/blinkdrive/anaconda3/envs/py3-mink/lib/python3.8/site-packages/torch/include/ATen/core/jit_type.h(1517): error: no suitable constructor exists to convert from "c10::ScalarType" to "c10::optional<c10::ScalarType>"

      /home/blinkdrive/anaconda3/envs/py3-mink/lib/python3.8/site-packages/torch/include/ATen/core/jit_type.h(1746): error: no instance of constructor "c10::optional<T>::optional [with T=std::string]" matches the argument list
                  argument types are: (std::string)

      /home/blinkdrive/anaconda3/envs/py3-mink/lib/python3.8/site-packages/torch/include/ATen/core/jit_type.h(1915): error: no suitable constructor exists to convert from "size_t" to "c10::optional<size_t>"

      /home/blinkdrive/anaconda3/envs/py3-mink/lib/python3.8/site-packages/torch/include/ATen/core/jit_type.h(2021): error: no suitable constructor exists to convert from "size_t" to "c10::optional<size_t>"

      /home/blinkdrive/anaconda3/envs/py3-mink/lib/python3.8/site-packages/torch/include/c10/util/TypeList.h(183): error: class "c10::guts::typelist::detail::contains<c10::impl::valid_dict_key_types, c10::IValue, void>" has no member "type"
                detected during:
                  instantiation of type "c10::guts::typelist::contains<c10::impl::valid_dict_key_types, c10::IValue>"
      /home/blinkdrive/anaconda3/envs/py3-mink/lib/python3.8/site-packages/torch/include/ATen/core/Dict.h(203): here
                  instantiation of class "c10::Dict<Key, Value> [with Key=c10::IValue, Value=c10::IValue]"
      /home/blinkdrive/anaconda3/envs/py3-mink/lib/python3.8/site-packages/torch/include/ATen/core/ivalue_inl.h(416): here

      /home/blinkdrive/anaconda3/envs/py3-mink/lib/python3.8/site-packages/torch/include/ATen/core/Dict.h(203): error: name followed by "::" must be a class or namespace name
                detected during instantiation of class "c10::Dict<Key, Value> [with Key=c10::IValue, Value=c10::IValue]"
      /home/blinkdrive/anaconda3/envs/py3-mink/lib/python3.8/site-packages/torch/include/ATen/core/ivalue_inl.h(416): here

      /home/blinkdrive/anaconda3/envs/py3-mink/lib/python3.8/site-packages/torch/include/ATen/core/ivalue_inl.h(499): error: no suitable user-defined conversion from "lambda [](c10::ivalue::Future &)->void" to "std::function<void (c10::ivalue::Future &)>" exists

      /home/blinkdrive/anaconda3/envs/py3-mink/lib/python3.8/site-packages/torch/include/ATen/core/ivalue_inl.h(1492): error: no instance of constructor "std::reference_wrapper<_Tp>::reference_wrapper [with _Tp=const std::string]" matches the argument list
                  argument types are: (const std::string)

      /home/blinkdrive/anaconda3/envs/py3-mink/lib/python3.8/site-packages/torch/include/c10/util/Optional.h(1116): error: no instance of constructor "c10::optional<T>::optional [with T=c10::string_view]" matches the argument list
                  argument types are: (c10::string_view)
                detected during instantiation of "c10::optional<std::decay<T>::type> c10::make_optional(T &&) [with T=c10::string_view]"
      /home/blinkdrive/anaconda3/envs/py3-mink/lib/python3.8/site-packages/torch/include/ATen/core/operator_name.h(31): here

      /home/blinkdrive/anaconda3/envs/py3-mink/lib/python3.8/site-packages/torch/include/ATen/Functions.h(471): error: no suitable constructor exists to convert from "c10::MemoryFormat" to "c10::optional<c10::MemoryFormat>"

      /home/blinkdrive/anaconda3/envs/py3-mink/lib/python3.8/site-packages/torch/include/ATen/Functions.h(473): error: no suitable constructor exists to convert from "c10::MemoryFormat" to "c10::optional<c10::MemoryFormat>"

      /home/blinkdrive/anaconda3/envs/py3-mink/lib/python3.8/site-packages/torch/include/ATen/Functions.h(536): error: no suitable constructor exists to convert from "int" to "c10::optional<int64_t>"

      /home/blinkdrive/anaconda3/envs/py3-mink/lib/python3.8/site-packages/torch/include/c10/util/ArrayRef.h(212): error: class "std::enable_if<<error-constant>, c10::ArrayRef<int64_t>>" has no member "type"
                detected during:
                  instantiation of "c10::ArrayRef<T>::operator= [with T=int64_t]" based on template argument <c10::optional<c10::IntArrayRef> &>
      /usr/include/c++/11/type_traits(1087): here
                  instantiation of class "std::is_assignable<_Tp, _Up> [with _Tp=c10::ArrayRef<int64_t> &, _Up=c10::optional<c10::IntArrayRef> &]"
      /home/blinkdrive/anaconda3/envs/py3-mink/lib/python3.8/site-packages/torch/include/ATen/Functions.h(2468): here

      /home/blinkdrive/anaconda3/envs/py3-mink/lib/python3.8/site-packages/torch/include/c10/util/Optional.h(585): error: class "std::enable_if<<error-constant>, c10::optional<c10::IntArrayRef> &>" has no member "type"
                detected during instantiation of "c10::optional<T>::operator= [with T=c10::IntArrayRef]" based on template argument <c10::optional<c10::IntArrayRef> &>
      /home/blinkdrive/anaconda3/envs/py3-mink/lib/python3.8/site-packages/torch/include/ATen/Functions.h(2468): here

      /home/blinkdrive/anaconda3/envs/py3-mink/lib/python3.8/site-packages/torch/include/ATen/Functions.h(2533): error: no suitable user-defined conversion from "c10::IntArrayRef" to "c10::optional<c10::IntArrayRef>" exists

      Error limit reached.
      100 errors detected in the compilation of "/tmp/pip-req-build-y03m21d6/src/global_pooling_gpu.cu".
      Compilation terminated.
      [12/21] c++ -MMD -MF /tmp/pip-req-build-y03m21d6/build/temp.linux-x86_64-cpython-38/tmp/pip-req-build-y03m21d6/src/direct_max_pool.o.d -pthread -B /home/blinkdrive/anaconda3/envs/py3-mink/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/blinkdrive/anaconda3/envs/py3-mink/lib/python3.8/site-packages/torch/include -I/home/blinkdrive/anaconda3/envs/py3-mink/lib/python3.8/site-packages/torch/include/torch/csrc/api/include -I/home/blinkdrive/anaconda3/envs/py3-mink/lib/python3.8/site-packages/torch/include/TH -I/home/blinkdrive/anaconda3/envs/py3-mink/lib/python3.8/site-packages/torch/include/THC -I/tmp/pip-req-build-y03m21d6/src -I/tmp/pip-req-build-y03m21d6/src/3rdparty -I/home/blinkdrive/anaconda3/envs/py3-mink/include/python3.8 -c -c /tmp/pip-req-build-y03m21d6/src/direct_max_pool.cpp -o /tmp/pip-req-build-y03m21d6/build/temp.linux-x86_64-cpython-38/tmp/pip-req-build-y03m21d6/src/direct_max_pool.o -fopenmp -O3 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++14
      cc1plus: warning: command-line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
      ninja: build stopped: subcommand failed.
      Traceback (most recent call last):
        File "/home/blinkdrive/anaconda3/envs/py3-mink/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 1666, in _run_ninja_build
          subprocess.run(
        File "/home/blinkdrive/anaconda3/envs/py3-mink/lib/python3.8/subprocess.py", line 516, in run
          raise CalledProcessError(retcode, process.args,
      subprocess.CalledProcessError: Command '['ninja', '-v', '-j', '12']' returned non-zero exit status 1.

      The above exception was the direct cause of the following exception:

      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/tmp/pip-req-build-y03m21d6/setup.py", line 320, in <module>
          setup(
        File "/home/blinkdrive/anaconda3/envs/py3-mink/lib/python3.8/site-packages/setuptools/__init__.py", line 103, in setup
          return distutils.core.setup(**attrs)
        File "/home/blinkdrive/anaconda3/envs/py3-mink/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 185, in setup
          return run_commands(dist)
        File "/home/blinkdrive/anaconda3/envs/py3-mink/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
          dist.run_commands()
        File "/home/blinkdrive/anaconda3/envs/py3-mink/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
          self.run_command(cmd)
        File "/home/blinkdrive/anaconda3/envs/py3-mink/lib/python3.8/site-packages/setuptools/dist.py", line 989, in run_command
          super().run_command(command)
        File "/home/blinkdrive/anaconda3/envs/py3-mink/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
          cmd_obj.run()
        File "/home/blinkdrive/anaconda3/envs/py3-mink/lib/python3.8/site-packages/wheel/bdist_wheel.py", line 364, in run
          self.run_command("build")
        File "/home/blinkdrive/anaconda3/envs/py3-mink/lib/python3.8/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
          self.distribution.run_command(command)
        File "/home/blinkdrive/anaconda3/envs/py3-mink/lib/python3.8/site-packages/setuptools/dist.py", line 989, in run_command
          super().run_command(command)
        File "/home/blinkdrive/anaconda3/envs/py3-mink/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
          cmd_obj.run()
        File "/home/blinkdrive/anaconda3/envs/py3-mink/lib/python3.8/site-packages/setuptools/_distutils/command/build.py", line 131, in run
          self.run_command(cmd_name)
        File "/home/blinkdrive/anaconda3/envs/py3-mink/lib/python3.8/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
          self.distribution.run_command(command)
        File "/home/blinkdrive/anaconda3/envs/py3-mink/lib/python3.8/site-packages/setuptools/dist.py", line 989, in run_command
          super().run_command(command)
        File "/home/blinkdrive/anaconda3/envs/py3-mink/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
          cmd_obj.run()
        File "/home/blinkdrive/anaconda3/envs/py3-mink/lib/python3.8/site-packages/setuptools/command/build_ext.py", line 88, in run
          _build_ext.run(self)
        File "/home/blinkdrive/anaconda3/envs/py3-mink/lib/python3.8/site-packages/setuptools/_distutils/command/build_ext.py", line 345, in run
          self.build_extensions()
        File "/home/blinkdrive/anaconda3/envs/py3-mink/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 709, in build_extensions
          build_ext.build_extensions(self)
        File "/home/blinkdrive/anaconda3/envs/py3-mink/lib/python3.8/site-packages/setuptools/_distutils/command/build_ext.py", line 467, in build_extensions
          self._build_extensions_serial()
        File "/home/blinkdrive/anaconda3/envs/py3-mink/lib/python3.8/site-packages/setuptools/_distutils/command/build_ext.py", line 493, in _build_extensions_serial
          self.build_extension(ext)
        File "/home/blinkdrive/anaconda3/envs/py3-mink/lib/python3.8/site-packages/setuptools/command/build_ext.py", line 249, in build_extension
          _build_ext.build_extension(self, ext)
        File "/home/blinkdrive/anaconda3/envs/py3-mink/lib/python3.8/site-packages/setuptools/_distutils/command/build_ext.py", line 548, in build_extension
          objects = self.compiler.compile(
        File "/home/blinkdrive/anaconda3/envs/py3-mink/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 530, in unix_wrap_ninja_compile
          _write_ninja_file_and_compile_objects(
        File "/home/blinkdrive/anaconda3/envs/py3-mink/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 1355, in _write_ninja_file_and_compile_objects
          _run_ninja_build(
        File "/home/blinkdrive/anaconda3/envs/py3-mink/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 1682, in _run_ninja_build
          raise RuntimeError(message) from e
      RuntimeError: Error compiling objects for extension
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for MinkowskiEngine
  Running setup.py clean for MinkowskiEngine
Failed to build MinkowskiEngine
ERROR: Could not build wheels for MinkowskiEngine, which is required to install pyproject.toml-based projects

My pip list are 

Package           Version
----------------- -------
mkl-fft           1.3.8
mkl-random        1.2.4
mkl-service       2.4.0
numpy             1.24.3
Pillow            9.3.0
pip               23.3.1
setuptools        68.2.2
torch             1.9.0
torchvision       0.10.0
typing_extensions 4.7.1
wheel             0.41.2

Conda library list :

(py3-mink) blinkdrive@blinkdrive-System-Product-Name:~/Documents/UNT/Project/Human3D/MinkowskiEngine$ conda list
# packages in environment at /home/blinkdrive/anaconda3/envs/py3-mink:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                        main  
_openmp_mutex             5.1                       1_gnu  
blas                      1.0                         mkl  
bzip2                     1.0.8                h7b6447c_0  
ca-certificates           2023.12.12           h06a4308_0  
cudatoolkit               11.1.74              h6bb024c_0    nvidia
ffmpeg                    4.3                  hf484d3e_0    pytorch
freetype                  2.12.1               h4a9f257_0  
giflib                    5.2.1                h5eee18b_3  
gmp                       6.2.1                h295c915_3  
gnutls                    3.6.15               he1e5248_0  
intel-openmp              2023.1.0         hdb19cb5_46306  
jpeg                      9b                   h024ee3a_2  
lame                      3.100                h7b6447c_0  
lcms2                     2.12                 h3be6417_0  
ld_impl_linux-64          2.38                 h1181459_1  
libffi                    3.4.4                h6a678d5_0  
libgcc-ng                 11.2.0               h1234567_1  
libgfortran-ng            8.4.0               he6ba991_17  
libgfortran5              8.4.0               he6ba991_17  
libgomp                   11.2.0               h1234567_1  
libiconv                  1.16                 h7f8727e_2  
libidn2                   2.3.4                h5eee18b_0  
libopenblas               0.3.2                h9ac9557_1  
libpng                    1.6.39               h5eee18b_0  
libstdcxx-ng              11.2.0               h1234567_1  
libtasn1                  4.19.0               h5eee18b_0  
libtiff                   4.2.0                h85742a9_0  
libunistring              0.9.10               h27cfd23_0  
libuv                     1.44.2               h5eee18b_0  
libwebp                   1.2.0                h89dd481_0  
libwebp-base              1.2.0                h27cfd23_0  
lz4-c                     1.9.4                h6a678d5_0  
mkl                       2023.1.0         h213fc3f_46344  
mkl-service               2.4.0            py38h5eee18b_1  
mkl_fft                   1.3.8            py38h5eee18b_0  
mkl_random                1.2.4            py38hdb19cb5_0  
ncurses                   6.4                  h6a678d5_0  
nettle                    3.7.3                hbbd107a_1  
ninja                     1.10.2               h06a4308_5  
ninja-base                1.10.2               hd09550d_5  
nomkl                     2.0                           0    anaconda
numpy                     1.24.3           py38hf6e8229_1  
numpy-base                1.24.3           py38h060ed82_1  
openblas-devel            0.3.2                         0    anaconda
openh264                  2.1.1                h4ff587b_0  
openssl                   3.0.12               h7f8727e_0  
pillow                    9.3.0            py38hace64e9_1  
pip                       23.3.1           py38h06a4308_0  
python                    3.8.18               h955ad1f_0  
pytorch                   1.9.0           py3.8_cuda11.1_cudnn8.0.5_0    pytorch
readline                  8.2                  h5eee18b_0  
setuptools                68.2.2           py38h06a4308_0  
sqlite                    3.41.2               h5eee18b_0  
tbb                       2021.8.0             hdb19cb5_0  
tk                        8.6.12               h1ccaba5_0  
torchvision               0.10.0               py38_cu111    pytorch
typing_extensions         4.7.1            py38h06a4308_0  
wheel                     0.41.2           py38h06a4308_0  
xz                        5.4.5                h5eee18b_0  
zlib                      1.2.13               h5eee18b_0  
zstd                      1.4.9                haebb681_0

gcc -version : 

gcc (Ubuntu 9.5.0-3ubuntu1) 9.5.0
Copyright (C) 2019 Free Software Foundation, Inc.

CUDA Version : 11.3

mary-ashk commented 10 months ago

I solved the ninja error with edit the setup.py. Ubuntu 18.04.5 Pytorch 1.10.0 Cuda11.3.1 gcc 7.5.0 g++ 7.5.0

error: subprocess.CalledProcessError: Command '['ninja', '-v', '-j', '12']' returned non-zero exit status 1.

Open the setup.py file and edit this line (cmdclass={"build_ext" :BuildExtension.with_options(use_ninja=False)} ) to False mode. it takes almost long time , but finally will be installed:)

Also check the gcc and g++ versions to be match.

fenfenglitech commented 10 months ago
image

you could have a try!

bring-nirachornkul commented 10 months ago

image you could have a try!

I did this before as well. Nothing change. What is your Linux OS version? 18.04 or 20.04?