ashawkey / stable-dreamfusion

Text-to-3D & Image-to-3D & Mesh Exportation with NeRF + Diffusion.
Apache License 2.0
7.99k stars 710 forks source link

Raymarching fails to install, cant find c++17 for PyTorch, SOS help #358

Open SubhamBurnwal-BiswaGames opened 2 months ago

SubhamBurnwal-BiswaGames commented 2 months ago

Description

Tried setting up this project on local machine but cannot install Raymarching, and other extensions due to various errors. I have solved some step by step but stuck right now.

Steps to Reproduce

I am following the Instant-NGP backbone collab notebook provided in README.md, locally on my machine: https://colab.research.google.com/drive/1MXT3yfOFvO0ooKEfiUUvTKwUkrrlCHpF?usp=sharing

  1. Created new conda environment starting with python latest.
  2. Loaded the above colab file with this environment
  3. It showed I need to install ipykernel to run the notebook in this environment, so I did: ! conda install ipykernel
  4. Installed Pytorch ! conda install pytorch torchvision torchaudio pytorch-cuda=11.8 -c pytorch -c nvidia
  5. Checked installation by importing torch:

image image

  1. git-cloned the repository as given in the notebook.

  2. once inside the repo, I commented out "torch" since we already installed above (this was suggested in another issue solution here)

  3. I install remaining dependencies: ! pip install -r requirements.txt

  4. Everything installed except PyMCubes. The error was it could not find VS C++17 to build. It was suggested online that I dont need to build it just install using this, so I did: ! pip install PyMCubes --no-cache-dir

  5. Now I reached this portion of building/installing extensions: The first extension was raymarching ! pip install ./raymarching I got this error: "Could not locate a supported Microsoft Visual C++ installation"

  6. I tracked this error to the file: stable-dreamfusion/raymarching/setup.py

    
    import os
    from setuptools import setup
    from torch.utils.cpp_extension import BuildExtension, CUDAExtension

_src_path = os.path.dirname(os.path.abspath(file))

nvcc_flags = [ '-O3', '-std=c++14', '-UCUDA_NO_HALF_OPERATORS', '-UCUDA_NO_HALF_CONVERSIONS', '-U__CUDA_NO_HALF2_OPERATORS__', ]

if os.name == "posix": c_flags = ['-O3', '-std=c++14'] elif os.name == "nt": c_flags = ['/O2', '/std:c++17']

# find cl.exe
def find_cl_path():
    import glob
    for edition in ["Enterprise", "Professional", "BuildTools", "Community"]:
        paths = sorted(glob.glob(r"C:\\Program Files x86\\Microsoft Visual Studio\\*\\%s\\VC\\Tools\\MSVC\\*\\bin\\Hostx64\\x64" % edition), reverse=True)
        if paths:
            return paths[0]

# If cl.exe is not on path, try to find it.
if os.system("where cl.exe >nul 2>nul") != 0:
    cl_path = find_cl_path()
    if cl_path is None:
        raise RuntimeError("Could not locate a supported Microsoft Visual C++ installation")
    os.environ["PATH"] += ";" + cl_path

Now my visual studio files were not on the path "C:\\Program Files\\Microsoft Visual Studio..." instead of "C:\\Program Files x86", I might be an idiot here but I removed the x86 from the path this setup script was searching, so that error went away.
But now I get this error log:

Processing d:\python\stable-dreamfusion\raymarching Preparing metadata (setup.py): started Preparing metadata (setup.py): finished with status 'done' Building wheels for collected packages: raymarching Building wheel for raymarching (setup.py): started Building wheel for raymarching (setup.py): still running... Building wheel for raymarching (setup.py): finished with status 'error' Running setup.py clean for raymarching Failed to build raymarching error: subprocess-exited-with-error

× python setup.py bdist_wheel did not run successfully. │ exit code: 1 ╰─> [100 lines of output] running bdist_wheel running build running build_ext building '_raymarching' extension creating D:\Python\stable-dreamfusion\raymarching\build creating D:\Python\stable-dreamfusion\raymarching\build\temp.win-amd64-cpython-312 creating D:\Python\stable-dreamfusion\raymarching\build\temp.win-amd64-cpython-312\Release creating D:\Python\stable-dreamfusion\raymarching\build\temp.win-amd64-cpython-312\Release\Python creating D:\Python\stable-dreamfusion\raymarching\build\temp.win-amd64-cpython-312\Release\Python\stable-dreamfusion creating D:\Python\stable-dreamfusion\raymarching\build\temp.win-amd64-cpython-312\Release\Python\stable-dreamfusion\raymarching creating D:\Python\stable-dreamfusion\raymarching\build\temp.win-amd64-cpython-312\Release\Python\stable-dreamfusion\raymarching\src C:\Users\BG.conda\envs\sdf1\Lib\site-packages\torch\utils\cpp_extension.py:1967: UserWarning: TORCH_CUDA_ARCH_LIST is not set, all archs for visible cards are included for compilation. If this is not desired, please set os.environ['TORCH_CUDA_ARCH_LIST']. warnings.warn( Emitting ninja build file D:\Python\stable-dreamfusion\raymarching\build\temp.win-amd64-cpython-312\Release\build.ninja... Compiling objects... Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N) [1/2] C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8\bin\nvcc --generate-dependencies-with-compile --dependency-output D:\Python\stable-dreamfusion\raymarching\build\temp.win-amd64-cpython-312\Release\Python\stable-dreamfusion\raymarching\src\raymarching.obj.d -std=c++17 --use-local-env -Xcompiler /MD -Xcompiler /wd4819 -Xcompiler /wd4251 -Xcompiler /wd4244 -Xcompiler /wd4267 -Xcompiler /wd4275 -Xcompiler /wd4018 -Xcompiler /wd4190 -Xcompiler /wd4624 -Xcompiler /wd4067 -Xcompiler /wd4068 -Xcompiler /EHsc -Xcudafe --diag_suppress=base_class_has_different_dll_interface -Xcudafe --diag_suppress=field_without_dll_interface -Xcudafe --diag_suppress=dll_interface_conflict_none_assumed -Xcudafe --diag_suppress=dll_interface_conflict_dllexport_assumed -IC:\Users\BG.conda\envs\sdf1\Lib\site-packages\torch\include -IC:\Users\BG.conda\envs\sdf1\Lib\site-packages\torch\include\torch\csrc\api\include -IC:\Users\BG.conda\envs\sdf1\Lib\site-packages\torch\include\TH -IC:\Users\BG.conda\envs\sdf1\Lib\site-packages\torch\include\THC "-IC:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8\include" -IC:\Users\BG.conda\envs\sdf1\include -IC:\Users\BG.conda\envs\sdf1\Include "-IC:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.39.33519\include" "-IC:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.39.33519\ATLMFC\include" "-IC:\Program Files\Microsoft Visual Studio\2022\Community\VC\Auxiliary\VS\include" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\shared" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\winrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\cppwinrt" "-IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.8\include\um" -c D:\Python\stable-dreamfusion\raymarching\src\raymarching.cu -o D:\Python\stable-dreamfusion\raymarching\build\temp.win-amd64-cpython-312\Release\Python\stable-dreamfusion\raymarching\src\raymarching.obj -DCUDA_NO_HALF_OPERATORS -DCUDA_NO_HALF_CONVERSIONS -DCUDA_NO_BFLOAT16_CONVERSIONS -DCUDA_NO_HALF2_OPERATORS --expt-relaxed-constexpr -O3 -std=c++14 -UCUDA_NO_HALF_OPERATORS -UCUDA_NO_HALF_CONVERSIONS -UCUDA_NO_HALF2_OPERATORS__ -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=_raymarching -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_75,code=compute_75 -gencode=arch=compute_75,code=sm_75 FAILED: D:/Python/stable-dreamfusion/raymarching/build/temp.win-amd64-cpython-312/Release/Python/stable-dreamfusion/raymarching/src/raymarching.obj C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8\bin\nvcc --generate-dependencies-with-compile --dependency-output D:\Python\stable-dreamfusion\raymarching\build\temp.win-amd64-cpython-312\Release\Python\stable-dreamfusion\raymarching\src\raymarching.obj.d -std=c++17 --use-local-env -Xcompiler /MD -Xcompiler /wd4819 -Xcompiler /wd4251 -Xcompiler /wd4244 -Xcompiler /wd4267 -Xcompiler /wd4275 -Xcompiler /wd4018 -Xcompiler /wd4190 -Xcompiler /wd4624 -Xcompiler /wd4067 -Xcompiler /wd4068 -Xcompiler /EHsc -Xcudafe --diag_suppress=base_class_has_different_dll_interface -Xcudafe --diag_suppress=field_without_dll_interface -Xcudafe --diag_suppress=dll_interface_conflict_none_assumed -Xcudafe --diag_suppress=dll_interface_conflict_dllexport_assumed -IC:\Users\BG.conda\envs\sdf1\Lib\site-packages\torch\include -IC:\Users\BG.conda\envs\sdf1\Lib\site-packages\torch\include\torch\csrc\api\include -IC:\Users\BG.conda\envs\sdf1\Lib\site-packages\torch\include\TH -IC:\Users\BG.conda\envs\sdf1\Lib\site-packages\torch\include\THC "-IC:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8\include" -IC:\Users\BG.conda\envs\sdf1\include -IC:\Users\BG.conda\envs\sdf1\Include "-IC:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.39.33519\include" "-IC:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.39.33519\ATLMFC\include" "-IC:\Program Files\Microsoft Visual Studio\2022\Community\VC\Auxiliary\VS\include" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\shared" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\winrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\cppwinrt" "-IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.8\include\um" -c D:\Python\stable-dreamfusion\raymarching\src\raymarching.cu -o D:\Python\stable-dreamfusion\raymarching\build\temp.win-amd64-cpython-312\Release\Python\stable-dreamfusion\raymarching\src\raymarching.obj -DCUDA_NO_HALF_OPERATORS -DCUDA_NO_HALF_CONVERSIONS -DCUDA_NO_BFLOAT16_CONVERSIONS -DCUDA_NO_HALF2_OPERATORS --expt-relaxed-constexpr -O3 -std=c++14 -UCUDA_NO_HALF_OPERATORS -UCUDA_NO_HALF_CONVERSIONS -UCUDA_NO_HALF2_OPERATORS -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=_raymarching -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_75,code=compute_75 -gencode=arch=compute_75,code=sm_75 cl : Command line warning D9025 : overriding '/D__CUDA_NO_HALF_OPERATORS' with '/UCUDA_NO_HALF_OPERATORS' cl : Command line warning D9025 : overriding '/DCUDA_NO_HALF_CONVERSIONS' with '/UCUDA_NO_HALF_CONVERSIONS' cl : Command line warning D9025 : overriding '/DCUDA_NO_HALF2_OPERATORS' with '/U__CUDA_NO_HALF2_OPERATORS__'

  raymarching.cu

  C:/Users/BG/.conda/envs/sdf1/Lib/site-packages/torch/include\c10/util/C++17.h(24): fatal error C1189: #error:  You need C++17 to compile PyTorch

  nvcc warning : incompatible redefinition for option 'std', the last value of this option was used

  raymarching.cu

  [2/2] cl /showIncludes /nologo /O2 /W3 /GL /DNDEBUG /MD /MD /wd4819 /wd4251 /wd4244 /wd4267 /wd4275 /wd4018 /wd4190 /wd4624 /wd4067 /wd4068 /EHsc -IC:\Users\BG\.conda\envs\sdf1\Lib\site-packages\torch\include -IC:\Users\BG\.conda\envs\sdf1\Lib\site-packages\torch\include\torch\csrc\api\include -IC:\Users\BG\.conda\envs\sdf1\Lib\site-packages\torch\include\TH -IC:\Users\BG\.conda\envs\sdf1\Lib\site-packages\torch\include\THC "-IC:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8\include" -IC:\Users\BG\.conda\envs\sdf1\include -IC:\Users\BG\.conda\envs\sdf1\Include "-IC:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.39.33519\include" "-IC:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.39.33519\ATLMFC\include" "-IC:\Program Files\Microsoft Visual Studio\2022\Community\VC\Auxiliary\VS\include" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\um" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\shared" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\winrt" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\cppwinrt" "-IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.8\include\um" -c D:\Python\stable-dreamfusion\raymarching\src\bindings.cpp /FoD:\Python\stable-dreamfusion\raymarching\build\temp.win-amd64-cpython-312\Release\Python\stable-dreamfusion\raymarching\src\bindings.obj /O2 /std:c++17 -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=_raymarching -D_GLIBCXX_USE_CXX11_ABI=0
  ninja: build stopped: subcommand failed.
  Traceback (most recent call last):
    File "C:\Users\BG\.conda\envs\sdf1\Lib\site-packages\torch\utils\cpp_extension.py", line 2107, in _run_ninja_build
      subprocess.run(
    File "C:\Users\BG\.conda\envs\sdf1\Lib\subprocess.py", line 571, in run
      raise CalledProcessError(retcode, process.args,
  subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

  The above exception was the direct cause of the following exception:

  Traceback (most recent call last):
    File "<string>", line 2, in <module>
    File "<pip-setuptools-caller>", line 34, in <module>
    File "D:\Python\stable-dreamfusion\raymarching\setup.py", line 44, in <module>
      setup(
    File "C:\Users\BG\.conda\envs\sdf1\Lib\site-packages\setuptools\__init__.py", line 103, in setup
      return distutils.core.setup(**attrs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "C:\Users\BG\.conda\envs\sdf1\Lib\site-packages\setuptools\_distutils\core.py", line 185, in setup
      return run_commands(dist)
             ^^^^^^^^^^^^^^^^^^
    File "C:\Users\BG\.conda\envs\sdf1\Lib\site-packages\setuptools\_distutils\core.py", line 201, in run_commands
      dist.run_commands()
    File "C:\Users\BG\.conda\envs\sdf1\Lib\site-packages\setuptools\_distutils\dist.py", line 969, in run_commands
      self.run_command(cmd)
    File "C:\Users\BG\.conda\envs\sdf1\Lib\site-packages\setuptools\dist.py", line 989, in run_command
      super().run_command(command)
    File "C:\Users\BG\.conda\envs\sdf1\Lib\site-packages\setuptools\_distutils\dist.py", line 988, in run_command
      cmd_obj.run()
    File "C:\Users\BG\.conda\envs\sdf1\Lib\site-packages\wheel\bdist_wheel.py", line 368, in run
      self.run_command("build")
    File "C:\Users\BG\.conda\envs\sdf1\Lib\site-packages\setuptools\_distutils\cmd.py", line 318, in run_command
      self.distribution.run_command(command)
    File "C:\Users\BG\.conda\envs\sdf1\Lib\site-packages\setuptools\dist.py", line 989, in run_command
      super().run_command(command)
    File "C:\Users\BG\.conda\envs\sdf1\Lib\site-packages\setuptools\_distutils\dist.py", line 988, in run_command
      cmd_obj.run()
    File "C:\Users\BG\.conda\envs\sdf1\Lib\site-packages\setuptools\_distutils\command\build.py", line 131, in run
      self.run_command(cmd_name)
    File "C:\Users\BG\.conda\envs\sdf1\Lib\site-packages\setuptools\_distutils\cmd.py", line 318, in run_command
      self.distribution.run_command(command)
    File "C:\Users\BG\.conda\envs\sdf1\Lib\site-packages\setuptools\dist.py", line 989, in run_command
      super().run_command(command)
    File "C:\Users\BG\.conda\envs\sdf1\Lib\site-packages\setuptools\_distutils\dist.py", line 988, in run_command
      cmd_obj.run()
    File "C:\Users\BG\.conda\envs\sdf1\Lib\site-packages\setuptools\command\build_ext.py", line 88, in run
      _build_ext.run(self)
    File "C:\Users\BG\.conda\envs\sdf1\Lib\site-packages\setuptools\_distutils\command\build_ext.py", line 345, in run
      self.build_extensions()
    File "C:\Users\BG\.conda\envs\sdf1\Lib\site-packages\torch\utils\cpp_extension.py", line 870, in build_extensions
      build_ext.build_extensions(self)
    File "C:\Users\BG\.conda\envs\sdf1\Lib\site-packages\setuptools\_distutils\command\build_ext.py", line 467, in build_extensions
      self._build_extensions_serial()
    File "C:\Users\BG\.conda\envs\sdf1\Lib\site-packages\setuptools\_distutils\command\build_ext.py", line 493, in _build_extensions_serial
      self.build_extension(ext)
    File "C:\Users\BG\.conda\envs\sdf1\Lib\site-packages\setuptools\command\build_ext.py", line 249, in build_extension
      _build_ext.build_extension(self, ext)
    File "C:\Users\BG\.conda\envs\sdf1\Lib\site-packages\setuptools\_distutils\command\build_ext.py", line 548, in build_extension
      objects = self.compiler.compile(
                ^^^^^^^^^^^^^^^^^^^^^^
    File "C:\Users\BG\.conda\envs\sdf1\Lib\site-packages\torch\utils\cpp_extension.py", line 842, in win_wrap_ninja_compile
      _write_ninja_file_and_compile_objects(
    File "C:\Users\BG\.conda\envs\sdf1\Lib\site-packages\torch\utils\cpp_extension.py", line 1783, in _write_ninja_file_and_compile_objects
      _run_ninja_build(
    File "C:\Users\BG\.conda\envs\sdf1\Lib\site-packages\torch\utils\cpp_extension.py", line 2123, in _run_ninja_build
      raise RuntimeError(message) from e
  RuntimeError: Error compiling objects for extension
  [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip. ERROR: Failed building wheel for raymarching ERROR: Could not build wheels for raymarching, which is required to install pyproject.toml-based projects



I have already installed MSVC++2017 from Visual studio installer. I also have gcc10 in path if the setup could alternatively use that but it still all fails. Please help. My job depends on this!

### Expected Behavior

Successfully installed Raymarching

### Environment

OS: Windows 11
CUDA Version supported: 12.2

It was suggested in other issues to use 11.8 etc since 12+ versions are not compatibale for this project so I installed that.

CUDA version I installed: cuda_11.8.0_522.06_windows.exe from nvidia site
PyTorch I installed as per instructions for this CUDA version:
! conda install pytorch torchvision torchaudio pytorch-cuda=11.8 -c pytorch -c nvidia
Duodecimus commented 1 day ago

In setup.py there are two places it has c++14. change these to c++17

This lets me compile it with python setup.py build_ext --inplace, but attempting to install the build with pip install . fails during wheel building with ModuleNotFoundError: No module named 'torch'

so your mileage may vary with this advice