search

facebookresearch / pytorch3d

PyTorch3D is FAIR's library of reusable components for deep learning with 3D data
https://pytorch3d.org/
Other
8.7k stars 1.3k forks source link

Error: Not compiled with GPU support on Windows (nvcc installed) #1062

Closed GiBucci closed 2 years ago

GiBucci commented 2 years ago

Hi.

I am having trouble installing pytorch3d library on windows. I know it can be tricky, but I have only this solution at the moment.

I am sorry in advance if this is a duplicate question, or something trivial, but I spent quite some time looking at the other issues and I didn't find a solution.

I followed the INSTALL.md guide, created a new conda environment, installed all the necessary libraries, and then installed from source with "pip install "git+https://github.com/facebookresearch/pytorch3d.git". It isn't specified if this works for windows too, but I tried and it seems it's fine anyway.

I encountered some troubles in importing the _C library but I tried installing Visual Studio tools and I managed to make it work anyway (I tried many things so I can't exactly tell you which steps I took, sorry).

I was finally ready to launch some demo tutorials, but when running the "deform_source_mesh_to_target_mesh.py" exercise, I find the error in the title, namely "RuntimeError: Not compiled with GPU support." when calling the underlying _C functions.

I saw that it might be a CUDA compiler error, so I downloaded all the tools, like nvcc. Now it's recognized correctly by the terminal, but it is not working. I also tried setting some environment variables, like CUDA_HOME, pointing at the nvcc directory, but either it is not working or I'm doing something wrong.

I didn't apply any patch to the headers yet, it was hard to find the instructions, but it seems that those changes are answering another range of problems that I didn't encounter (yet).

It would be very appreciated if you could suggest me something in order to try and solve this matter. Thank you in advance for your kindness and your consideration.

Here you can find some details on my environment. Let me know if I can provide additional information.

OS: Windows 11 NVIDIA RTX 3060

bottler commented 2 years ago

I would clone this repository locally so and try to pip install . so that it doesn't have to download each time and you can make changes. Your problem I think is that PyTorch3D has done a CPU build and that's not what you want. For whether to build the CUDA parts of pytorch3d, you can see the condition here in setup.py. It is if (torch.cuda.is_available() and CUDA_HOME is not None) or force_cuda:.

You can add code to check your expectations before that line, e.g.:

assert CUDA_HOME is not None
assert torch.cuda.is_available()
GiBucci commented 2 years ago

Hi,

Thank you very much for your time and your help.

I figured that as well, because probably I messed up some installation order.

Now I uninstalled pytorch3d, cloned the repo locally, and tried reinstalling.

The new problem is that both by building from x64 Native Tools Command Prompt, and the pip install -e . , it tells me that it's not finding the compiler I guess. The error message is:

UserWarning: Error checking compiler version for cl: [WinError 2] System cannot find the specified file warnings.warn(f'Error checking compiler version for {compiler}: {error}') error: [WinError 2] System cannot find the specified file

I tried setting the cl.exe path inside the Path environment variable, but it seems it is not working. EDIT: I tried restarting the pc, now the error is the same, but it is not pointing to cl anymore, in fact there is no indication about what is the missing file, even when launching pip with --verbose option.

running build_ext C:\Users\GiovanniBucci\anaconda3\envs\fit3d\lib\site-packages\setuptools\command\easy_install.py:159: EasyInstallDeprecationWarning: easy_install command is deprecated. Use build and pip and other standards-based tools. EasyInstallDeprecationWarning, C:\Users\GiovanniBucci\anaconda3\envs\fit3d\lib\site-packages\setuptools\command\install.py:37: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools. setuptools.SetuptoolsDeprecationWarning, error: [WinError 2] Impossibile trovare il file specificato Error in atexit._run_exitfuncs: Traceback (most recent call last): File "C:\Users\GiovanniBucci\anaconda3\envs\fit3d\lib\site-packages\colorama\ansitowin32.py", line 59, in closed return stream.closed ValueError: underlying buffer has been detached ERROR: Command errored out with exit status 1: 'C:\Users\GiovanniBucci\anaconda3\envs\fit3d\python.exe' -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\Users\GiovanniBucci\Documents\WeWear\Model3D\pytorch3d\setup.py'"'"'; file='"'"'C:\Users\GiovanniBucci\Documents\WeWear\Model3D\pytorch3d\setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(file) if os.path.exists(file) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' develop --no-deps Check the logs for full command output. Exception information: Traceback (most recent call last): File "C:\Users\GiovanniBucci\anaconda3\envs\fit3d\lib\site-packages\pip_internal\cli\base_command.py", line 180, in _main status = self.run(options, args) File "C:\Users\GiovanniBucci\anaconda3\envs\fit3d\lib\site-packages\pip_internal\cli\req_command.py", line 205, in wrapper return func(self, options, args) File "C:\Users\GiovanniBucci\anaconda3\envs\fit3d\lib\site-packages\pip_internal\commands\install.py", line 402, in run pycompile=options.compile, File "C:\Users\GiovanniBucci\anaconda3\envs\fit3d\lib\site-packages\pip_internal\req__init.py", line 85, in install_given_reqs pycompile=pycompile, File "C:\Users\GiovanniBucci\anaconda3\envs\fit3d\lib\site-packages\pip_internal\req\req_install.py", line 773, in install unpacked_source_directory=self.unpacked_source_directory, File "C:\Users\GiovanniBucci\anaconda3\envs\fit3d\lib\site-packages\pip_internal\operations\install\editable_legacy.py", line 46, in install_editable cwd=unpacked_source_directory, File "C:\Users\GiovanniBucci\anaconda3\envs\fit3d\lib\site-packages\pip_internal\utils\subprocess.py", line 244, in call_subprocess raise InstallationSubprocessError(proc.returncode, command_desc) pip._internal.exceptions.InstallationSubprocessError: Command errored out with exit status 1: 'C:\Users\GiovanniBucci\anaconda3\envs\fit3d\python.exe' -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\Users\GiovanniBucci\Documents\WeWear\Model3D\pytorch3d\setup.py'"'"'; file='"'"'C:\Users\GiovanniBucci\Documents\WeWear\Model3D\pytorch3d\setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(file) if os.path.exists(file) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file__, '"'"'exec'"'"'))' develop --no-deps Check the logs for full command output.

I am currently looking for solutions about this, if you already have any hint please let me know.

Thank you again very much in any case, and have a nice day.

bottler commented 2 years ago

Can you check which components of visual studio you have installed? You only get the compiler if you ask for it.

GiBucci commented 2 years ago

I have some troubles in understanding if the compiler has been installed correctly, Visual Studio installer is not clear.

I tried creating and compiling a cpp script in Visual Studio and it seems everything is working fine.

In order to bypass this problem I also installed a separate compiler, namely mingw-w64 from msys2, added to the path environment variable, and tested. GCC is working fine: gcc --version gcc.exe (Rev8, Built by MSYS2 project) 11.2.0 Copyright (C) 2021 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Both attempts did not solve the problem though, with the setup process returning the cl not found error.

What should I check after installing Visual Studio? How can I be sure the compiler can be seen also outside of visual studio?

bottler commented 2 years ago

Don't try to build with mingw - I'm pretty sure you won't be able to get it to work.

You must have the compiler. If you open the right VS code command prompt you should be able to type cl. Can you try each of the VS command prompt links in the start menu?

GiBucci commented 2 years ago

Ok then nevermind.

I can confirm you that if I type cl both in Anaconda Prompt and in x64 Native Tools Command Prompt this works just fine.

cl Microsoft (R) C/C++ Optimizing Compiler versione 19.29.30139 per x86 Copyright (C) Microsoft Corporation. Tutti i diritti sono riservati.

uso: cl [ opzione... ] nomefile... [ /link opzione di collegamento... ]

Even if the terminal shows this correctly, the installation (in both prompts) fails with the error in the EDIT above. I'll report it here again for readability:

running build_ext

C:\Users\GiovanniBucci\anaconda3\envs\fit3d\lib\site-packages\setuptools\command\easy_install.py:159: EasyInstallDeprecationWarning: easy_install command is deprecated. Use build and pip and other standards-based tools. EasyInstallDeprecationWarning, C:\Users\GiovanniBucci\anaconda3\envs\fit3d\lib\site-packages\setuptools\command\install.py:37: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools. setuptools.SetuptoolsDeprecationWarning,

error: [WinError 2] Impossibile trovare il file specificato (System cannot find the specified file)

Error in atexit._run_exitfuncs: Traceback (most recent call last): File "C:\Users\GiovanniBucci\anaconda3\envs\fit3d\lib\site-packages\colorama\ansitowin32.py", line 59, in closed return stream.closed ValueError: underlying buffer has been detached ERROR: Command errored out with exit status 1: 'C:\Users\GiovanniBucci\anaconda3\envs\fit3d\python.exe' -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\Users\GiovanniBucci\Documents\WeWear\Model3D\pytorch3d\setup.py'"'"'; file='"'"'C:\Users\GiovanniBucci\Documents\WeWear\Model3D\pytorch3d\setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(file) if os.path.exists(file) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' develop --no-deps Check the logs for full command output.

Exception information: Traceback (most recent call last): File "C:\Users\GiovanniBucci\anaconda3\envs\fit3d\lib\site-packages\pip_internal\cli\base_command.py", line 180, in main status = self.run(options, args) File "C:\Users\GiovanniBucci\anaconda3\envs\fit3d\lib\site-packages\pip_internal\cli\req_command.py", line 205, in wrapper return func(self, options, args) File "C:\Users\GiovanniBucci\anaconda3\envs\fit3d\lib\site-packages\pip_internal\commands\install.py", line 402, in run pycompile=options.compile, File "C:\Users\GiovanniBucci\anaconda3\envs\fit3d\lib\site-packages\pip_internal\req_init.py", line 85, in install_given_reqs pycompile=pycompile, File "C:\Users\GiovanniBucci\anaconda3\envs\fit3d\lib\site-packages\pip_internal\req\req_install.py", line 773, in install unpacked_source_directory=self.unpacked_source_directory, File "C:\Users\GiovanniBucci\anaconda3\envs\fit3d\lib\site-packages\pip_internal\operations\install\editable_legacy.py", line 46, in install_editable cwd=unpacked_source_directory, File "C:\Users\GiovanniBucci\anaconda3\envs\fit3d\lib\site-packages\pip_internal\utils\subprocess.py", line 244, in call_subprocess raise InstallationSubprocessError(proc.returncode, command_desc) pip._internal.exceptions.InstallationSubprocessError: Command errored out with exit status 1: 'C:\Users\GiovanniBucci\anaconda3\envs\fit3d\python.exe' -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\Users\GiovanniBucci\Documents\WeWear\Model3D\pytorch3d\setup.py'"'"'; file='"'"'C:\Users\GiovanniBucci\Documents\WeWear\Model3D\pytorch3d\setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(file) if os.path.exists(file) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' develop --no-deps Check the logs for full command output.

It is still an error message about missing file, but it doesn't specify which is the missing file. Last exit message talks about some kind of log but I don't understand where to find it and if that could be useful for this matter.

Any idea on what could it be? Same error anyway? Or different problem now?

bottler commented 2 years ago

When you say "the installation", what is the command you were running?

GiBucci commented 2 years ago

As you suggested, I made a local clone of the repository, and when I'm in the folder, I run either

as stated by the install.md file or your first comment.

The result is the same with both commands, it stops at running build_ext as per my last comment. I also tried removing the ./build folder, seems like nothing changes

bottler commented 2 years ago

Is there a file with build errors or build logs (something .txt) ?

GiBucci commented 2 years ago

I'd love to. In the error mentions something like a log file, but I can't find it anywhere. If I run pip install with --log option specifying a file it outputs the error above without any additional information

I don't know if there's a log file on a lower level, and in case where it could be located. Still looking for it.

UPDATE:

I tried logging some stuff from torch cpp_extension BuildExtension, the missing file is detected before starting the compilation process. When it reaches the first file (ball_query.cu), the error is error: Don't know how to compile C:\Users\GiovanniBucci\Documents\WeWear\Model3D\pytorch3d\pytorch3d\csrc\ball_query\ball_query.cu

So it seems it is again a compilation error. I am not familiar with the building process, so I don't really know what steps are followed before the compilation, and it is hard for me to understand where to try and log stuff.

I'll keep my search, trying to find exactly which file it is looking for.

Hope this update helps us narrow down the possible problems.

Have a nice evening.

GiBucci commented 2 years ago

UPDATE 2:

I finally found the issue. CUDA_HOME was not inizialized corretly, as the path I inserted and the one expected by torch were overlapping by one folder. Now it runs the compiler correctly. Sadly more issues came out, mainly regarding compilation. Looking at other issues it seems like the compiler version is too new, I'll need to understand what should I change and how to downgrade the components.

If you are still willing to help me through the process I can keep you updated on the new errors, in other case thank you so much for your help until now and feel free to close the issue, since I think we are going off-topic.

Have a nice day.

github-actions[bot] commented 2 years ago

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.

github-actions[bot] commented 2 years ago

This issue was closed because it has been stalled for 5 days with no activity.