DBraun / PyTorchTOP

GPU PyTorch TOP in TouchDesigner with CUDA-enabled OpenCV
Other
77 stars 5 forks source link

Build error when Windows username has whitespace #5

Open vrtx-voiod opened 3 years ago

vrtx-voiod commented 3 years ago

First of all: Thank you for this implementation, I really look forward to test PyTorch in Touchdesigner :)

However I was successfully building OpenCV 4.5.1 from source and during the final stages of the Release build of PyTorchTOP I ran into some issues:

During CMake compiling of the PyTorchTop I get this error message:

`CMake Warning (dev) in CMakeLists.txt: Policy CMP0111 is not set: An imported target missing its location property fails during generation. Run "cmake --help-policy CMP0111" for policy details. Use the cmake_policy command to set the policy and suppress this warning.

IMPORTED_IMPLIB not set for imported target "torch" configuration "Debug". This warning is for project developers. Use -Wno-dev to suppress it.

-- Generating done -- Build files have been written to: "_my_directory_/PyTorchTOP/build`

This seems to be not really an issue since I could find the PyTorchTOP.sln in the build folder, but when I try to build the solution in Visual Studio 2019 Community 16.8.3 I get these errors:

Severity Code Description Project File Line Suppression State Error MSB3073 The command "setlocal "C:\Program Files\CMake\bin\cmake.exe" -E copy_if_different "_**my_directory**_/PyTorchTOP/build/Release/PyTorchTOP.dll" %USERPROFILE%/Documents/Derivative/Plugins if %errorlevel% neq 0 goto :cmEnd :cmEnd endlocal & call :cmErrorLevel %errorlevel% & goto :cmDone :cmErrorLevel exit /b %1 :cmDone if %errorlevel% neq 0 goto :VCEnd setlocal "C:\Program Files\CMake\bin\cmake.exe" -E copy_if_different C:/libtorch/libtorch-win-shared-with-deps-1.7.1+cu110/libtorch/lib/asmjit.dll C:/libtorch/libtorch-win-shared-with-deps-1.7.1+cu110/libtorch/lib/c10.dll C:/libtorch/libtorch-win-shared-with-deps-1.7.1+cu110/libtorch/lib/c10_cuda.dll C:/libtorch/libtorch-win-shared-with-deps-1.7.1+cu110/libtorch/lib/caffe2_detectron_ops_gpu.dll C:/libtorch/libtorch-win-shared-with-deps-1.7.1+cu110/libtorch/lib/caffe2_module_test_dynamic.dll C:/libtorch/libtorch-win-shared-with-deps-1.7.1+cu110/libtorch/lib/caffe2_nvrtc.dll C:/libtorch/libtorch-win-shared-with-deps-1.7.1+cu110/libtorch/lib/cublas64_11.dll C:/libtorch/libtorch-win-shared-with-deps-1.7.1+cu110/libtorch/lib/cublasLt64_11.dll C:/libtorch/libtorch-win-shared-with-deps-1.7.1+cu110/libtorch/lib/cudart64_110.dll C:/libtorch/libtorch-win-shared-with-deps-1.7.1+cu110/libtorch/lib/cudnn64_8.dll C:/libtorch/libtorch-win-shared-with-deps-1.7.1+cu110/libtorch/lib/cudnn_adv_infer64_8.dll C:/libtorch/libtorch-win-shared-with-deps-1.7.1+cu110/libtorch/lib/cudnn_adv_train64_8.dll C:/libtorch/libtorch-win-shared-with-deps-1.7.1+cu110/libtorch/lib/cudnn_cnn_infer64_8.dll C:/libtorch/libtorch-win-shared-with-deps-1.7.1+cu110/libtorch/lib/cudnn_cnn_train64_8.dll C:/libtorch/libtorch-win-shared-with-deps-1.7.1+cu110/libtorch/lib/cudnn_ops_infer64_8.dll C:/libtorch/libtorch-win-shared-with-deps-1.7.1+cu110/libtorch/lib/cudnn_ops_train64_8.dll C:/libtorch/libtorch-win-shared-with-deps-1.7.1+cu110/libtorch/lib/cufft64_10.dll C:/libtorch/libtorch-win-shared-with-deps-1.7.1+cu110/libtorch/lib/cufftw64_10.dll C:/libtorch/libtorch-win-shared-with-deps-1.7.1+cu110/libtorch/lib/curand64_10.dll C:/libtorch/libtorch-win-shared-with-deps-1.7.1+cu110/libtorch/lib/cusolver64_10.dll C:/libtorch/libtorch-win-shared-with-deps-1.7.1+cu110/libtorch/lib/cusolverMg64_10.dll C:/libtorch/libtorch-win-shared-with-deps-1.7.1+cu110/libtorch/lib/cusparse64_11.dll C:/libtorch/libtorch-win-shared-with-deps-1.7.1+cu110/libtorch/lib/fbgemm.dll C:/libtorch/libtorch-win-shared-with-deps-1.7.1+cu110/libtorch/lib/libiomp5md.dll C:/libtorch/libtorch-win-shared-with-deps-1.7.1+cu110/libtorch/lib/libiompstubs5md.dll C:/libtorch/libtorch-win-shared-with-deps-1.7.1+cu110/libtorch/lib/nvToolsExt64_1.dll C:/libtorch/libtorch-win-shared-with-deps-1.7.1+cu110/libtorch/lib/nvrtc-builtins64_110.dll C:/libtorch/libtorch-win-shared-with-deps-1.7.1+cu110/libtorch/lib/nvrtc64_110_0.dll C:/libtorch/libtorch-win-shared-with-deps-1.7.1+cu110/libtorch/lib/torch.dll C:/libtorch/libtorch-win-shared-with-deps-1.7.1+cu110/libtorch/lib/torch_cpu.dll C:/libtorch/libtorch-win-shared-with-deps-1.7.1+cu110/libtorch/lib/torch_cuda.dll C:/libtorch/libtorch-win-shared-with-deps-1.7.1+cu110/libtorch/lib/torch_global_deps.dll C:/libtorch/libtorch-win-shared-with-deps-1.7.1+cu110/libtorch/lib/uv.dll %USERPROFILE%/Documents/Derivative/Plugins if %errorlevel% neq 0 goto :cmEnd :cmEnd endlocal & call :cmErrorLevel %errorlevel% & goto :cmDone :cmErrorLevel exit /b %1 :cmDone if %errorlevel% neq 0 goto :VCEnd :VCEnd" exited with code 1. PyTorchTOP C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\MSBuild\Microsoft\VC\v160\Microsoft.CppCommon.targets 153 Warning C4005 'NOMINMAX': macro redefinition PyTorchTOP _**my_directory**_\repositories\PyTorchTOP\src\CPlusPlus_Common.h 26 Warning C4102 'cv': unreferenced label PyTorchTOP _**my_directory**_\repositories\PyTorchTOP\src\PyTorchTOP.cpp 470 Error Target (for copy_if_different command) "_**my_directory**_/Documents/Derivative/Plugins" is not a directory. PyTorchTOP _**my_directory**_\repositories\PyTorchTOP\build\EXEC_ 1

It looks like there is an issue with _my_directory since my username has a white space inbetween, so I manually copied the output of the my_directory_\PyTorchTOP\build\Release folder to my Touchdesigner Plugins folder.

After that I get this error message:

error

I've tried to make a Debug build with the libtorch debug libraries, but then I get this Error Messages:

Severity Code Description Project File Line Suppression State Error LNK1181 cannot open input file '_**my_directory**_\repositories\opencv\build\lib\Debug\opencv_world451d.lib' PyTorchTOP _**my_directory**_\repositories\PyTorchTOP\build\LINK 1 Warning C4005 'NOMINMAX': macro redefinition PyTorchTOP _**my_directory**_\repositories\PyTorchTOP\src\CPlusPlus_Common.h 26 Warning C4102 'cv': unreferenced label PyTorchTOP _**my_directory**_\repositories\PyTorchTOP\src\PyTorchTOP.cpp 470

riebschlager commented 3 years ago

I ran into this same issue today. I don't have a space in my username, but I'm getting the same Plugin Load Error.

DBraun commented 3 years ago

@vrtx-voiod Thanks for trying it.

  1. I'm not concerned about the first CMake issue since we're just building in Release.
  2. I agree that it's probably failing because of the whitespace in your username. You were right to copy PyTorchTOP.dll over but I want to make sure you've also copied all of the Libtorch DLLs over. In your case they might be at C:/libtorch/lib/*.dll.
  3. When building in Debug, it's looking for a debug build of OpenCV. That's why it's looking for opencv_world451d.lib (with a d suffix) in a Debug folder instead of opencv_world451.lib. I'm not sure if you meant to say you already built opencv with Debug.

Let's look for a solution to 2. Could you modify these lines so that %USERPROFILE%/Documents/Derivative/Plugins becomes quote escaped? "%USERPROFILE%/Documents/Derivative/Plugins"

Two places to modify: https://github.com/DBraun/PyTorchTOP/blob/4ff0644e9c5975c492a32798056eb8a11e8fa789/CMakeLists.txt#L142 https://github.com/DBraun/PyTorchTOP/blob/4ff0644e9c5975c492a32798056eb8a11e8fa789/CMakeLists.txt#L150

Then redo the CMake step that creates the PyTorchTOP.sln.

@riebschlager Maybe you're encountering the same issue with the Libtorch DLLs not being next to the PyTorchTOP.dll. Let me know.

riebschlager commented 3 years ago

Hmm. Still no dice in my case. It appears that everything is getting built can copied to the plugins directory. Here's the contents of that directory, does this seem correct?

-rw-r--r-- 1 User 197121       1410 Dec 30 12:26 Plugins.json
-rwxr-xr-x 1 User 197121     218112 Dec 30 12:25 PyTorchTOP.dll*
-rwxr-xr-x 1 User 197121     242176 Dec 30 09:40 asmjit.dll*
-rwxr-xr-x 1 User 197121     355328 Dec 30 09:40 c10.dll*
-rwxr-xr-x 1 User 197121     230400 Dec 30 09:40 c10_cuda.dll*
-rwxr-xr-x 1 User 197121   21731840 Dec 30 09:40 caffe2_detectron_ops_gpu.dll*
-rwxr-xr-x 1 User 197121      69120 Dec 30 09:40 caffe2_module_test_dynamic.dll*
-rwxr-xr-x 1 User 197121      15360 Dec 30 09:40 caffe2_nvrtc.dll*
-rwxr-xr-x 1 User 197121     879616 Dec 30 09:40 caffe2_observers.dll*
-rwxr-xr-x 1 User 197121     222720 Dec 30 09:40 cudnn64_8.dll*
-rwxr-xr-x 1 User 197121   99908096 Dec 30 09:40 cudnn_adv_infer64_8.dll*
-rwxr-xr-x 1 User 197121   65183232 Dec 30 09:40 cudnn_adv_train64_8.dll*
-rwxr-xr-x 1 User 197121  508834304 Dec 30 09:40 cudnn_cnn_infer64_8.dll*
-rwxr-xr-x 1 User 197121   51145728 Dec 30 09:40 cudnn_cnn_train64_8.dll*
-rwxr-xr-x 1 User 197121  240333312 Dec 30 09:40 cudnn_ops_infer64_8.dll*
-rwxr-xr-x 1 User 197121   25651712 Dec 30 09:40 cudnn_ops_train64_8.dll*
-rwxr-xr-x 1 User 197121    3268096 Dec 30 09:40 fbgemm.dll*
-rwxr-xr-x 1 User 197121 1098036736 Dec 29 20:44 opencv_world451.dll*
-rwxr-xr-x 1 User 197121      55808 Dec 30 09:40 shm.dll*
-rwxr-xr-x 1 User 197121       9728 Dec 30 09:40 torch.dll*
-rwxr-xr-x 1 User 197121  186286080 Dec 30 09:40 torch_cpu.dll*
-rwxr-xr-x 1 User 197121  797611520 Dec 30 09:40 torch_cuda.dll*
-rwxr-xr-x 1 User 197121       9216 Dec 30 09:40 torch_global_deps.dll*
-rwxr-xr-x 1 User 197121    9321472 Dec 30 09:40 torch_python.dll*
-rwxr-xr-x 1 User 197121     195072 Dec 30 09:40 uv.dll*
DBraun commented 3 years ago

There are some differences between my list of DLLs and yours.

from libtorch-win-shared-with-deps-1.7.1+cu110.zip:

12/07/2020 11:49 AM 242,176 asmjit.dll 12/07/2020 11:50 AM 355,328 c10.dll 12/07/2020 11:50 AM 230,912 c10_cuda.dll 12/07/2020 12:02 PM 21,732,352 caffe2_detectron_ops_gpu.dll 12/07/2020 12:02 PM 69,120 caffe2_module_test_dynamic.dll 12/07/2020 11:50 AM 15,360 caffe2_nvrtc.dll 06/26/2020 06:56 PM 88,860,672 cublas64_11.dll 06/26/2020 06:56 PM 159,866,368 cublasLt64_11.dll 06/12/2020 02:50 AM 401,408 cudart64_110.dll 09/22/2020 11:38 PM 222,720 cudnn64_8.dll 09/23/2020 12:09 AM 99,908,096 cudnn_adv_infer64_8.dll 09/23/2020 12:20 AM 65,183,232 cudnn_adv_train64_8.dll 09/23/2020 12:19 AM 508,834,304 cudnn_cnn_infer64_8.dll 09/23/2020 12:27 AM 51,145,728 cudnn_cnn_train64_8.dll 09/22/2020 11:45 PM 240,333,312 cudnn_ops_infer64_8.dll 09/22/2020 11:55 PM 25,651,712 cudnn_ops_train64_8.dll 06/12/2020 02:50 AM 162,937,856 cufft64_10.dll 06/12/2020 02:50 AM 272,896 cufftw64_10.dll 06/12/2020 02:50 AM 52,446,720 curand64_10.dll 06/12/2020 02:50 AM 505,477,120 cusolver64_10.dll 06/12/2020 02:50 AM 294,363,136 cusolverMg64_10.dll 06/12/2020 02:50 AM 160,346,112 cusparse64_11.dll 12/07/2020 11:49 AM 3,268,096 fbgemm.dll 06/22/2020 10:03 PM 1,961,328 libiomp5md.dll 06/22/2020 10:03 PM 110,448 libiompstubs5md.dll 06/12/2020 02:50 AM 5,117,440 nvrtc-builtins64_110.dll 06/12/2020 02:50 AM 16,161,792 nvrtc64_110_0.dll 08/04/2016 06:47 PM 48,128 nvToolsExt64_1.dll 12/07/2020 12:02 PM 9,728 torch.dll 12/07/2020 11:55 AM 186,265,088 torch_cpu.dll 12/07/2020 12:02 PM 797,608,960 torch_cuda.dll 12/07/2020 11:50 AM 9,216 torch_global_deps.dll 09/18/2020 03:02 AM 195,072 uv.dll

Are some of these missing from your Plugins folder?

riebschlager commented 3 years ago

YES! Thank you! I must have screwed something up during the install process. I found those DLL's elsewhere on my machine. I copied them in manually and it works great.

Thanks so much for doing this work.

vrtx-voiod commented 3 years ago

@DBraun I also could manage to run it by manually setting my ~/Documents/Derivative/Plugins in the ~/PyTorchTOP/CMakeLists.txt as you've mentioned, thanks :)

DBraun commented 3 years ago

Cool. I did just push an update that probably handles whitespace usernames better. At least it didn't break my system.

DBraun commented 3 years ago

The issue with whitespace in the username is not fixed yet.

DBraun commented 3 years ago

The temporary solution is to hardcode the path to your plugins directory:

https://github.com/DBraun/PyTorchTOP/blob/4ff0644e9c5975c492a32798056eb8a11e8fa789/CMakeLists.txt#L142 https://github.com/DBraun/PyTorchTOP/blob/4ff0644e9c5975c492a32798056eb8a11e8fa789/CMakeLists.txt#L150