Fatal Error when building PytorchTOP using cmake in cmd

[majinshaoyuindustry]

Hi David,

I am getting this error below:

fatal error C1083: Cannot open include file: 'pthread.h': No such file or directory

my setup is:

VS 2019 cmake version 3.19.2

Is there anyway to fix it?

Thank you

[DBraun]

Could you post the exact line you're running with cmake and any output?

for example: cmake -DCMAKE_PREFIX_PATH=C:\libtorch-1.7 .. and then any output. Feel free to censor your username etc. Also hopefully your Visual Studio is fairly up to date. I'm using Visual Studio 2019.

[majinshaoyuindustry]

Could you post the exact line you're running with cmake and any output?

for example: cmake -DCMAKE_PREFIX_PATH=C:\libtorch-1.7 .. and then any output. Feel free to censor your username etc. Also hopefully your Visual Studio is fairly up to date. I'm using Visual Studio 2019.

Hi Thank you for your fast response and happy new year!

Here is what I typed: cmake -DCMAKE_PREFIX_PATH=C:\Machinelearning\libtorch ..

BTW I used the release version of libtorch: https://download.pytorch.org/libtorch/cu110/libtorch-win-shared-with-deps-1.7.1%2Bcu110.zip

**I am running on VS 2019, cmake 3.19.2

Here is what I got returned:**

-- Building for: Visual Studio 16 2019 -- Selecting Windows SDK version 10.0.18362.0 to target Windows 10.0.19042. -- The C compiler identification is MSVC 19.28.29335.0 -- The CXX compiler identification is MSVC 19.28.29335.0 -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Enterprise/VC/Tools/MSVC/14.28.29333/bin/Hostx64/x64/cl.exe - skipped -- Detecting C compile features -- Detecting C compile features - done -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Enterprise/VC/Tools/MSVC/14.28.29333/bin/Hostx64/x64/cl.exe - skipped -- Detecting CXX compile features -- Detecting CXX compile features - done x64 architecture in use -- Found CUDA: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.0 (found version "11.0") -- Looking for pthread.h -- Looking for pthread.h - not found -- Found Threads: TRUE -- Caffe2: CUDA detected: 11.0 -- Caffe2: CUDA nvcc is: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.0/bin/nvcc.exe -- Caffe2: CUDA toolkit directory: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.0 -- Caffe2: Header version is: 11.0 -- Found CUDNN: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.0/lib/x64/cudnn.lib -- Found cuDNN: v? (include: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.0/include, library: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.0/lib/x64/cudnn.lib) CMake Error at C:/Machinelearning/libtorch/share/cmake/Caffe2/public/cuda.cmake:174 (message): PyTorch requires cuDNN 7 and above. Call Stack (most recent call first): C:/Machinelearning/libtorch/share/cmake/Caffe2/Caffe2Config.cmake:88 (include) C:/Machinelearning/libtorch/share/cmake/Torch/TorchConfig.cmake:40 (find_package) CMakeLists.txt:126 (find_package)

-- Configuring incomplete, errors occurred! See also "C:/Machinelearning/PyTorchTOP/build/CMakeFiles/CMakeOutput.log". See also "C:/Machinelearning/PyTorchTOP/build/CMakeFiles/CMakeError.log".

And here is what's in the CMakeError.log:

Determining if the include file pthread.h exists failed with the following output: Change Dir: C:/Machinelearning/PyTorchTOP/build/CMakeFiles/CMakeTmp

Run Build Command(s):C:/Program Files (x86)/Microsoft Visual Studio/2019/Enterprise/MSBuild/Current/Bin/MSBuild.exe cmTC_6a073.vcxproj /p:Configuration=Debug /p:Platform=x64 /p:VisualStudioVersion=16.0 /v:m && Microsoft (R) Build Engine version 16.8.2+25e4d540b for .NET Framework

Copyright (C) Microsoft Corporation. All rights reserved.

Microsoft (R) C/C++ Optimizing Compiler Version 19.28.29335 for x64

Copyright (C) Microsoft Corporation. All rights reserved.

CheckIncludeFile.c

cl /c /W1 /WX- /diagnostics:column /O2 /D "CMAKE_INTDIR=\"Debug\"" /D _MBCS /Gm- /MD /GS /fp:precise /Zc:wchar_t /Zc:forScope /Zc:inline /Fo"cmTC_6a073.dir\Debug\" /Fd"cmTC_6a073.dir\Debug\vc142.pdb" /Gd /TC /errorReport:queue C:\Machinelearning\PyTorchTOP\build\CMakeFiles\CMakeTmp\CheckIncludeFile.c

C:\Machinelearning\PyTorchTOP\build\CMakeFiles\CMakeTmp\CheckIncludeFile.c(1,10): fatal error C1083: Cannot open include file: 'pthread.h': No such file or directory [C:\Machinelearning\PyTorchTOP\build\CMakeFiles\CMakeTmp\cmTC_6a073.vcxproj]

It also seems like it doesn't recognize my Cudnn version which is 8.x for CUDA 11.0. I have triple-checked on that ends... Thank you!!!

[DBraun]

Thanks happy new year to you too. I’ll take another look tomorrow. I think the pthread.h is less urgent than the cudnn error. But it sounded like you installed it correctly.

[majinshaoyuindustry]

Thanks happy new year to you too. I’ll take another look tomorrow. I think the pthread.h is less urgent than the cudnn error. But it sounded like you installed it correctly.

Sounds awesome, thank you for your help and also for this amazing repo. I got torch to work with ScriptTOP but I am relatively new to c++. Last thought, do you think the pthread.h should be part of VS or libtorch? I had never came across this header file and didn't know what was it for. Just a direction so I will see if I can debug it tonight.

[DBraun]

These are the links I used: https://developer.nvidia.com/cuda-11.0-update1-download-archive https://developer.nvidia.com/compute/machine-learning/cudnn/secure/8.0.5/11.0_20201106/cudnn-11.0-windows-x64-v8.0.5.39.zip

The first discrepancy between my log and yours is that mine says

-- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Community/VC/Tools/MSVC/14.27.29110/bin/Hostx64/x64/cl.exe -- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Community/VC/Tools/MSVC/14.27.29110/bin/Hostx64/x64/cl.exe -- works

whereas yours says skipped. I can confirm that my cl.exe exists at that path. Does yours?

The next difference is mine says

-- Found cuDNN: v8.0.5 (include: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.0/include, library: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.0/lib/x64/cudnn.lib)

but yours says

Found cuDNN: v?

If you haven't already try

set CUDA_HOME=C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.0
set CUDA_PATH=C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.0
set PATH=%CUDA_HOME%;%CUDA_PATH%;%PATH%

before using cmake.

Also some people are troubleshooting that Found cuDNN: v? message: https://github.com/pytorch/pytorch/issues/ 40965 (space inserted before 40965 to prevent linking to this page)

I do have the messages about pthread.h but they're not fatal, just

-- Looking for pthread.h -- Looking for pthread.h - not found

[majinshaoyuindustry]

UPDATE: It finally worked!

NOTE: I still have the pthread.h problem but as you said it doesn't matter. And I still have the "cl.exe skipped" showed up even after I tried to give target/giving admin privilege during the cmake process. But the real problem was when the Cudnn 8.0.5 wasn't recognized. I had tried many things and repeat the process many times and the key for me is:

COPY AND PASTE everything in the three folder .dll, .h and .lib. For some reason the tutorials only asked you to put cudnn.h, cudnn64_8.dll and cudnn.lib and it was fine for cuda 10.1: https://docs.nvidia.com/deeplearning/cudnn/install-guide/index.html

I would assume it won't have any negative impact for pasting more stuffs. Thanks David!

[DBraun]

Great!