Install maskrcnn-benchmark on Windows(VS2017+CUDA10.1)

[msqiu]

Raw file

Install Pytorch maskrcnn on Windows(VS2017+CUDA10.1).md

Enviornments

• windows 10 64 bit (laptop)
• Nvdia gt 1060 (notebook)
• install date: 10.08.2019

Useful references

• main steps: https://github.com/facebookresearch/maskrcnn-benchmark/issues/547#issuecomment-475903282
• build maskrcnn-benchmark for win10+vs2017.md
• https://github.com/NVIDIA/apex
• https://www.jianshu.com/p/e3efaaf7e655
• https://www.jianshu.com/p/e9680d0bfa5c
• https://github.com/facebookresearch/maskrcnn-benchmark/issues/254

Steps

1. Install Visio Studio 2017 Community (latest version, 64bit). Make certain you click 'VC++ 2017 version 15.4 v14.11 toolset' below the 'Individual components' tab.
2. Install "Build Tools For Visual Studio"(VS2017 latest version) , run it. Select: 'Workloads - Visual C++ build tools' and select from individual components also 'Windows 10 SDK'(latest version) and 'VC++ 2015.3 v14.00 (v140) toolset for desktop'.
3. Add this to your PATH environment variables: C:\Program Files (x86)\Windows Kits\10\bin\x64
4. Copy rc.exe & rcdll.dll from C:\Program Files (x86)\Windows Kits\8.1\bin\x86 to C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\bin
5. restart the computer
6. Install CUDA 10.1(64 bit for windows), standard install.
7. restart the computer
8. Download cudnn-10.1-windows10-x64-v7.5.0.56 (search from Internet). Extract to a temp folder. As putting it into a folder and adding to %PATH% won't work anymore for some reason, do the following:
   • Copy \cuda\bin\cudnn64_7.dll to C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.1\bin.
   • Copy \cuda\include\cudnn.h to C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.1\include.
   • Copy \cuda\lib\x64\cudnn.lib to C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.1\lib\x64.
9. Install Anaconda if you didn't do it yet.

10. Execute the following commands:

    conda create -n maskrcnn-benchmark python=3.6
    activate maskrcnn-benchmark
    conda install ipython
    pip install ninja yacs cython matplotlib tqdm requests
    conda install -c pytorch pytorch-nightly torchvision cudatoolkit=10.0
    conda install -c anaconda git
    conda install nb_conda
    pip install opencv-python opencv-contrib-python
    pip install "git+https://github.com/philferriere/cocoapi.git#egg=pycocotools&subdirectory=PythonAPI"

11. Install Apex:

    git clone https://github.com/NVIDIA/apex.git
    cd apex

    Comment this part of the code in setup.py :

    '''
    if (bare_metal_major != torch_binary_major) or (bare_metal_minor != torch_binary_minor):
            raise RuntimeError("Cuda extensions are being compiled with a version of Cuda that does " +
                            "not match the version used to compile Pytorch binaries.  " +
                            "Pytorch binaries were compiled with Cuda {}.\n".format(torch.version.cuda) +
                            "In some cases, a minor-version mismatch will not cause later errors:  " +
                            "https://github.com/NVIDIA/apex/pull/323#discussion_r287021798.  "
                            "You can try commenting out this check (at your own risk).")
                           '''

    Then execute the following commands:

    pip install -v --no-cache-dir --global-option="--cpp_ext" --global-option="--cuda_ext"
    cd ..

12. PyTorch solution

    • Download/unzip or git this solution, open up Anaconda prompt and navigate into the working folder.

    • Navigate to maskrcnn_benchmark\csrc\cuda and modify ROIAlign_cuda.cu, ROIPool_cuda.cu and SigmoidFocalLoss_cuda.cu in the following manner: replace the call for THCCeilDiv to ceil_div1,ceil_div2,ceil_div3 respectively (replace all two calls in each of these files) and add the following function (in all of these files, so for the first file it is like below, just the name changes because of proper linking later):

      long ceil_div1(long a, long b) { 
      return (a + b - 1) / b; 
      }

      do not forget to change the string type of ceil_divs from (int) to (long), for example:

      ```
      /* replace the lines with THCCeilDiv, there are 2 palces in each file */
      dim3  grid(std::min(ceil_div((long)output_size, 512), 4096));
      // dim3  grid(std::min(THCCeilDiv(output_size, 512L), 4096L));
      ```

    • Navigate back to the root folder of the solution, open up setup.py and modify the following two parts like this:

      # add 'extra_compile_args' and 'extra_link_args' in get_extensions()
      extra_compile_args  = {"cxx": ['/MD']}
      extra_link_args  = ['/NODEFAULTLIB:LIBCMT.LIB']
      
      # change the ext_modules in get_extensions()
      ext_modules = [
          extension(
              "maskrcnn_benchmark._C",
              sources,
              include_dirs=include_dirs,
              define_macros=define_macros,
              extra_compile_args=extra_compile_args,
              extra_link_args = extra_link_args,
              )
      ]

13. Go back to Anaconda command prompt and execute:

    set "VS150COMNTOOLS=c:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Auxiliary\Build"
    set CMAKE_GENERATOR=Visual Studio 15 2017 Win64
    set DISTUTILS_USE_SDK=1
    call "%VS150COMNTOOLS%\vcvarsall.bat" x64 -vcvars_ver=14.11
    python setup.py build develop

14. Enjoy! You can try the following demo and it will work if you install successfully(use the camera).

    cd to maskrcnn-benchmark/demo
    python webcam.py --min-image-size 800

Remember

1. If you change the files in maskrcnn-benchmark, don't forget to rebuild maskrcnn-benchmark!
2. The location of the library maskrcnn-benchmark is not in Anacoda-envs but in the folder where you maskrcnn-benchmark is. So better to put your code in the maskrcnn-benchmark folder.
3. If you tried to install CUDA or Visio Studio but failed, better use uninstall tool to uninstall the software and delete the remains.

[CoinCheung]

Could I skip all the steps related to cuda if I only would like to use cpu on windows?

[nguyen14ck]

Windows 10 Cuda 10.1 Python 3.6 Pytorch 1.2 I use VStudio 2019, but it does not work. Any idea to help? Thanks.

nvcc fatal   : Could not set up the environment for Microsoft Visual Studio using 'C:/Program Files (x86)/Microsoft Visual Studio/2019/Community/VC/Tools/MSVC/14.21.27702/bin/HostX64/x64/../../../../../../../VC/Auxiliary/Build/vcvars64.bat'
error: command 'C:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v10.1\\bin\\nvcc.exe' failed with exit status 1

[msqiu]

Windows 10 Cuda 10.1 Python 3.6 Pytorch 1.2 I use VStudio 2019, but it does not work. Any idea to help? Thanks.

nvcc fatal   : Could not set up the environment for Microsoft Visual Studio using 'C:/Program Files (x86)/Microsoft Visual Studio/2019/Community/VC/Tools/MSVC/14.21.27702/bin/HostX64/x64/../../../../../../../VC/Auxiliary/Build/vcvars64.bat'
error: command 'C:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v10.1\\bin\\nvcc.exe' failed with exit status 1

try vs 2017, and all the specific version I wrote in the document.

[elnazsn1988]

Hi, this and many related links use vs2017, can we not use vs 2019? with the details provided above? @msqiu

[burhr2]

Windows 10 Cuda 10.1 Python 3.6 Pytorch 1.2 I use VStudio 2019, but it does not work. Any idea to help? Thanks.

nvcc fatal   : Could not set up the environment for Microsoft Visual Studio using 'C:/Program Files (x86)/Microsoft Visual Studio/2019/Community/VC/Tools/MSVC/14.21.27702/bin/HostX64/x64/../../../../../../../VC/Auxiliary/Build/vcvars64.bat'
error: command 'C:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v10.1\\bin\\nvcc.exe' failed with exit status 1

try vs 2017, and all the specific version I wrote in the document.

Hi @msqiu, I am receiving the same error as @nguyen14ck while following the same step you mentioned (vs2017)