Building on Windows 10: VC 2017 15.7.6 + CUDA 9.2 Step by Step

[Videodr0me]

I just installed a fresh laptop and tried the build instructions here https://github.com/LeelaChessZero/lc0/blob/master/windows_build.md. I ran into some issues, that can be solved. This serves as documentation as well as a reminder that the instructions might need an update. Please also read the link given above as the following is based on it. This is for the cuda version only, but most likely applies to other builds as well:

1. Install VC 2017. Current version is 15.7.6, and this does not work with current CUDA 9.2. This will be fixed in a later step. Add "Desktop developement in C++", "Develop for universal Windows Plattform" and "Python-Development". The latter saves you from seperately installing pyhton.

2. Install GIT for Windows: https://git-scm.com/download/win

3. Install CUDA 9.2: https://developer.nvidia.com/cuda-downloads

4. Install cuDNN: https://developer.nvidia.com/rdp/cudnn-download. This consists of three files that are put in three different CUDA directories as described here: https://docs.nvidia.com/deeplearning/sdk/cudnn-install/index.html. These are the official paths - remember these, they are needed later.

5. You will find python in "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64". Check if this has been added to the path variable, if not add. Also add "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\Scripts"

6. Install Meson: pip3 install --upgrade meson

7. Fetch source (manually or by using the GITHub extension of VS2017) to C:\Users\XXXXXXX\source\repos\lc0

8. Edit/Replace build-cuda.cmd to reflect the paths above, and that you use VC2017:

   
   rd /s build

call "C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Auxiliary\Build\vcvarsall.bat" amd64 meson.py build --backend vs2017 --buildtype release ^ -Dcudnn_libdirs="C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v9.2\lib\x64","C:\dev\cuDNN\cuda\lib\x64" ^ -Dcudnn_include="C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v9.2\include" ^ -Ddefault_library=static

cd build

"C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\MSBuild\15.0\Bin\MSBuild.exe" ^ /p:Configuration=Release ^ /p:Platform=x64 ^ /p:PreferredToolArchitecture=x64 lc0.sln ^ /filelogger

9. Starting build-cuda would yield an error, because cuda 9.2 does not work with the latest VC2017. You have to edit one number in: "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v9.2\include\crt\host_config.h" to the following: 

if _MSC_VER < 1600 || _MSC_VER > 2000

error -- unsupported Microsoft Visual Studio version! Only the versions 2012, 2013, 2015 and 2017 are supported!

This just increases the maximum version number.

10. Run build-cuda. Done. 

[SunnyWar]

Coincidentally I worked out those same steps last night. It worked just as you've outlined. I also sent feedback to NVidia on the version number issue. Hopefully they will fix it for the next release.

[Tilps]

As an alternative to messing with the cuda headers you can also install the older compiler side by side as part of the latest visual studio install and edit the build-cuda to point to it. (Similarly to in the issue I raised a while back.)

[borg323]

You can install an older version of VC++ with VS2017, either 2015.3 (v14.0) or an older 2017 version which should work with CUDA just fine. This is how Appveyor builds lc0, using the 2015 compiler under VS2017.

[Videodr0me]

Yes, I installed the older compiler on a different system some time back, to adress the same issue in a previous CUDA version. I personally like having only one compiler, but if you don't like "hacking" the version check installing the older compiler is a safe alternative.

[Eric-Ryan]

I even copy and pasted the build-cuda script written. I followed every step closely and whenever I build-cuda I get this:

C:\Users\eric\lc0>rd /s build build, Are you sure (Y/N)? y The system cannot find the file specified.

C:\Users\eric\lc0>call "C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Auxiliary\Build\vcvarsall.bat" amd64

---

Visual Studio 2017 Developer Command Prompt v15.8.1 Copyright (c) 2017 Microsoft Corporation

---

[vcvarsall.bat] Environment initialized for: 'x64'

The system cannot find the path specified. Microsoft (R) Build Engine version 15.8.166+gd4e8d81a88 for .NET Framework Copyright (C) Microsoft Corporation. All rights reserved.

MSBUILD : error MSB1009: Project file does not exist. Switch: lc0.sln

[borg323]

@Eric-Ryan it looks like meson did not run. Check whether the directory it is installed in is included in the path.

[Eric-Ryan]

Thank you. Spent the entire day trying to fix the problem. For some reason installing meson would not allow me to use commands such as: "$meson --help" by default. I ended up having to install Python with advanced settings, and explicitly click "install for all users". I really hope this helps someone one day.

[borg323]

Still sounds like a path problem. I will be installing a new vm sometime next week, so will try it again and update the instructions.

[Eric-Ryan]

yes actually I'm still fixing it lol. Could you tell me your meson environment variable path? I can't call "meson --help"

[Eric-Ryan]

now I'm getting "SSL connection failed"

[Eric-Ryan]

ok waving the white flag... I'm just gna do it on ubuntu

[borg323]

Can't help you right now - I only have a windows vm for that (using virtualbox under ubuntu) and can't access this pc now. However did you try to run meson.py instead of just meson?

[borg323]

The python part of the path is: C:\Users\User\AppData\Local\Programs\Python\Python36\Scripts\;C:\Users\User\AppData\Local\Programs\Python\Python36\ and meson.py is in C:\Users\User\AppData\Local\Programs\Python\Python36\Scripts\meson.py

[Eric-Ryan]

I can't even get it to work on ubuntu lol... Well I was getting an SSL Connection error for zlib... I looked in the discord and someone said to just download zlib.zip and put it as a path inside zlib.wrap

[Eric-Ryan]

it still just says "couldn't use fallback subproject zlib as dependency"

[Eric-Ryan]

I got it to work but I'm sorry to say I'm not really sure what I did. I just deleted the entire lc0 folder, and one small thing I did differently was use the git GUI rather than the command prompt... I copy and pasted the code in the original post into build-cuda and it ran... Perhaps another small thing I changed was setting default .sh files to open with git...

[tedsummers]

The information provided here was very helpful. I was finally able to compile lc0 without any issuse.

[porzo]

C:\Users\Pham Hoang Viet\Downloads\lc0-release\lc0-release>rd /s build build, Are you sure (Y/N)? y

C:\Users\Pham Hoang Viet\Downloads\lc0-release\lc0-release>call "C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\VC\Auxiliary\Build\vcvarsall.bat" amd64

---

Visual Studio 2017 Developer Command Prompt v15.8.4 Copyright (c) 2017 Microsoft Corporation

---

[vcvarsall.bat] Environment initialized for: 'x64' The Meson build system Version: 0.47.2 Source dir: C:\Users\Pham Hoang Viet\Downloads\lc0-release\lc0-release Build dir: C:\Users\Pham Hoang Viet\Downloads\lc0-release\lc0-release\build Build type: native build Project name: lc0 Project version: undefined Native C++ compiler: cl (msvc 19.15.26729) Build machine cpu family: x86_64 Build machine cpu: x86_64 Library libprotobuf found: NO Found Pkg-config: NO Dependency protobuf found: NO Program protoc found: NO Traceback (most recent call last): File "c:\program files (x86)\python37-32\lib\site-packages\mesonbuild\mesonmain.py", line 351, in run app.generate() File "c:\program files (x86)\python37-32\lib\site-packages\mesonbuild\mesonmain.py", line 117, in generate self._generate(env) File "c:\program files (x86)\python37-32\lib\site-packages\mesonbuild\mesonmain.py", line 146, in _generate intr.run() File "c:\program files (x86)\python37-32\lib\site-packages\mesonbuild\interpreter.py", line 3699, in run super().run() File "c:\program files (x86)\python37-32\lib\site-packages\mesonbuild\interpreterbase.py", line 380, in run self.evaluate_codeblock(self.ast, start=1) File "c:\program files (x86)\python37-32\lib\site-packages\mesonbuild\interpreterbase.py", line 404, in evaluate_codeblock raise e File "c:\program files (x86)\python37-32\lib\site-packages\mesonbuild\interpreterbase.py", line 398, in evaluate_codeblock self.evaluate_statement(cur) File "c:\program files (x86)\python37-32\lib\site-packages\mesonbuild\interpreterbase.py", line 419, in evaluate_statement return self.evaluate_if(cur) File "c:\program files (x86)\python37-32\lib\site-packages\mesonbuild\interpreterbase.py", line 480, in evaluate_if self.evaluate_codeblock(i.block) File "c:\program files (x86)\python37-32\lib\site-packages\mesonbuild\interpreterbase.py", line 404, in evaluate_codeblock raise e File "c:\program files (x86)\python37-32\lib\site-packages\mesonbuild\interpreterbase.py", line 398, in evaluate_codeblock self.evaluate_statement(cur) File "c:\program files (x86)\python37-32\lib\site-packages\mesonbuild\interpreterbase.py", line 443, in evaluate_statement return self.evaluate_plusassign(cur) File "c:\program files (x86)\python37-32\lib\site-packages\mesonbuild\interpreterbase.py", line 639, in evaluate_plusassign addition = self.evaluate_statement(node.value) File "c:\program files (x86)\python37-32\lib\site-packages\mesonbuild\interpreterbase.py", line 413, in evaluate_statement return self.method_call(cur) File "c:\program files (x86)\python37-32\lib\site-packages\mesonbuild\interpreterbase.py", line 709, in method_call obj = self.evaluate_statement(invokable) File "c:\program files (x86)\python37-32\lib\site-packages\mesonbuild\interpreterbase.py", line 409, in evaluate_statement return self.function_call(cur) File "c:\program files (x86)\python37-32\lib\site-packages\mesonbuild\interpreterbase.py", line 699, in function_call return func(node, posargs, kwargs) File "c:\program files (x86)\python37-32\lib\site-packages\mesonbuild\interpreterbase.py", line 273, in wrapped return f(*wrapped_args, wrapped_kwargs) File "c:\program files (x86)\python37-32\lib\site-packages\mesonbuild\interpreterbase.py", line 162, in wrapped return f(*wrapped_args, *wrapped_kwargs) File "c:\program files (x86)\python37-32\lib\site-packages\mesonbuild\interpreterbase.py", line 133, in wrapped return f(wrapped_args, wrapped_kwargs) File "c:\program files (x86)\python37-32\lib\site-packages\mesonbuild\interpreter.py", line 2209, in func_subproject return self.do_subproject(dirname, kwargs) File "c:\program files (x86)\python37-32\lib\site-packages\mesonbuild\interpreter.py", line 2233, in do_subproject resolved = r.resolve(dirname) File "c:\program files (x86)\python37-32\lib\site-packages\mesonbuild\wrap\wrap.py", line 126, in resolve if self.resolve_git_submodule(dirname): File "c:\program files (x86)\python37-32\lib\site-packages\mesonbuild\wrap\wrap.py", line 159, in resolve_git_submodule ret, out = quiet_git(['rev-parse'], self.subdir_root) File "c:\program files (x86)\python37-32\lib\site-packages\mesonbuild\wrap\wrap.py", line 44, in quiet_git stdout=subprocess.PIPE, stderr=subprocess.PIPE) File "c:\program files (x86)\python37-32\lib\subprocess.py", line 756, in init restore_signals, start_new_session) File "c:\program files (x86)\python37-32\lib\subprocess.py", line 1155, in _execute_child startupinfo) FileNotFoundError: [WinError 2] The system cannot find the file specified

And now I get this error

[borg323]

It looks like you need to install git.

[porzo]

I got git already doing everything step by step till now, maybe configuring it wrong? I have Git Bash, Git CMD, and Git GUI all in the Git folder

[porzo]

Well I tried again this morning and magically it worked, but at the end...:

Done executing task "MSBuild" -- FAILED. Done building target "Build" in project "syzygy_test@exe.vcxproj.metaproj" -- FAILED. Done Building Project "C:\Users\Pham Hoang Viet\Downloads\lc0-release\lc0-release\build\syzygy_test@exe.vcxproj.metaproj" (default targets) -- FAILED.

Done executing task "MSBuild" -- FAILED. Done building target "Build" in project "lc0.sln" -- FAILED. Done Building Project "C:\Users\Pham Hoang Viet\Downloads\lc0-release\lc0-release\build\lc0.sln" (default targets) -- FAILED.

Build FAILED.

"C:\Users\Pham Hoang Viet\Downloads\lc0-release\lc0-release\build\lc0.sln" (default target) (1) -> "C:\Users\Pham Hoang Viet\Downloads\lc0-release\lc0-release\build\subprojects\zlib-1.2.11\Windows resource for file 'subprojectszlib-1.2.11win32_zlib1.rc'@cus.vcxproj.metaproj" (default target) (15) -> "C:\Users\Pham Hoang Viet\Downloads\lc0-release\lc0-release\build\subprojects\zlib-1.2.11\Windows resource for file 'subprojectszlib-1.2.11win32_zlib1.rc'@cus.vcxproj" (default target) (16) -> (InitializeBuildStatus target) -> C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\Common7\IDE\VC\VCTargets\Microsoft.CppBuild.targets(321,5): error MSB3491: Could not write lines to file "Windows resource for file 'subprojectszlib-1.2.11win32_zlib1.rc'@cus\Windows .031EDF02.tlog\Windows resource for file 'subprojectszlib-1.2.11win32_zlib1.rc'.lastbuildstate". The specified path, file name, or both are too long. The fully qualified file name must be less than 260 characters, and the directory name must be less than 248 characters. [C:\Users\Pham Hoang Viet\Downloads\lc0-release\lc0-release\build\subprojects\zlib-1.2.11\Windows resource for file 'subprojectszlib-1.2.11win32_zlib1.rc'@cus.vcxproj]

0 Warning(s)
1 Error(s)

Time Elapsed 00:12:43.73

...it failed. Any suggestion? This error I found in the msbuild.log

[borg323]

It looks like you are running into a path length limitation. Maybe moving the lc0-release directory to C:\ (at least temporarily) before building will help. Also, sometimes running the batch file in a clean cmd window helps.

[porzo]

Hooray thank you very much it built completely fine.

[porzo]

Now... kinda disappointed. Maybe it's my computer but Lc0 took more than 2hrs and couldn't load into the Fritz 16's UCI engine

[borg323]

Can't help you with Fritz, but what do you mean with "took more than 2hrs", was it 2 hrs to build? That does seem a bit slow. You can get pre-built binaries from https://github.com/LeelaChessZero/lc0/releases.

[porzo]

No I meant to load lc0.exe into fritz, I have let it all day long and it hasn't done yet... building was about 45mins-1hr and it's done already.

[borg323]

Loading shouldn't take so long, something is going wrong, but I have no experience with Fritz. You can try asking in the #help channel in https://discord.gg/pKujYxD.

[porzo]

C:\lc0-release>rd /s build build, Are you sure (Y/N)? y The system cannot find the file specified.

C:\lc0-release>call "C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\VC\Auxiliary\Build\vcvarsall.bat" amd64

---

Visual Studio 2017 Developer Command Prompt v15.8.5 Copyright (c) 2017 Microsoft Corporation

---

[vcvarsall.bat] Environment initialized for: 'x64' 'meson.py' is not recognized as an internal or external command, operable program or batch file. The system cannot find the path specified. Microsoft (R) Build Engine version 15.8.169+g1ccb72aefa for .NET Framework Copyright (C) Microsoft Corporation. All rights reserved.

I have to install Lc0 again because my antivirus caught something and deleted it. Now still getting this error even though I left everything as they were, everything installed and ready and still getting this error, can you guys help me again?

[borg323]

Either meson is not installed or you installed very recently (last few days). The latest meson release is an executable and not a python script. Edit the build script and change meson.py to meson.

[porzo]

Would it just be meson or meson.exe? Because it's .exe so I'm just asking for sure

[borg323]

Both should work.

[tedsummers]

meson.py should be changed to changed to: meson.exe in the build-cuda.bat file.

On Sun, Sep 30, 2018 at 4:05 AM borg323 notifications@github.com wrote:

Both should work.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/LeelaChessZero/lc0/issues/240#issuecomment-425702986, or mute the thread https://github.com/notifications/unsubscribe-auth/AHMt57f9rqxnt4zC7yJ3T-qAtTQRAFn5ks5ugHtggaJpZM4VzmWf .

-- Thank You,

---

Theodore P. Summers (Ted) Certified IT Security Consultant

[mooskagh]

Outdated but may be still relevant. Moving to documentation.