taichi-dev / taichi

Productive, portable, and performant GPU programming in Python.
https://taichi-lang.org
Apache License 2.0
25.51k stars 2.28k forks source link

[Docker] [Windows] Cmake cannot find instance of visual studio #3811

Closed qiao-bo closed 2 years ago

qiao-bo commented 2 years ago

Cmake complains not being able to find instance of visual studio when building with the following Dockerfile.

third-party files used: vs_buildtools_2019.exe, taichi-llvm-10.0.0-msvc2019.zip, clang-10.0.0-win.zip

# escape=`

# Use the latest Windows Server Core image
FROM mcr.microsoft.com/windows/servercore:ltsc2019

# Use Powerhell for running commands
SHELL ["powershell", "-Command", "$ErrorActionPreference = 'Stop'; $ProgressPreference = 'Continue'; $verbosePreference='Continue';"]

# Install VC++ Build Tools and Windows SDK
COPY third-party/vs_buildtools_2019.exe C:\vs_buildtools_2019.exe
RUN C:\vs_buildtools_2019.exe --quiet --wait --norestart --nocache `
    --installPath C:\BuildTools `
    --add Microsoft.VisualStudio.Workload.VCTools `
    --add Microsoft.VisualStudio.Component.Windows10SDK.17763; `
    while (tasklist | Select-String "vs_buildtools_2019.exe") { echo .; Start-Sleep 10 }; `
    Remove-Item C:\vs_buildtools_2019.exe; `
    Remove-Item -Recurse 'C:\ProgramData\Package Cache'

# Install Chocolatey and required tools
RUN Set-ExecutionPolicy Bypass -Scope Process -Force; `
    [System.Net.ServicePointManager]::SecurityProtocol = `
        [System.Net.ServicePointManager]::SecurityProtocol -bor 3072; `
    iex ((New-Object System.Net.WebClient).DownloadString('https://chocolatey.org/install.ps1')); `
    choco install -y cmake --version 3.22.0; `
    choco install -y git; `
    choco install -y miniconda3; `
    Remove-Item -Recurse C:\ProgramData\chocolatey\lib\cmake*; `
    Remove-Item -Recurse C:\ProgramData\chocolatey\lib\git*; `
    [Environment]::SetEnvironmentVariable(\"Path\", `
        \"C:\Program Files\CMake\bin;C:\Program Files\Git\bin;C:\tools\miniconda3\Scripts;\" + $Env:Path, `
        [System.EnvironmentVariableTarget]::Machine)

RUN conda --version
RUN choco install -y 7zip

COPY third-party/taichi-llvm-10.0.0-msvc2019.zip C:\taichi-llvm-10.0.0-msvc2019.zip
COPY third-party/clang-10.0.0-win.zip C:\clang-10.0.0-win.zip

RUN 7z x C:\taichi-llvm-10.0.0-msvc2019.zip -otaichi_llvm
RUN 7z x C:\clang-10.0.0-win.zip -otaichi_clang

RUN [Environment]::SetEnvironmentVariable(\"Path\", `
        \"C:\taichi_llvm\bin;C:\taichi_clang\bin;\" + $Env:Path, `
        [System.EnvironmentVariableTarget]::Machine)

RUN clang --version
RUN conda init powershell
RUN conda create -n py37 python=3.7
RUN conda activate py37

RUN [Environment]::SetEnvironmentVariable(\"CXX\", `
        \"C:taichi_clang\bin\clang++.exe\", `
        [System.EnvironmentVariableTarget]::Machine)

RUN [Environment]::SetEnvironmentVariable(\"TAICHI_CMAKE_ARGS\", `
        \"-G 'Visual Studio 16 2019' -A x64 -DLLVM_DIR=C:\taichi_llvm\lib\cmake\llvm -DPYTHON_EXECUTABLE=C:\tools\miniconda3\envs\py37\python.exe -DCMAKE_GENERATOR_INSTANCE=C:\BuildTools"\", `
        [System.EnvironmentVariableTarget]::Machine)

RUN git clone --recursive https://github.com/taichi-dev/taichi.git
RUN cd taichi; `
        python -m pip install -r requirements_dev.txt; `
        python -m pip install -r requirements_test.txt

RUN cd taichi; python -m pip install -v -e .

The problem can be reproduced on a windows server 10.0.17763.2183.

possible related issue here: In our case, vswhere also need to use -products * to be able to find the build tools. Strangely, the flags show isLaunchable: 1. We also tried to increased the mem size during build to avoid out of memory error. No luck so far!

Any comments/suggestions/helps are appreciated!

bobcao3 commented 2 years ago

You need to launch the build script within the Visual Studio dev env

qiao-bo commented 2 years ago

close this as we now have a Dockerfile built successfully. PR #4667