search

actions / runner-images

GitHub Actions runner images
MIT License
10.09k stars 3.04k forks source link

vcpkg not integrated for cpp project using MSBuild #3689

Closed xarthurx closed 3 years ago

xarthurx commented 3 years ago

Description

From here: https://github.com/actions/virtual-environments/blob/main/images/win/Windows2019-Readme.md vcpkg is included in the windows virtual machine on github.

So I setup a workflow for using vcpkg and the msbuild environment in a github action.

jobs:
  build:
    runs-on: windows-latest

    steps:
    - uses: actions/checkout@v2

    - name: Add msbuild to PATH (include NuGet, vcpkg)
      uses: microsoft/setup-msbuild@v1.0.2

    - name: Install packages and dependencies
      run: |
        bootstrap-vcpkg
        vcpkg install eigen3 libigl
        vcpkg integrate install
        nuget install Grasshopper

    - name: Build
      run: msbuild

The solution is a mixed cpp + c# project, where the cpp project needs the 2 libraries installed by vcpkg.

I tested it on local machine using msbuild from command line, and everything works fine. But when using GitHub action to do it, it cannot find certain header files from the two libraries.

Run msbuild
  msbuild
  shell: C:\Program Files\PowerShell\7\pwsh.EXE -command ". '{0}'"
Microsoft (R) Build Engine version 16.10.2+857e5a733 for .NET Framework
Copyright (C) Microsoft Corporation. All rights reserved.

Building the projects in this solution one at a time. To enable parallel build, please add the "-m" switch.
Build started 7/5/2021 3:10:04 PM.
Project "D:\a\igl-grasshopper\igl-grasshopper\gh-igl.sln" on node 1 (default targets).
ValidateSolutionConfiguration:
  Building solution configuration "Debug|Any CPU".
Project "D:\a\igl-grasshopper\igl-grasshopper\gh-igl.sln" (1) is building "D:\a\igl-grasshopper\igl-grasshopper\igl_RhinoCommon\igl-RhinoCommon.csproj.metaproj" (2) on node 1 (default targets).
Project "D:\a\igl-grasshopper\igl-grasshopper\igl_RhinoCommon\igl-RhinoCommon.csproj.metaproj" (2) is building "D:\a\igl-grasshopper\igl-grasshopper\igl_cppPort\igl_cppPort.vcxproj" (3) on node 1 (default targets).
PrepareForBuild:
  Creating directory "x64\Debug\".
  Creating directory "D:\a\igl-grasshopper\igl-grasshopper\x64\Debug\".
  Creating directory "x64\Debug\igl_cppPort.tlog\".
InitializeBuildStatus:
  Creating "x64\Debug\igl_cppPort.tlog\unsuccessfulbuild" because "AlwaysCreate" was specified.
VcpkgTripletSelection:
  Using triplet "x64-windows" from "C:\vcpkg\scripts\buildsystems\msbuild\..\..\..\installed\x64-windows\"
ClCompile:
  C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.29.30037\bin\HostX86\x64\CL.exe /c /I"C:\vcpkg\scripts\buildsystems\msbuild\..\..\..\installed\x64-windows\include" /ZI /JMC /nologo /W3 /WX- /diagnostics:column /sdl /Od /D _DEBUG /D _WINDOWS /D _USRDLL /D RH_DLL_EXPORTS /D _SILENCE_ALL_CXX17_DEPRECATION_WARNINGS /D _WINDLL /D _UNICODE /D UNICODE /Gm- /EHsc /RTC1 /MDd /GS /fp:precise /permissive- /Zc:wchar_t /Zc:forScope /Zc:inline /openmp- /std:c++17 /Fo"x64\Debug\\" /Fd"x64\Debug\vc142.pdb" /external:env:EXTERNAL_INCLUDE /external:W3 /Gd /TP /FC /errorReport:queue geolib.cpp igl_functions.cpp
  geolib.cpp
D:\a\igl-grasshopper\igl-grasshopper\igl_cppPort\geolib.h(3,10): fatal error C1083: Cannot open include file: 'igl/adjacency_list.h': No such file or directory [D:\a\igl-grasshopper\igl-grasshopper\igl_cppPort\igl_cppPort.vcxproj]
  igl_functions.cpp
D:\a\igl-grasshopper\igl-grasshopper\igl_cppPort\igl_functions.h(3,10): fatal error C1083: Cannot open include file: 'Eigen/Core': No such file or directory [D:\a\igl-grasshopper\igl-grasshopper\igl_cppPort\igl_cppPort.vcxproj]
  Generating Code...
Done Building Project "D:\a\igl-grasshopper\igl-grasshopper\igl_cppPort\igl_cppPort.vcxproj" (default targets) -- FAILED.
Done Building Project "D:\a\igl-grasshopper\igl-grasshopper\igl_RhinoCommon\igl-RhinoCommon.csproj.metaproj" (default targets) -- FAILED.
Done Building Project "D:\a\igl-grasshopper\igl-grasshopper\gh-igl.sln" (default targets) -- FAILED.

Build FAILED.

"D:\a\igl-grasshopper\igl-grasshopper\gh-igl.sln" (default target) (1) ->
"D:\a\igl-grasshopper\igl-grasshopper\igl_RhinoCommon\igl-RhinoCommon.csproj.metaproj" (default target) (2) ->
"D:\a\igl-grasshopper\igl-grasshopper\igl_cppPort\igl_cppPort.vcxproj" (default target) (3) ->
(ClCompile target) -> 
  D:\a\igl-grasshopper\igl-grasshopper\igl_cppPort\geolib.h(3,10): fatal error C1083: Cannot open include file: 'igl/adjacency_list.h': No such file or directory [D:\a\igl-grasshopper\igl-grasshopper\igl_cppPort\igl_cppPort.vcxproj]
  D:\a\igl-grasshopper\igl-grasshopper\igl_cppPort\igl_functions.h(3,10): fatal error C1083: Cannot open include file: 'Eigen/Core': No such file or directory [D:\a\igl-grasshopper\igl-grasshopper\igl_cppPort\igl_cppPort.vcxproj]

    0 Warning(s)
    2 Error(s)

Time Elapsed 00:00:08.78
Error: Process completed with exit code 1.

Virtual environments affected

Image version and build link

not sure where to find.

Is it regression?

No response

Expected behavior

Project build successfully.

Actual behavior

Project build failed.

Repro steps

  1. setup a cpp project calling a header file from some of the ports in vcpkg.
  2. setup github actions for installing packages from vcpkg.
  3. using msbuildto build the project.
LeonidLapshin commented 3 years ago

Hey, @xarthurx ! We'll take a look and return back with results :)

xarthurx commented 3 years ago

@LeonidLapshin https://github.com/microsoft/vcpkg/issues/18821#event-4981852381 This may help.

MaksimZhukov commented 3 years ago

Hello @xarthurx! Thank you for the repro steps! I forked the microsoft/cpprestsdk repository and configured a workflow file to build the project. I was not able to reproduce the issue, the build passed successfully.

Could you please provide us with a public simple repository where the issue can be reproduced? Thanks!

MaksimZhukov commented 3 years ago

Hello @xarthurx! We would be grateful if you could provide us with a public repository containing repro steps for further investigation. Thanks!

xarthurx commented 3 years ago

Hello @xarthurx! We would be grateful if you could provide us with a public repository containing repro steps for further investigation. Thanks!

Hi sorry, I was on vacation. Will check this this week and update.

MaksimZhukov commented 3 years ago

Hello @xarthurx! Do you have any updates?

xarthurx commented 3 years ago

@MaksimZhukov Hello, sorry for the late reply, and I really appreciate your effort in this issue.

I've created a test repo of what I was trying to do for my own repo. I think one difference between mine and the one you mentioned in the following post is that I'm not using CMake for the cpp part, since my project is a mixed of cpp + c#.

I'm relying on the vcpkg integrate install to make Visual Studio know the vcpkg setup internally. I assume this might be the reason why the github action does not work here.

You can find my test repo here: https://github.com/xarthurx/ghActionTest

Hello @xarthurx! Thank you for the repro steps! I forked the microsoft/cpprestsdk repository and configured a workflow file to build the project. I was not able to reproduce the issue, the build passed successfully.

Could you please provide us with a public simple repository where the issue can be reproduced? Thanks!

xarthurx commented 3 years ago

You may also check what the developer of vcpkg says here: https://github.com/microsoft/vcpkg/discussions/19073

MaksimZhukov commented 3 years ago

Hello @xarthurx ! Thank you for the repro steps! I was able to reproduce the issue on my local Windows machine with the same error. I have followed the steps from the Install-Vcpkg.ps1 script to install vcpkg and I have Visual Studio 16.10.4 preinstalled. I was wondering if there are any configuration steps that have been done on your local machine and can be missed in the workflow file?

xarthurx commented 3 years ago

Hello @MaksimZhukov , you need to install libigl and eigen, as I did here: https://github.com/xarthurx/ghActionTest/blob/9c25ce21701582a160276a37e2dc0ffed5405300/.github/workflows/main.yml#L26

I think the main issue is running vcpkg integrate install on the server cannot make Visual Studio recognize the package installed in vcpkg.

MaksimZhukov commented 3 years ago

I followed all steps from your workflow, but it didn't help. I'm still getting the same error.

xarthurx commented 3 years ago

OK. This is strange. I re-run it both locally and remotely (server).

Server github-action error:

Run msbuild
  msbuild
  shell: C:\Program Files\PowerShell\7\pwsh.EXE -command ". '{0}'"
  env:
    NUGET: C:\hostedtoolcache\windows\nuget.exe\5.10.0\x64/nuget.exe
Microsoft (R) Build Engine version 16.10.2+857e5a733 for .NET Framework
Copyright (C) Microsoft Corporation. All rights reserved.

Building the projects in this solution one at a time. To enable parallel build, please add the "-m" switch.
Build started 7/31/2021 10:19:36 AM.
Project "D:\a\ghActionTest\ghActionTest\ghActionTest.sln" on node 1 (default targets).
ValidateSolutionConfiguration:
  Building solution configuration "Debug|Any CPU".
Project "D:\a\ghActionTest\ghActionTest\ghActionTest.sln" (1) is building "D:\a\ghActionTest\ghActionTest\igl_RhinoCommon\RhinoCommon.csproj.metaproj" (2) on node 1 (default targets).
Project "D:\a\ghActionTest\ghActionTest\igl_RhinoCommon\RhinoCommon.csproj.metaproj" (2) is building "D:\a\ghActionTest\ghActionTest\igl_cppPort\igl_cppPort.vcxproj" (3) on node 1 (default targets).
PrepareForBuild:
  Creating directory "x64\Debug\".
  Creating directory "D:\a\ghActionTest\ghActionTest\x64\Debug\".
  Creating directory "x64\Debug\igl_cppPort.tlog\".
InitializeBuildStatus:
  Creating "x64\Debug\igl_cppPort.tlog\unsuccessfulbuild" because "AlwaysCreate" was specified.
VcpkgTripletSelection:
  Using triplet "x64-windows" from "C:\vcpkg\scripts\buildsystems\msbuild\..\..\..\installed\x64-windows\"
ClCompile:
  C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.29.30037\bin\HostX86\x64\CL.exe /c /I"C:\vcpkg\scripts\buildsystems\msbuild\..\..\..\installed\x64-windows\include" /ZI /JMC /nologo /W3 /WX- /diagnostics:column /sdl /Od /D _DEBUG /D _WINDOWS /D _USRDLL /D RH_DLL_EXPORTS /D _SILENCE_ALL_CXX17_DEPRECATION_WARNINGS /D _WINDLL /D _UNICODE /D UNICODE /Gm- /EHsc /RTC1 /MDd /GS /fp:precise /permissive- /Zc:wchar_t /Zc:forScope /Zc:inline /openmp- /std:c++17 /Fo"x64\Debug\\" /Fd"x64\Debug\vc142.pdb" /external:W3 /Gd /TP /FC /errorReport:queue igl_functions.cpp
  igl_functions.cpp
D:\a\ghActionTest\ghActionTest\igl_cppPort\igl_functions.h(2,10): fatal error C1083: Cannot open include file: 'Eigen/Core': No such file or directory [D:\a\ghActionTest\ghActionTest\igl_cppPort\igl_cppPort.vcxproj]
Done Building Project "D:\a\ghActionTest\ghActionTest\igl_cppPort\igl_cppPort.vcxproj" (default targets) -- FAILED.
Done Building Project "D:\a\ghActionTest\ghActionTest\igl_RhinoCommon\RhinoCommon.csproj.metaproj" (default targets) -- FAILED.
Done Building Project "D:\a\ghActionTest\ghActionTest\ghActionTest.sln" (default targets) -- FAILED.

Build FAILED.

"D:\a\ghActionTest\ghActionTest\ghActionTest.sln" (default target) (1) ->
"D:\a\ghActionTest\ghActionTest\igl_RhinoCommon\RhinoCommon.csproj.metaproj" (default target) (2) ->
"D:\a\ghActionTest\ghActionTest\igl_cppPort\igl_cppPort.vcxproj" (default target) (3) ->
(ClCompile target) -> 
  D:\a\ghActionTest\ghActionTest\igl_cppPort\igl_functions.h(2,10): fatal error C1083: Cannot open include file: 'Eigen/Core': No such file or directory [D:\a\ghActionTest\ghActionTest\igl_cppPort\igl_cppPort.vcxproj]

    0 Warning(s)
    1 Error(s)

Time Elapsed 00:00:09.42
Error: Process completed with exit code 1.

Local success:

Build started...
1>------ Build started: Project: igl_cppPort, Configuration: Debug x64 ------
1>igl_functions.cpp
1>C:\Users\xarthur\source\repos\ghActionTest\igl_cppPort\igl_functions.cpp(40,32): warning C4273: 'Add': inconsistent dll linkage
1>C:\Users\xarthur\source\repos\ghActionTest\igl_cppPort\igl_functions.h(30,41): message : see previous definition of 'Add'
1>C:\Users\xarthur\source\repos\ghActionTest\igl_cppPort\igl_functions.cpp(56,1): warning C4267: 'argument': conversion from 'size_t' to '_Ty', possible loss of data
1>        with
1>        [
1>            _Ty=int
1>        ]
1>C:\Users\xarthur\source\repos\ghActionTest\igl_cppPort\igl_functions.cpp(68,39): warning C4267: '=': conversion from 'size_t' to 'int', possible loss of data
1>C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.29.30037\include\numeric(35,1): warning C4267: 'argument': conversion from 'size_t' to 'int', possible loss of data
1>C:\Users\xarthur\source\repos\ghActionTest\igl_cppPort\igl_functions.cpp(68): message : see reference to function template instantiation '_Ty std::accumulate<std::_Vector_iterator<std::_Vector_val<std::_Simple_types<std::vector<int,std::allocator<int>>>>>,size_t,igl_adjacency_list::<lambda_d5e7e7866e4c8d80e4cf2dacde6b1625>>(const _InIt,const _InIt,_Ty,_Fn)' being compiled
1>        with
1>        [
1>            _Ty=size_t,
1>            _InIt=std::_Vector_iterator<std::_Vector_val<std::_Simple_types<std::vector<int,std::allocator<int>>>>>,
1>            _Fn=igl_adjacency_list::<lambda_d5e7e7866e4c8d80e4cf2dacde6b1625>
1>        ]
1>   Creating library C:\Users\xarthur\source\repos\ghActionTest\x64\Debug\igl_cppPort.lib and object C:\Users\xarthur\source\repos\ghActionTest\x64\Debug\igl_cppPort.exp
1>igl_cppPort.vcxproj -> C:\Users\xarthur\source\repos\ghActionTest\x64\Debug\igl_cppPort.dll
1>Done building project "igl_cppPort.vcxproj".
2>------ Build started: Project: RhinoCommon, Configuration: Debug Any CPU ------
3>------ Build started: Project: Grasshopper, Configuration: Debug Any CPU ------
2>  RhinoCommon -> C:\Users\xarthur\source\repos\ghActionTest\igl_RhinoCommon\bin\igl_RhinoCommon.dll
3>C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\MSBuild\Current\Bin\Microsoft.Common.CurrentVersion.targets(2202,5): warning MSB3270: There was a mismatch between the processor architecture of the project being built "MSIL" and the processor architecture of the reference "C:\Users\xarthur\source\repos\ghActionTest\igl_RhinoCommon\bin\igl_RhinoCommon.dll", "AMD64". This mismatch may cause runtime failures. Please consider changing the targeted processor architecture of your project through the Configuration Manager so as to align the processor architectures between your project and references, or take a dependency on references with a processor architecture that matches the targeted processor architecture of your project.
3>  Grasshopper -> C:\Users\xarthur\source\repos\ghActionTest\igl_GrassHopper\bin\igl_GrassHopper.dll
3>          1 file(s) copied.
========== Build: 3 succeeded, 0 failed, 0 up-to-date, 0 skipped ==========

The error is different as I simplified some of my files again. But on my local machine, everything was fine.

BTW, the downloading of external libs in my original action script was not necessary, as vcpkg should resolve every dependency.

Could you post your error?

MaksimZhukov commented 3 years ago

I forked your repository and ran builds. They failed with the following error:

  D:\a\ghActionTest\ghActionTest\igl_cppPort\igl_functions.h(25,28): error C2653: 'Eigen': is not a class or namespace name [D:\a\ghActionTest\ghActionTest\igl_cppPort\igl_cppPort.vcxproj]
  D:\a\ghActionTest\ghActionTest\igl_cppPort\igl_functions.h(25,35): error C2061: syntax error: identifier 'MatrixXd' [D:\a\ghActionTest\ghActionTest\igl_cppPort\igl_cppPort.vcxproj]
  D:\a\ghActionTest\ghActionTest\igl_cppPort\igl_functions.h(27,28): error C2653: 'Eigen': is not a class or namespace name [D:\a\ghActionTest\ghActionTest\igl_cppPort\igl_cppPort.vcxproj]
  D:\a\ghActionTest\ghActionTest\igl_cppPort\igl_functions.h(27,35): error C2061: syntax error: identifier 'MatrixXi' [D:\a\ghActionTest\ghActionTest\igl_cppPort\igl_cppPort.vcxproj]
  D:\a\ghActionTest\ghActionTest\igl_cppPort\igl_functions.cpp(4,10): fatal error C1083: Cannot open include file: 'igl/adjacency_list.h': No such file or directory [D:\a\ghActionTest\ghActionTest\igl_cppPort\igl_cppPort.vcxproj]

I have the same error locally. You can find more details in the build logs.

I've tried to use different versions of vcpkg, but unfortunately it didn't help, all builds have the same error. Please see this workflow run for more information.

xarthurx commented 3 years ago

Yes, the error clearly shows vcpkg doesn't resolve the installed library dependency to Visual Studio. So that the installed library was not found.

Usually on a local machine, vcpkg integrate install should resolve this. Let me find a computer to check.

xarthurx commented 3 years ago

The doc also shows this should work. https://vcpkg.readthedocs.io/en/latest/examples/installing-and-using-packages/#vsmsbuild-project-user-wide-integration

xarthurx commented 3 years ago

Another possible reason I can think of, is that the vcpkg integrate install takes effect (as the doc said) for newly created VS project, but not the ones created before running the command on a certain machine.

That's why on my machine it works (I run that command before creating the VS project), but not on yours or on the server (you use the VS project I created, with some records of my machine info, but not your machine or the server).

We need to find a way to update the VS project to be aware of the new environment. But I have no clue how to do this.

MaksimZhukov commented 3 years ago

I think the vcpkg integrate install command can be used not only before creating a new VS project. For example, it is successfully used during builds of the Tracy Profiler project (see the workflow).

I forked your repository with the latest changes and updated the project file. Now the build passes without errors (link to the corresponding workflow).

xarthurx commented 3 years ago

@MaksimZhukov

Thank you for the effort. Two questions:

  1. Why do you use x64-windows-static instead of x64-windows (default)?
  2. Is the modification of my project file made by hand? (If I understand correctly, if vcpkg integrate install is working as expected, those modification should not be made manually)
MaksimZhukov commented 3 years ago

Hello @xarthurx! As I mentioned earlier, the issue is reproduced on my local machine. We install the vcpkg on images following steps from the microsoft/vcpkg repository, so I don't expect that it can cause any issues. Installing different versions of vcpkg in different directories also does not help. It seems to me that the issue is most likely related to your project and is not reproduced for you due to additional configuration of your local machine. Unfortunately, we were not able to find out what configuration changes should be made to fix the issue. I'm going to close the issue, but feel free to contact us if you find something that can help with further investigation. Thanks!

vSzemkel commented 2 years ago

As of June 2022 the command: vcpkg integrate install just works in Github Actions environment. For example see my GitHub