microsoft / vcpkg

C++ Library Manager for Windows, Linux, and MacOS
MIT License
23.38k stars 6.46k forks source link

gitlab CI runs as a service under user SYSTEM and binary caches aren't reused #35638

Open LegalizeAdulthood opened 11 months ago

LegalizeAdulthood commented 11 months ago

Describe the bug When a gitlab runner invokes vcpkg with binary caching, the artifacts are written to %LOCALAPPDATA%\vcpkg\archives as described in the documentation. Howeve, when code is executed as user SYSTEM (e.g. from inside a service), %LOCALAPPDATA% resolves to something like C:\Windows\system32\config\systemprofile\AppData\Local which is underneath a system directory with restricted permissions. The odd result is that while vcpkg correctly creates cached binary packages, it can't ever uncompress them and therefore rebuilds them on every CI build.

Environment

To Reproduce Steps to reproduce the behavior:

  1. Create a gitlab CI environment with vcpkg as a submodule
  2. Host a gitlab CI runner on a Windows machine (it runs as a service)
  3. Invoke CI builds on the Windows machine. With --debug you see output similar to the following:
    [DEBUG] 1109: CreateProcessW("C:\gitlab-runner\builds\6158xdR5T\0\[...]\vcpkg\downloads\tools\7zip-23.01-windows\7za.exe" x "C:\Windows\system32\config\systemprofile\AppData\Local\vcpkg\archives\27\2730730bb9d7fc09f765e820720ba39d77c1dde64a360371eafdf0b8b0dfe4e3.zip" "-oC:\gitlab-runner\builds\6158xdR5T\0\[...]\vcpkg\packages\openimageio_x64-windows" -y)
    [...]
    [DEBUG] 1109: cmd_execute_and_stream_data() returned 2 after    71792 us
    [...]
    [DEBUG] Failed to decompress archive package: C:\Windows\system32\config\systemprofile\AppData\Local\vcpkg\archives\27\2730730bb9d7fc09f765e820720ba39d77c1dde64a360371eafdf0b8b0dfe4e3.zip
    Restored 0 package(s) from C:\Windows\system32\config\systemprofile\AppData\Local\vcpkg\archives in 321 ms. Use --debug to see more details.

Expected behavior It's unclear what permission is missing in order for 7za.exe to correctly unzip the archive. I added Full Control for all Users and all Authenticated Users to the vcpkg subdirectory under %LOCALAPPDATA% and that wasn't enough. It could be solely due to the way 7za attempts to open it's input files; it's unclear. Perhaps vcpkg could detect when the user is SYSTEM and adopt a different default binary cache location?

Failure logs See above snippets

Additional context Sorry this is a corporate gitlab instance or I would point you at full relevant logs.

LegalizeAdulthood commented 11 months ago

I passed --debug down to vcpkg via -DVCPKG_INSTALL_OPTIONS when configuring CMake via CTest.

I also tried with and without --x-abi-tools-use-exact-versions and it made no difference on Windows.

On our Linux builds, the caching works just fine as expected, so all the evidence is pointing to the reduced permissions for %LOCALAPPDATA% for user SYSTEM.

LegalizeAdulthood commented 11 months ago

A workaround is to set VCPKG_BINARY_SOURCES to point to a local files source with a directory that has suitable permissions for all users to read/write. I validated with --debug that the new cache was populated and correctly consumed.

honeway commented 11 months ago

you can set system environment variable VCPKG_DEFAULT_BINARY_CACHE to a path to avoid the issue.

Cheney-W commented 11 months ago

https://learn.microsoft.com/en-us/vcpkg/consume/binary-caching-default?pivots=shell-cmd

LegalizeAdulthood commented 11 months ago

Not really sure why this is closed; IMO vcpkg should attempt to issue a relevant diagnostic in this scenario. Debugging this requires that you know lots of intimate details about user SYSTEM of which the average developer is unaware.

dg0yt commented 11 months ago

IMO vcpkg should refuse to run as SYSTEM.

github-actions[bot] commented 10 months ago

This is an automated message. Per our repo policy, stale issues get closed if there has been no activity in the past 28 days. The issue will be automatically closed in 14 days. If you wish to keep this issue open, please add a new comment.

Cheney-W commented 10 months ago

Keep-Alive

LegalizeAdulthood commented 10 months ago

This is an automated message. Per our repo policy, stale issues get closed if there has been no activity in the past 28 days. The issue will be automatically closed in 14 days. If you wish to keep this issue open, please add a new comment.

Dear Microsoft,

PLEASE STOP MANAGING YOUR TEAMS BY METRIC OF OPEN ISSUES.

You're not helping anyone. Don't penalize your team for having open bugs with no activity in a while. Many of us that report bugs have full-time jobs that don't involve supporting requests for more information within a narrow time window from your QA team regarding some bug that I filed.

I filed a bunch of bugs on the refactoring support in Visual Studio in 2017 for my refactoring test suite. All the bugs were closed, none with any stated fix or action taken. Some of the issues no longer occur, but the relevant bug was never updated with information.

The end result is that I'm not going to bother filing bugs on Microsoft products anymore as it is a complete waste of my time because you're in a rush to close as many bugs as possible as soon as possible instead of working to help solve the problem and actually make your products better.

LegalizeAdulthood commented 10 months ago

For comparison, there are some bugs that I filed against the refactoring support in CLion or ReSharper for C++ and those bugs remain open and available for me to add new comment should I have additional information and remain available for other people to find and easily see that they have not been fixed. When people search github for unresolved problems they search OPEN bugs not closed ones. You're pretending you have fewer problems than you actually have when you close them for "inactivity".

szakeetm commented 8 months ago

Many thanks for reporting this, I ran into the same issue and you helped