Suggestion: Using GitHub cache action to cache the downloaded files between CI runs

[ravenexp]

GitHub Action CI provides basic support for caching downloaded dependencies: https://docs.github.com/en/actions/using-workflows/caching-dependencies-to-speed-up-workflows#using-the-cache-action

OTOH, the build.yml workflow file contains the following lines:

      - name: Test using the installed tools
        run: |
          test/test.sh $(pwd)/msvc
      # Intentionally not storing any artifacts with the downloaded tools;
      # the installed files aren't redistributable!

I've been working on the automated WDK setup, which adds another 700 MB to the total downloads size, this will increase the load on the Microsoft servers even more if added to the default CI pipeline.

Is it ok to use GitHub Actions caching given that the downloaded files are not redistributable?

[mstorsjo]

Hmm, that is a very good question. I'm not entirely sure. Is it possible to, outside of github actions, fetch a snapshot from an actions cache?

Does this download add much extra runtime to the pipeline, or is the concern purely out of kindness towards the MS download servers?

In the current setting, the default pipeline already downloads a lot (one run says Selected 520 packages, for a total download size of 2.7 GB, install size of 8.8 GB), so another 700 MB to download on top of that, while it in itself is a lot, still wouldn't be that big relative increase.

While we do run the pipeline every now and then (on each push and each PR), I think the load might not be that bad (as it is running on GitHub, which is Microsoft/Azure hosted, and fetching from MS servers).

[ravenexp]

Does this download add much extra runtime to the pipeline, or is the concern purely out of kindness towards the MS download servers?

It takes significant time to download just the WDK, but the again our CI does not run on Azure, so I'm not sure about the impact here.

I guess I'll just add WDK installation to the GitHub CI in my feature branch and see.

[huangqinjin]

A GitHub Action like https://github.com/msys2/setup-msys2 is highly desirable from my POV, which provides easy installation and caching, (though current installation is already easy).

[huangqinjin]

WDK now it is available on NuGet (105 MB in size) https://learn.microsoft.com/en-us/windows-hardware/drivers/install-the-wdk-using-nuget. Is it enough to setup the development?

[ravenexp]

WDK now it is available on NuGet (105 MB in size) https://learn.microsoft.com/en-us/windows-hardware/drivers/install-the-wdk-using-nuget. Is it enough to setup the development?

Only the latest WDK version (10.0.26100.1) seems to be available on nuget.org, and by design the WDK version must match the Windows SDK version exactly. So I guess switching to NuGet install can only be done after upgrading the Windows SDK from v10.0.22621 to v10.0.26100.

It also adds a new dependency on nuget.exe CLI tool, which can not be installed by the current MSVC installer (at least I couldn't figure out how to do it) and must be installed separately.

[huangqinjin]

A nupkg is just a normal zip file, so the python zipfile package is enough to unzip it, nuget.exe is not required.

[ravenexp]

I tried unzipping .nupkg, but it looks its internal directory structure will require some patching before it could be applied to an existing windows kits directory that vsdownload.py creates. BTW, it also declares https://www.nuget.org/packages/Microsoft.Windows.SDK.CPP as its direct dependency. Maybe it makes more sense to install both of those as NuGet packages at the same time? The current Win10 SDK unpacking code in vsdownload.py is anything but simple.