Add Swift 5.3 for Windows

[MaxDesiatov]

Tool information

• Tool name: Swift toolchain and SDK
• Tool license: Apache 2.0
• Add or update? Add
• Desired version: 5.3
• Approximate size: 350 MB
• If this is an add request:
  • Brief description of tool: Swift is a general-purpose, multi-paradigm, cross-platform, compiled programming language.
  • URL for tool's homepage: https://swift.org/download/
  • Provide a basic test case to validate the tool's functionality:

print("Hello, World!")

Area for Triage:

Question, Bug, or Feature?:

Virtual environments affected

• [ ] macOS 10.15
• [ ] Ubuntu 16.04 LTS
• [ ] Ubuntu 18.04 LTS
• [ ] Ubuntu 20.04 LTS
• [ ] Windows Server 2016 R2
• [x] Windows Server 2019

Can this tool be installed during the build?

Probably yes.

Tool installation time in runtime

A couple of minutes.

Are you willing to submit a PR?

Not at the moment.

[maxim-lobanov]

No objections to bake Swift into the Windows images. Feel free to submit PR if you want or we will take care about it when we have capacity

[vsafonkin]

Hello, I've installed Swift 5.3 on Windows image in GitHub Actions and I have important clarification: Swift installation requires about 2.9 Gb: link, see Install Swift 5.3 step

cc: @maxim-lobanov , @AlenaSviridenko

[AlenaSviridenko]

Hi @MaxDesiatov, considering the size of Swift 5.3 we'd like not to keep it on the image. Installation in runtime takes ~2 minutes, here are steps:

steps:
      - uses: actions/checkout@v2

      - name: Install Swift 5.3
         run: |
          Install-Binary -Url "https://github.com/compnerd/swift-build/releases/latest/download/installer.exe" -Name "installer.exe" -ArgumentList ("-q")

      - name: Set env variables
         run: |
          echo "::set-env name=SDKROOT::C:\Library\Developer\Platforms\Windows.platform\Developer\SDKs\Windows.sdk"
          echo "::add-path::C:\Library\Developer\Toolchains\unknown-Asserts-development.xctoolchain\usr\bin"
          echo "::add-path::C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\Common7\IDE\CommonExtensions\Microsoft\CMake\Ninja"

      - name: Check version
         run: swift --version

Does this installation in runtime work for you?

[MaxDesiatov]

That's fine, thank you. I hope it could be embedded in the image at some point in the future after all. (cc @compnerd)

[compnerd]

Hmm, it would be nice to be able to enable that from the default GitHub actions template. @AlenaSviridenko do you know if actions/starter-workflows has any requirements for tooling to be pre-installed?

[compnerd]

BTW, the snippet that was provided doesn't really work entirely as the path adjustment is insufficient and it doesn't setup the VSDevEnv.

Is there something that allows getting the VSDevEnv environment? I did find seanmiddleditch/gha-setup-vsdevenv which works well.

[vsafonkin]

BTW, the snippet that was provided doesn't really work entirely as the path adjustment is insufficient and it doesn't setup the VSDevEnv.

Hi @compnerd , I thought it was not necessary to do this when we're using installer.exe, am I wrong?

Is there something that allows getting the VSDevEnv environment? I did find seanmiddleditch/gha-setup-vsdevenv which works well.

May be setup-msbuild?

[AlenaSviridenko]

Hi @compnerd, not sure about any requirements for starter-workflows as we don't maintain that repo, I think the suggested tool installation should be just reasonable to add to starter workflows. However, there are a bunch of setup-swift tasks on marketplace, for example, this one where you can specify the version of Swift to be installed. Probably one of them should fit your needs.

I am going to close this issue since installation in runtime is acceptable. Please, feel free to file new issue in case of any questions. Thanks!

[compnerd]

BTW, the snippet that was provided doesn't really work entirely as the path adjustment is insufficient and it doesn't setup the VSDevEnv.

Hi @compnerd , I thought it was not necessary to do this when we're using installer.exe, am I wrong?

It is still required with the installer. The environment variables that the developer command prompt sets up are needed (primarily the INCLUDE and LIB variables) to find the system headers and libraries.

Is there something that allows getting the VSDevEnv environment? I did find seanmiddleditch/gha-setup-vsdevenv which works well.

May be setup-msbuild?

Ah, nice, that would certainly be preferable, thanks!

[alex-taffe]

Just in case anyone in the future finds this thread and is trying to figure out how to setup Swift on Windows on GitHub Actions, here’s what you need:

    - name: "Enable developer command propmpt"
      uses: ilammy/msvc-dev-cmd@v1
    - name: Install Swift
      shell: cmd
      run: |
        git clone https://github.com/compnerd/swift-build.git
        cd swift-build
        cd utilities
        python -m pip install tabulate azure-devops
        python swift-build.py --download --build-id VS2019 --latest-artifacts --filter installer.exe --quiet
        unzip installer.exe.zip
        cd installer.exe
        installer.exe /s
        call "C:\ProgramData\Microsoft\Windows\Start Menu\Programs\Visual Studio 2019\Visual Studio Tools\Developer Command Prompt for VS 2019.lnk"
        set SDKROOT=%SystemDrive%\Library\Developer\Platforms\Windows.platform\Developer\SDKs\Windows.sdk
        setlocal enableextensions
        dir "C:\Program Files (x86)\Windows Kits\10\Include"
        copy "%SDKROOT%\usr\share\ucrt.modulemap" "C:\Program Files (x86)\Windows Kits\10\Include\10.0.10240.0\ucrt\module.modulemap"
        copy "%SDKROOT%\usr\share\ucrt.modulemap" "C:\Program Files (x86)\Windows Kits\10\Include\10.0.14393.0\ucrt\module.modulemap"
        copy "%SDKROOT%\usr\share\ucrt.modulemap" "C:\Program Files (x86)\Windows Kits\10\Include\10.0.16299.0\ucrt\module.modulemap"
        copy "%SDKROOT%\usr\share\ucrt.modulemap" "C:\Program Files (x86)\Windows Kits\10\Include\10.0.17134.0\ucrt\module.modulemap"
        copy "%SDKROOT%\usr\share\ucrt.modulemap" "C:\Program Files (x86)\Windows Kits\10\Include\10.0.17763.0\ucrt\module.modulemap"
        copy "%SDKROOT%\usr\share\ucrt.modulemap" "C:\Program Files (x86)\Windows Kits\10\Include\10.0.18362.0\ucrt\module.modulemap"
        copy "%SDKROOT%\usr\share\ucrt.modulemap" "C:\Program Files (x86)\Windows Kits\10\Include\10.0.19041.0\ucrt\module.modulemap"
        copy "%SDKROOT%\usr\share\visualc.modulemap" "%VCToolsInstallDir%\include\module.modulemap"
        copy "%SDKROOT%\usr\share\visualc.apinotes" "%VCToolsInstallDir%\include\visualc.apinotes"
        copy "%SDKROOT%\usr\share\winsdk.modulemap" "C:\Program Files (x86)\Windows Kits\10\Include\10.0.10240.0\um\module.modulemap"
        copy "%SDKROOT%\usr\share\winsdk.modulemap" "C:\Program Files (x86)\Windows Kits\10\Include\10.0.14393.0\um\module.modulemap"
        copy "%SDKROOT%\usr\share\winsdk.modulemap" "C:\Program Files (x86)\Windows Kits\10\Include\10.0.16299.0\um\module.modulemap"
        copy "%SDKROOT%\usr\share\winsdk.modulemap" "C:\Program Files (x86)\Windows Kits\10\Include\10.0.17134.0\um\module.modulemap"
        copy "%SDKROOT%\usr\share\winsdk.modulemap" "C:\Program Files (x86)\Windows Kits\10\Include\10.0.17763.0\um\module.modulemap"
        copy "%SDKROOT%\usr\share\winsdk.modulemap" "C:\Program Files (x86)\Windows Kits\10\Include\10.0.18362.0\um\module.modulemap"
        copy "%SDKROOT%\usr\share\winsdk.modulemap" "C:\Program Files (x86)\Windows Kits\10\Include\10.0.19041.0\um\module.modulemap"
    - name: Add Swift to path
      shell: bash
      run: |
        echo "C:\Library\Developer\Toolchains\unknown-Asserts-development.xctoolchain\usr\bin;C:\Library\Swift\Current\bin;C:\Library\icu-67\usr\bin;C:\Library\Developer\Platforms\Windows.platform\Developer\SDKs\Windows.sdk\usr\bin;C:\Library\Swift-development\bin" >> $GITHUB_PATH
    - shell: cmd
      run: |
        refreshenv
    - name: Build Swift
      shell: cmd
      run: |
        call "C:\ProgramData\Microsoft\Windows\Start Menu\Programs\Visual Studio 2019\Visual Studio Tools\Developer Command Prompt for VS 2019.lnk"
        set SDKROOT=%SystemDrive%\Library\Developer\Platforms\Windows.platform\Developer\SDKs\Windows.sdk
        %SystemDrive%\Library\Developer\Toolchains\unknown-Asserts-development.xctoolchain\usr\bin\swift-build.exe -c release  -Xlinker /INCREMENTAL:NO -v --enable-test-discovery # do what you need here
        if not exist .build\x86_64-unknown-windows-msvc\release\MY_PROGRAM.exe exit 1 #change me
        copy %SystemDrive%\Library\Swift-development\bin\*.dll .build\x86_64-unknown-windows-msvc\release
        copy %SystemDrive%\Library\Swift-development\bin\*.exe .build\x86_64-unknown-windows-msvc\release
        copy %SystemDrive%\Library\icu-67\usr\bin\*.dll .build\x86_64-unknown-windows-msvc\release

A few notes about this.

• This downloads the latest nightly build of the Swift toolchain. As of the time of writing, the release version does not support Swift package manager, but the nightlies do
• Copying the UCRT module maps manually is necessary as %UniversalCRTSdkDir% and %UCRTVersion% are undefined on GHA for some reason. It would be better to for loop over the Include directory and all of the sub versions, copying into each, but I frankly couldn’t be bothered to figure out the loop and just brute forced it. So this may break when new versions are added to that directory
• The call into developer command prompts are necessary but can sometimes mute echos past that point, so be wary
• You will need to manually copy Swift DLL’s into your build folder to run the program, that’s why the end copies are present. Same goes if you’re exporting the binary for download through something like an AWS upload or GHA artifact
• The refreshenv stage may be unnecessary, I was too lazy to try it out without it
• Make sure to change the name of MY_PROGRAM. Swift doesn’t fail the GHA workflow if the build fails, that if statement ensures it.
• One of the module map copies failed, didn’t feel like figuring out which it was, but it doesn’t affect operation

CC @compnerd

[dstoker-cricut]

@alex-taffe Glad to see this come through today! Been dealing with Swift on Windows GitHub Actions issues for almost a week straight now. I tried your solution and did see some discrepancies. First, copy "%SDKROOT%\usr\share\winsdk.modulemap" "C:\Program Files (x86)\Windows Kits\10\Include\10.0.10240.0\um\module.modulemap" causes the system to complain that The system cannot find the path specified.. Might be due to an issue with the the GHA image but that isn't found. The second (and potentially bigger) issue is that when I try to build my system using

call "C:\ProgramData\Microsoft\Windows\Start Menu\Programs\Visual Studio 2019\Visual Studio Tools\Developer Command Prompt for VS 2019.lnk"
set SDKROOT=%SystemDrive%\Library\Developer\Platforms\Windows.platform\Developer\SDKs\Windows.sdk
%SystemDrive%\Library\Developer\Toolchains\unknown-Asserts-development.xctoolchain\usr\bin\swift build -c release

the system reports error: unable to invoke subcommand: D:\a\CriKitCLI\CriKitCLI\swift-build (). I simply copied your solution in place and then tacked on the build -c release to see what occurred. No other changes have been made. Any ideas on this? I will continue to work on it as well and will report any finding here.

[alex-taffe]

@dstoker-cricut yeah it was super painful getting it going lol. I have that same The system cannot find the path specified message, but it still ends up building properly.

On line 2 of your example, you forgot the SDKROOT after set so be careful with that.

The explicit command I’ve been running for swift build is like this:

%SystemDrive%\Library\Developer\Toolchains\unknown-Asserts-development.xctoolchain\usr\bin\swift-build.exe -c release  -Xlinker /INCREMENTAL:NO -v --enable-test-discovery

The incremental flag seems to be necessary

[dstoker-cricut]

@alex-taffe Oops you are right I accidentally deleted the SDKROOT word while formatting it here in Markdown lol. I edited the post above to read properly. Looks like things are moving forward now! Thank you so much!

[alex-taffe]

No problem! One other thing to note is that Swift for whatever reason won’t fail the action if the build fails. I added:

if not exist .build\x86_64-unknown-windows-msvc\release\MY_PROGRAM.exe exit 1

right after the swift build command to fix that

[compnerd]

Is there something specific that you need from the very latest snapshot? Overall, that seems vastly more complicated than anything that I've needed thus far for getting GitHub Actions setup.

Overall, the only "unnecessary" part is that because PowerShell does not have an equivalent to refreshenv from CMD, you need to set 2 environment variables manually and you will need to adjust the path. However, even at that point, it is nowhere near the complexity of what you have shared.

I have ~4 projects using the following, and I know of a few other projects which have copied pretty much the same set of rules to pretty good effect:

    - uses: seanmiddleditch/gha-setup-vsdevenv@master

    - name: Install swift-DEVELOPMENT-SNAPSHOT-2020-11-17-a
      run: |
        Install-Binary -Url "https://swift.org/builds/development/windows10/swift-DEVELOPMENT-SNAPSHOT-2020-11-17-a/swift-DEVELOPMENT-SNAPSHOT-2020-11-17-a-windows10.exe" -Name "installer.exe" -ArgumentList ("-q")
    - name: Set Environment Variables
      run: |
        echo "SDKROOT=C:\Library\Developer\Platforms\Windows.platform\Developer\SDKs\Windows.sdk" | Out-File -FilePath $env:GITHUB_ENV -Encoding utf8 -Append
        echo "DEVELOPER_DIR=C:\Library\Developer" | Out-File -FilePath $env:GITHUB_ENV -Encoding utf8 -Append
    - name: Adjust Paths
      run: |
        echo "C:\Library\Swift-development\bin;C:\Library\icu-67\usr\bin" | Out-File -FilePath $env:GITHUB_PATH -Encoding utf8 -Append
        echo "C:\Library\Developer\Toolchains\unknown-Asserts-development.xctoolchain\usr\bin" | Out-File -FilePath $env:GITHUB_PATH -Encoding utf8 -Append
    - name: Install Supporting Files
      run: |
        Copy-Item "$env:SDKROOT\usr\share\ucrt.modulemap" -destination "$env:UniversalCRTSdkDir\Include\$env:UCRTVersion\ucrt\module.modulemap"
        Copy-Item "$env:SDKROOT\usr\share\visualc.modulemap" -destination "$env:VCToolsInstallDir\include\module.modulemap"
        Copy-Item "$env:SDKROOT\usr\share\visualc.apinotes" -destination "$env:VCToolsInstallDir\include\visualc.apinotes"
        Copy-Item "$env:SDKROOT\usr\share\winsdk.modulemap" -destination "$env:UniversalCRTSdkDir\Include\$env:UCRTVersion\um\module.modulemap"

That augmented properly in the default Swift workflow default value should be more than sufficient for getting a working environment going. Of course, it comes at the slight cost of the snapshot needing to be updated manually. However, that should be easy enough to swap out in favour of the installation via the swift-build.py.

[dstoker-cricut]

Looks like @compnerd is correct, the simplified version works really well on our projects. I don't mind the snapshot needing to be updated manually as we can pin a version that makes sense for us, though we will probably link to a release version eventually. Thanks!

[MaxDesiatov]

TWIMC, I've grouped these steps into a single composite action, which currently uses Swift 5.4.2 by default. https://github.com/marketplace/actions/swift-for-windows-action

[compnerd]

My recommended handling for installation has changed with the 5.5 release now officially complete. (@MaxDesiatov, there was a reason that I didn't make my original composite action more broadly accessible ;-)) I now recommend this:

strategy:
  matrix:
    include:
      - branch: swift-5.5-release
        tag: 5.5-RELEASE

steps:
  - name: Install Swift ${{ matrix.tag }}
    run: |
      function Update-EnvironmentVariables {
        foreach ($level in "Machine", "User") {
          [Environment]::GetEnvironmentVariables($level).GetEnumerator() | % {
            # For Path variables, append the new values, if they're not already in there
            if ($_.Name -Match 'Path$') {
              $_.Value = ($((Get-Content "Env:$($_.Name)") + ";$($_.Value)") -Split ';' | Select -Unique) -Join ';'
            }
            $_
          } | Set-Content -Path { "Env:$($_.Name)" }
        }
      }

      Install-Binary -Url "https://swift.org/builds/${{ matrix.branch }}/windows10/swift-${{ matrix.tag }}/swift-${{ matrix.tag }}-windows10.exe" -Name "installer.exe" -ArgumentList ("-q")
      Update-EnvironmentVariables

      # Reset Path and environment
      echo "$env:Path" | Out-File -FilePath $env:GITHUB_PATH -Encoding utf8
      Get-ChildItem Env: | % { echo "$($_.Name)=$($_.Value)" | Out-File -FilePath $env:GITHUB_ENV -Encoding utf8 -Append }

The reason for the matrix is to allow constructing the URL more easily and makes updating the snapshots easier. It also does allow testing for multiple versions, which is a nice bonus.

If there is a simpler way to update the environment for the runner, that would be amazing to use as the vast majority of the complexity is in the refreshing of the environment. I'll note that the image generation scripts have something similar though not as thorough as the function that I use. Perhaps adding that to the Windows environment by default would be possible, as that would simplify this to Install-Binary ; Update-EnvironmentVariables

I do have this as a more accessible composite action at compnerd/gha-setup-swift (which is expected to grow additional features).