According to your CI, "latest" on macOS appears to b 5.9.2

[dabrahams]

https://github.com/SwiftyLab/setup-swift/actions/runs/8107841107/job/22160040552

Also someone I know explicitly requesting 5.10 with development: false is getting a development snapshot.

[tayloraswift]

link CI log: https://github.com/tayloraswift/swift-unidoc/actions/runs/8397721460/job/23001498576#step:2:21

[soumyamahunt]

For Swift 5.10, it seems apple has changed installation for Windows, which is why I didn't update this action to support Swift 5.10.

Unfortunately I don't have any Windows device to check what is the change, so temporarily I have decided to not support Swift 5.10 and above on Windows, the latest release should allow you to use Swift 5.10 stable toolchains.

Also someone I know explicitly requesting 5.10 with development: false is getting a development snapshot.

Yeah this is expected on previous release which didn't have Swift 5.10 stable toolchain support.

[dabrahams]

If you tell me what you need to know about the windows installation I can check it on a VM. I can also put you in touch with @compnerd, who maintains swift on Windows

[soumyamahunt]

@dabrahams I would love the following details for adding Swift 5.10 and above support for Windows:

• Environment variables before Swift installation.
• Environment variables after Swift installation.
• Swift toolchain and SDK installation directories (the more details you can share about the paths the better)

If you can provide these details for both Swift 5.10 stable toolchain snapshot and latest Swift 6 development snapshot, it will help me identify patterns for future versions and coming with a generalized solution.

[compnerd]

The environment variables are subject to change on any release - they are not guaranteed. You will have to do that manually per release. The paths are likewise dependent on the release. As a concrete example, we changed the structure between 5.9 and 5.10. We changed environment variables in the 5.7 to 5.8 transition IIRC.

[dabrahams]

@compnerd in that case, would you be able to supply the specific information requested for the Swift 5.10 stable toolchain snapshot and latest Swift 6 development snapshot?

[compnerd]

Not really; I guess once it is out and the dust has settled, maybe? I think that the maintainer of the action should be responsible for setting up, testing, and extracting the necessary information though. I would rather that @soumyamahunt figure out how to extract that information for each release.

[dabrahams]

@soumyamahunt is there perhaps something you could run in GitHub CI (like a manually triggered action) that would dump the information you need?

[soumyamahunt]

@dabrahams currently environment variables are logged, but installation path is something that needs to be determined manually.

[dabrahams]

@compnerd perhaps there's something you could do on your end to make this information derivable? What would a user (like me) do to determine that info after installing? @soumyamahunt The idea that the user can do it but the process can't be automated is a little hard to understand.

[compnerd]

Hmm, I think that the easiest thing would be to write some tooling to extract the information from source code. The installer source code is available at https://github.com/apple/swift-installer-scripts/tree/main/platforms/Windows. You should be able to read all the XML content to extract the toolchain layouts and setup. There may be some re-introduction of C++ code for some operations, but I've tried pretty consistently to minimise that.

I do not intend to document the layout as that is an internal detail of the toolchain and something which keeps needing to change due to various changes that occur without public planning (e.g. introduction of backtracing without doing the entire implementation in the open on Windows first). The alternative is to follow all the changes as they go in to try to keep up with all the alterations.

[dabrahams]

…but surely if the GH action needs the information, a regular user might need it too, and the installer source code is not really a user-consumable artifact, is it?

@soumyamahunt how is the action using this information?

[compnerd]

It is intended to only be needed for repairing defects in the toolchain, not for actual day-to-day usage. (I would agree that it can be useful for users to understand it if we can require that they also understand the semantics of SIL and LLVM IR for using the toolchain.)

The GH action needs to understand the information only because GitHub refuses to install the package by default and the setup of the environment is such that the environment cannot be fully re-instantiated post-installation (aka a reboot is not possible) and thus the action must re-constitute the changes to apply them manually. This is a limitation of GitHub, not of the Swift toolchain distribution. If you can convince GitHub to pre-install the toolchain, I suspect that the problem would disappear.

[tayloraswift]

i believe it is unrealistic to expect an organization the size of Microsoft to accommodate our individual needs. while it is obviously not ideal, the infrastructure must adapt to GitHub, not the other way around.

[compnerd]

@tayloraswift I disagree on that - this is a design choice of Microsoft - changes cannot simply be reloaded by launching a sub-shell (e.g. on Linux, you can just either source /etc/profile or launch a new subshell). The result is that post-installation of anything, you often need to restart the session. There is no good way to do that with GHA. I'd be completely happy if GHA had a way to restart the session somehow.

[dabrahams]

Would it be possible for the GHA to get the information by crawling some directory tree? Or is it the root of the tree that's the problem? Is there a reason the Swift installer couldn't leave the information in an environment variable?

[compnerd]

The root of the tree is not guaranteed, so that needs to be determined dynamically. The environment variables are limited, so we cannot just create a random environment variable that has a set of environment variables to consider (yes, I learnt this by exhausting the environment block).

[soumyamahunt]

Hmm, I think that the easiest thing would be to write some tooling to extract the information from source code. The installer source code is available at https://github.com/apple/swift-installer-scripts/tree/main/platforms/Windows. You should be able to read all the XML content to extract the toolchain layouts and setup. There may be some re-introduction of C++ code for some operations, but I've tried pretty consistently to minimise that.

I do not intend to document the layout as that is an internal detail of the toolchain and something which keeps needing to change due to various changes that occur without public planning (e.g. introduction of backtracing without doing the entire implementation in the open on Windows first). The alternative is to follow all the changes as they go in to try to keep up with all the alterations.

@compnerd I would love if this data can be retrieved by providing some option to the installer.

…but surely if the GH action needs the information, a regular user might need it too, and the installer source code is not really a user-consumable artifact, is it?

@soumyamahunt how is the action using this information?

This action uses the path to set environment variables, and implement caching of installation. Recently I have added one fallback approach (similar to that of @compnerd's action) as part of #276, which will allow action to work even if path isn't known with the limitation being caching won't work.

[dabrahams]

@soumyamahunt That's pretty darned good! One last thought: if you've gotten that far, it means you can run the tools. At that point, are there commands you can issue that would reveal the paths you need for caching?

[soumyamahunt]

@soumyamahunt That's pretty darned good! One last thought: if you've gotten that far, it means you can run the tools. At that point, are there commands you can issue that would reveal the paths you need for caching?

As @compnerd mentioned the root installation directory path isn't guaranteed, and without knowing the path caching won't be possible. The current approach I am using is to compare difference between environment variables before and after installation and set the new/changed variables for subsequent steps in actions.

For Swift 5.10 the output comes like this:

{
  "newPaths": [
    "C:\\Users\\runneradmin\\AppData\\Local\\Programs\\Swift\\Runtimes\\5.10.0\\usr\\bin\\",
    "C:\\Users\\runneradmin\\AppData\\Local\\Programs\\Swift\\Toolchains\\5.10.0+Asserts\\usr\\bin\\",
    "C:\\Users\\runneradmin\\AppData\\Local\\Programs\\Swift\\Tools\\5.10.0\\"
  ],
  "variables": {
    "SDKROOT": "C:\\Users\\runneradmin\\AppData\\Local\\Programs\\Swift\\Platforms\\5.10.0\\Windows.platform\\Developer\\SDKs\\Windows.sdk\\"
  }
}

While I might be able to check common parent directory across all paths in newPaths to determine the root, I don't think there is any guarantee to work with future versions of toolchain. Correct me if I am wrong here @compnerd.

[dabrahams]

without knowing the path caching won't be possible.

I understand that. I'm wondering if there's a command you can run that will reveal that path once the tools are runnable. For example, maybe setting TOOLCHAINS plus xode-select -p gives you the information. Or maybe there's a small swift project you can build with swift build -v that will reveal the necessary path in the output. Just a thought.

[compnerd]

While I might be able to check common parent directory across all paths in newPaths to determine the root, I don't think there is any guarantee to work with future versions of toolchain. Correct me if I am wrong here @compnerd.

Correct. The layout has changed a few times, sometimes with a single parent, sometimes with multiple parents. Unfortunately, there is no good way to guarantee this at any given point.

Or maybe there's a small swift project you can build with swift build -v that will reveal the necessary path in the output. Just a thought.

This is something that I have thought about in the past - https://github.com/compnerd/swift-devenv. A lot has changed since then, but perhaps that can be revived and tweaked for this.