Project-local toolchain selection

[kateinoigakukun]

One of the current missing pieces compared with swiftenv is "project-local" Swift version.

@patrickfreed said https://github.com/swift-server/swiftly/issues/52#issuecomment-1537531322:

I think you've pointed out the two main use-cases, namely a local specification of the in-use toolchain (e.g. a .swift-version file or some such) or some alternative one-off syntax for selecting a version (similar to +nightly in cargo). Regarding the former, it was an explicit decision to omit that feature for now for and to see if it turned out to be something that's important to users.

In our case, this feature is the main blocker to migrating from swiftenv to swiftly, so it's critical at least for us. I'm happy to design and implement it but I'd like to hear if this feature could be accepted before starting detailed design.

This feature would depend on an architectural change: https://github.com/swift-server/swiftly/issues/52

[patrickfreed]

Project-local swift version specification was only left out for simplicity's sake because we weren't sure if it was that important to users--I think it's a totally valid feature. If it's critical to your adoption of swiftly, then I imagine there's others in the same situation, so I would be in favor of introducing it. And design/implementation is more than welcome if you'd like to contribute that, thanks in advance!

A few design notes/questions:

• Per the conversation in #52, I think the shims would need to have functionality for retrieving the actual toolchain path.
• I'm also wondering which executables we need to write shims for (is it just swift? or is it common to invoke swiftc or any of the other swift-xxx executables directly? what about lldb?).
• We'll need to provide a transition path, which I think will need to be implemented in swiftly itself due to the self-update command. Maybe we have a new field in config.json that indicates whether the transition has completed or not which we can check during install / use? (this could be something like a version number to be useful going forward)
  • In the future, we might want to consider separating the swiftly post-update functionality to a separate script.
• Should we support .swift-version for ease of transition? Or should we only support a swiftly-specific file? I somewhat like the idea of supporting the former as a backup if the swiftly-specific one isn't found.

What do you think @adam-fowler?

[kateinoigakukun]

Thanks for your consideration @patrickfreed!

I'm also wondering which executables we need to write shims for (is it just swift? or is it common to invoke swiftc or any of the other swift-xxx executables directly? what about lldb?).

I think we can follow the way macOS does for /usr/bin binaries. Some of /usr/bin binaries including swift and clang are actually shims that corporates with xcode-select status. You can find which executables are shimed by the following script:

$ for bin in /usr/bin/*; do if otool -L $bin | grep libxcselect.dylib &> /dev/null; then echo $bin; fi; done

List of shims

``` /usr/bin/DeRez /usr/bin/GetFileInfo /usr/bin/ResMerger /usr/bin/Rez /usr/bin/SetFile /usr/bin/SplitForks /usr/bin/actool /usr/bin/agvtool /usr/bin/ar /usr/bin/as /usr/bin/asa /usr/bin/atos /usr/bin/bison /usr/bin/c++ /usr/bin/c++filt /usr/bin/c89 /usr/bin/c99 /usr/bin/cc /usr/bin/clang /usr/bin/clang++ /usr/bin/clangd /usr/bin/cmpdylib /usr/bin/codesign_allocate /usr/bin/cpp /usr/bin/ctags /usr/bin/ctf_insert /usr/bin/desdp /usr/bin/dsymutil /usr/bin/dwarfdump /usr/bin/filtercalltree /usr/bin/flex /usr/bin/flex++ /usr/bin/g++ /usr/bin/gatherheaderdoc /usr/bin/gcc /usr/bin/gcov /usr/bin/genstrings /usr/bin/git /usr/bin/git-receive-pack /usr/bin/git-shell /usr/bin/git-upload-archive /usr/bin/git-upload-pack /usr/bin/gm4 /usr/bin/gnumake /usr/bin/gperf /usr/bin/hdxml2manxml /usr/bin/headerdoc2html /usr/bin/heap /usr/bin/ibtool /usr/bin/ictool /usr/bin/indent /usr/bin/install_name_tool /usr/bin/kmutil /usr/bin/ld /usr/bin/leaks /usr/bin/lex /usr/bin/libtool /usr/bin/lipo /usr/bin/lldb /usr/bin/llvm-g++ /usr/bin/llvm-gcc /usr/bin/lorder /usr/bin/m4 /usr/bin/make /usr/bin/malloc_history /usr/bin/mig /usr/bin/nm /usr/bin/nmedit /usr/bin/objdump /usr/bin/opendiff /usr/bin/otool /usr/bin/pagestuff /usr/bin/pip3 /usr/bin/python3 /usr/bin/ranlib /usr/bin/resolveLinks /usr/bin/rpcgen /usr/bin/sample /usr/bin/sdef /usr/bin/sdp /usr/bin/segedit /usr/bin/size /usr/bin/sourcekit-lsp /usr/bin/stapler /usr/bin/stringdups /usr/bin/strings /usr/bin/strip /usr/bin/swift /usr/bin/swiftc /usr/bin/symbols /usr/bin/unifdef /usr/bin/unifdefall /usr/bin/vmmap /usr/bin/vtool /usr/bin/xcdebug /usr/bin/xcode-select /usr/bin/xcodebuild /usr/bin/xcrun /usr/bin/xcscontrol /usr/bin/xcsdiagnose /usr/bin/xctrace /usr/bin/xed /usr/bin/xml2man /usr/bin/yacc ```

We can choose those executables that is included in both Swift toolchain package and /usr/bin shims.

We'll need to provide a transition path, which I think will need to be implemented in swiftly itself due to the self-update command. Maybe we have a new field in config.json that indicates whether the transition has completed or not which we can check during install / use? (this could be something like a version number to be useful going forward)

Yeah, will have detailed design later.

Should we support .swift-version for ease of transition? Or should we only support a swiftly-specific file? I somewhat like the idea of supporting the former as a backup if the swiftly-specific one isn't found.

I think we should support .swift-version for migration path from swiftenv. And also for future possibility of custom toolchain support and Swift SDK installation support, it would be nice to have more structural configuration format like rust-toolchain.toml.

[patrickfreed]

That is a large number of shims, and I think maintaining that list would be pretty difficult (e.g. if Swift removes or adds a new executable to their toolchains, we'll be out of date and users will need to update swiftly itself). To work around that, it might be best to just shim swift-specific executables plus. That would be something like:

swift-api-digester
swift-api-extract
swift-autolink-extract
swift-build
swift-build-sdk-interfaces
swift-build-tool
swiftc
swift-demangle
swift-driver
swift-frontend
swift-help
swift-package
swift-package-collection
swift-package-registry
swift-run
swift-symbolgraph-extract
swift-test
sourcekit-lsp

If we wanted to be even more minimal, we could probably just do swift, swiftc, and sourcekit-lsp, since swift gets us all the swift-xxx executables via its commands. That might be a good starting point at least.

[adam-fowler]

If we wanted to be even more minimal, we could probably just do swift, swiftc, and sourcekit-lsp, since swift gets us all the swift-xxx executables via its commands. That might be a good starting point at least.

That's pretty much how it works on macOS. swift, lldb and sourcekit-lsp are in /usr/bin and everything else is in the toolchain normally inside the Xcode.app folder. Although I'm not sure where it gets the version of clang or ld from as there are versions of this in /usr/bin.

[cmcgee1024]

150 might address this.