Transition from toolchain Foundation to package

parkera commented 1 year ago

In the Swift 6 timeframe, we should see if we can stop including swift-corelibs-foundation in the Linux toolchain completely, and transition clients to the package.

It's unclear what kind of knock-on effects this may have on existing clients which import Foundation, and Swift package manifests which have an import Foundation themselves for usage of the Bundle type.

parkera commented 1 year ago

cc @FranzBusch

FranzBusch commented 1 year ago

Thanks for opening this issue @parkera. There are few different things we have to consider here.

Developing cross platform packages

Currently, Foundation is present on Darwin and the Linux SDKs which allows users to just write import Foundation. Going forward we want users to add a dependency on this package to pull in Foundation on Linux. We already have a working model for this in https://github.com/apple/swift-crypto where we are using these compiler conditionals:

#if (os(macOS) || os(iOS) || os(watchOS) || os(tvOS)) && CRYPTO_IN_SWIFTPM && !CRYPTO_IN_SWIFTPM_FORCE_BUILD_API
@_exported import CryptoKit
#else
...
#endif

This works very nicely in swift-crypto and enables the following workflows:

Adopters can depend on the package and write their own cross platform package with the same APIs
We are picking the types from the Darwin SDK when possible
As a developer of swift-crypto you can force building the package implementation by setting the environment variable

The "downside" here is that new APIs only become available cross platform once an OS release includes it. However, it already allows experimentation before with the package. I would highly recommend this approach here as well. @Lukasa if you have any thoughts on this please feel free to chime in as well!

Impact on existing packages

The impact on existing packages can be categorised into three distinct groups.

Darwin only packages
Cross platform packages that use Foundation
Packages that currently avoid using Foundation

The transition for the first package is easy since they won't have to change anything. The second group is the most interesting one. In the best case, they just have to add the dependency to their manifest it should work. In the worst case, they just lost some APIs with no replacement. The last group is where we have a lot of potential to regain their trust and get them to adopt the new split up modules from Foundation.

Impact on places where Foundation is implicitly available

One of the bigger challenges is to first identify all places where Foundation is available and used right now and then finding solutions for those. This is a non complete list of places:

Package.swift file
SPM plugin implementations
Macro implementations
SPM generated code for resources uses Foundation.Bundle
~AFAIK Foundation is linked in when we have an async main method~ Just checked and this is not the case

In my opinion, we should decide on a solution for first topic before we are going to tackle the other ones. Since, the distribution and usage of the new packages probably impacts the solution space for the other two.

0xTim commented 1 year ago

One thing I want to call out is that I haven't seen mentioned currently is Swift scripts, e.g. executable files that begin:

#!/usr/bin/swift
import Foundation

These can just be run as ./myScript.swift and would be another thing that would break with the above changes

stevapple commented 1 year ago

Given how Foundation is widely used in the Swift world, I was pessimistic about removing swift-corelibs-foundation from the SDK. There’s another way to ease the transition without breaking compatibility and is different from swift-crypto way, as shown by swift-system. To summarize:

Use FoundationPackage as the name of umbrella module of the new Foundation project. If one wants to switch to this package completely, the simplest way is to add the dependency and change import Foundation into import FoundationPackage.
For most users, use conditional imports like
```
#if canImport(FoundationEssentials)
import FoundationEssentials
#else
import Foundation
#endif
```
to use the new implementation by default. Users may need to care about breakage between the two implementations.
For legacy codes, keep import Foundation behavior as-is. We can emit compiler notes or warnings some time to push users migrating their codes.

compnerd commented 1 year ago

At least on Windows, Foundation is critical to the toolchain itself - the driver and SPM depend on Foundation pretty heavily due to URL, FileManager, and Process at the very least.

Lukasa commented 1 year ago

For what it's worth, canImport doesn't play super well with SwiftPM. It can be extremely brittle based on how SwiftPM arranges the compile order, and that can be machine and cache dependent. This would tend to lead to weird and surprising bugs.

FranzBusch commented 1 year ago

@stevapple even if canImport would properly work in SPM it still has the downside that all the Darwin SDK frameworks that expect Foundation types wouldn't be usable with the package. This is one of the major benefits of SwiftCrypto approach since it really uses the symbols from the SDK when running on Darwin.

apple / swift-foundation