search

swiftlang / swift-package-manager

The Package Manager for the Swift Programming Language
Apache License 2.0
9.76k stars 1.35k forks source link

`--swift-sdk` option cannot uniquely select a Swift SDK #7973

Open kateinoigakukun opened 2 months ago

kateinoigakukun commented 2 months ago

Is it reproducible with SwiftPM command-line tools: swift build, swift test, swift package etc?

Description

A Swift SDK can be identified by a combination of an artifact ID and a target triple by its definition. However, the current Swift SDK selector accepted by the --swift-sdk flag only allows specifying either an artifact ID or a target triple, making it impossible to uniquely select the desired Swift SDK in the following situation:

A developer installs two Artifacts containing two Swift SDKs with target triples A and B respectively:

~/.config/swiftpm/swift-sdks
├── swift-DEVELOPMENT-SNAPSHOT-2024-06-13-a_static-linux-0.0.1.artifactbundle
│   └── swift-DEVELOPMENT-SNAPSHOT-2024-06-13-a_static-linux-0.0.1/swift-linux-musl/musl-1.2.5.sdk
│       ├── aarch64
│       └── x86_64
└── swift-DEVELOPMENT-SNAPSHOT-2024-07-02-a_static-linux-0.0.1.artifactbundle
    └── swift-DEVELOPMENT-SNAPSHOT-2024-07-02-a_static-linux-0.0.1/swift-linux-musl/musl-1.2.5.sdk
        ├── aarch64
        └── x86_64

Then there are 4 Swift SDKs for:

If a developer specifies swift build --swift-sdk x86_64-swift-linux-musl, there are two candidates (Swift SDK B and D) If a developer specifies swift build --swift-sdk swift-DEVELOPMENT-SNAPSHOT-2024-07-02-a_static-linux-0.0.1, there are still two candidates (Swift SDK C and D)

Potential Solutions

I think we have two options to allow a developer to specify the combination of an Artifact ID and a target triple:

  1. Use value specified by --triple option if specified with --swift-sdk (Currently, --swift-sdk and --triple are both specified, --triple option is preferred, and --swift-sdk is just ignored.)
    • Pros: No new option
    • Cons: It's a little bit confusing because --swift-sdk does not identify a Swift SDK but just identify an Artifact containing Swift SDKs.
  2. Extend the Swift SDK selector expression to allow specifying the both. e.g. --swift-sdk swift-DEVELOPMENT-SNAPSHOT-2024-07-02-a_static-linux-0.0.1:x86_64-swift-linux-musl
    • Pros: --swift-sdk identifies a Swift SDK clearly
    • Cons: As we don't limit the character set of an artifact ID, so not sure if we can interpret the selector expression without ambiguity
  3. Assign Swift SDK ID to each SDK that are independent from artifact ID
  4. Restrict an artifact to have only a single Swift SDK

Swift Package Manager version/commit hash

da65429043cb0a5151bdd5e9666df770dfbc72a5

kateinoigakukun commented 2 months ago

I'd like to hear @MaxDesiatov's opinion here :pray:

dschaefer2 commented 2 months ago

I only have a higher level comment for now. This is a confusing area with Swift SDKs I stumbled on early. It feels like Swift SDKs should be versioned like Packages are and you should be able to specify what SDKs you depend on with version constraints. I'd like to see the two concepts come together, maybe where SDKs are define in Packages, for example.

But much more study is needed to understand the full side effects of that. And does that even help solve the problem you are having here?

kateinoigakukun commented 2 months ago

I'm not sure if a package should declare SDK dependencies because a package itself does not depend on a SDK and builder developers do. And this problem is not related to SDK version management, but the structure of SDK identification.

However, I totally agree that we need to re-organize concepts around Swift SDK. I'll post several concerns of the current Swift SDK design later separately anyway.

dschaefer2 commented 2 months ago

However, I totally agree that we need to re-organize concepts around Swift SDK. I'll post several concerns of the current Swift SDK design later separately anyway.

Awesome. Thanks!

finagolfin commented 2 months ago

I repeatedly hammered on exactly this issue during the review of that proposal, but for whatever reason, Max refused to address it.

stevapple commented 2 months ago

Use value specified by --triple option if specified with --swift-sdk (Currently, --swift-sdk and --triple are both specified, --triple option is preferred, and --swift-sdk is just ignored.)

  • Pros: No new option
  • Cons: It's a little bit confusing because --swift-sdk does not identify a Swift SDK but just identify an Artifact containing Swift SDKs.

I would like to go one step further in this way: if only one SDK is capable for the specified --triple, swift build should use it without requiring --swift-sdk. This makes SDK selection fully deterministic by --triple and --swift-sdk, and allows omitting either of them if there's no ambiguity.

MaxDesiatov commented 2 months ago

I would like to go one step further in this way: if only one SDK is capable for the specified --triple, swift build should use it without requiring --swift-sdk. This makes SDK selection fully deterministic by --triple and --swift-sdk, and allows omitting either of them if there's no ambiguity.

That one's already tracked as https://github.com/swiftlang/swift-package-manager/issues/7330

MaxDesiatov commented 2 months ago

Currently, --swift-sdk and --triple are both specified, --triple option is preferred, and --swift-sdk is just ignored.

IMO, it's a bug. We should implement option 1 (i.e. not ignore --swift-sdk), unless there are known important cases where this would be a harmful regression and/or egregiously breaking change.

dschaefer2 commented 2 months ago

The problem is that if you take this out further and we'll have multiple Linux SDKs for cross compilation, they'll all have the same triple. We need a way to identify which Linux/SDK you're trying to target.

I also think there's a mix of requirements here. A lot of packages are cross platform and can build with any SDK. I do think we need to also allow for package targets that only support a subset of SDKs, to take advantage of special capabilities they may have for example. This will be especially important as we add in support for embedded. We should at least allow for targets to be constrained to a specific set of SDKs.

I'd also love if we were able to put together a strong ecosystem of SDKs that could be managed by package registries, for example. And at that point we'll need it to be easy for users to select which SDKs they want to target and "everything just works" :tm: