Swift compiler does not chose the correct sync/async overload

[groue]

Description

Hello,

SE-0296 contains those paragraphs in its Overloading and overload resolution section:

[...] we propose an overload-resolution rule to select the appropriate function based on the context of the call. Given a call, overload resolution prefers non-async functions within a synchronous context (because such contexts cannot contain a call to an async function). Furthermore, overload resolution prefers async functions within an asynchronous context (because such contexts should avoid stepping out of the asynchronous model into blocking APIs).

This rule is crucially important, because when a library ships an async overload of a sync function, it may well be with the intent of avoiding the grave problems that happen when the cooperative thread pool is blocked by the sync overload. The language must respect the intended design of SE-0296, and the care put by API authors into helping their users write correct programs. See this very interesting forum thread for a longer discussion.

The reason for this issue is that the overload resolution is not correctly implemented, and the compiler mistakenly allows sync calls in asynchronous contexts.

Reproduction

struct Runner {
    func run() { }
    func run() async { }
}

func testSync() {
    // 👍 sync overload chosen
    Runner().run()
}

func testAsync() async {
    // 👍 compiler error: expression is 'async' but is not marked with 'await'
    // Runner().run()
    await Runner().run()
}

func testTask() {
    Task {
        // ❌ sync overload wrongly chosen. No error and no warning.
        Runner().run()
    }
}

Expected behavior

The testTask function does not compile, and the compiler complains with "expression is 'async' but is not marked with 'await'"

Environment

Problem exists in Swift 5.10, and still exists in Xcode 16 beta:

swift-driver version: 1.90.11.1 Apple Swift version 5.10 (swiftlang-5.10.0.13 clang-1500.3.9.4) Target: arm64-apple-macosx14.0

swift-driver version: 1.109.2 Apple Swift version 6.0 (swiftlang-6.0.0.3.300 clang-1600.0.20.10) Target: arm64-apple-macosx14.0

Additional information

@_disfavoredOverload does not help. I don't know any workaround that would have the compiler correctly identify the incorrect use of the sync overload in an async context.

[groue]

Bug still present in Xcode 16.0 beta 3 (16A5202i)

[hborla]

This looks like it's behaving-as-designed based on this rule further down in the section (emphasis mine)

In non-async functions, and closures without any await expression, the compiler selects the non-async overload:

func f() async {
 let f2 = {
   // In a synchronous context, the non-async overload is preferred:
   doSomething()
 }
 f2()
}

If a closure does not have any awaits, it's considered a synchronous context, even if the contextual type is always async. Fixing this issue requires a semantic change to the language that should go through Swift Evolution.

[groue]

Hello @hborla,

Thanks for looking at this issue. I'm the author of the amendment that specified, so I'm all responsible for this.

Let's provide my own emphasis:

In non-async functions, and closures without any await expression, the compiler selects the non-async overload:

The Task initializer accepts an async closure, and I would have expected the chosen overload to be the async one.

The spirit of the amendment is that the async overload should always be preferred in async contexts, unless the programmer explicitly opts out.

The letter of the amendment has allowed the interpretation that leads to the current very asymmetrical behavior: async functions always require await (unless opt out), but async closures never require await (unless they contain another suspension point)!

Indeed

```swift func acceptAsyncParameter(_ f: () async -> Void) { } func testAsyncParameter() { acceptAsyncParameter { // 😖 sync overload chosen. Runner().run() } } ```

I'm very sorry about that. This is the wrong behavior, because blocking the cooperative thread pool can lead to deadlock. API designers who provide an async overload rely on the compiler to help them not blocking the cooperative thread pool.

cc @DougGregor, who encouraged me writing the amendment.

We need a new one, that is better written.

[groue]

BTW, since fixing this issue would be a breaking change (some code that is currently accepted would stop compiling with an "expression is 'async' but is not marked with 'await'" error message), it would be very cool to address it before Swift 6 is released for good.

🤞

[qusc]

Upgrading to Xcode 16 / Swift 6 broke my code, consider this very simple async overload of Optional.map that I have:

// Run this snippet in a Swift playground
extension Optional {
    func map<U>(_ transform: (Wrapped) async throws -> U) async rethrows -> U? {
        if let self { return try await transform(self) }
        else { return nil }
    }
}

let optional: Void? = .none

Task {
    await asyncOperation(()) // Having this line causes a compiler error for the line below!
    optional.map { print($0) }
}

func asyncOperation(_ parameter: Void) async { }

This compiled with Xcode 15.6 / Swift 5.10.

Apparently now the synchronous overload of Optional.map (The "built-in" one) doesn't take precedence in an asynchronous context anymore.

Is this intentional? If so, why?

How can I make this compile? I see no obvious way of giving one overload precedence.

[qusc]

I understand that the synchronous version is not supposed to be chosen in an asynchronous context, but there seems to be no way of choosing the asynchronous overload either now, right?! Simply adding await before calling the overload does not fix this. @groue

[groue]

This must be a bug since the async overload is not even a real option here since the line is not marked with await, right?

@qusc, this is not a bug. This is the behavior described by SE-0296 (section "Overloading and overload resolution"). The presence of await asyncOperation makes the closure an async one, and this triggers the election the async overload of map. So your example behaves exactly as expected and described by SE-0296.

This issue is about the fact that the compiler does not require async in the case below (assuming there exists an async overload of map as in your app):

Task {
    // await not required
    optional.map { print($0) }
}

I plead that this is unexpected and harmful behavior. SE-0296 acknowledges that when a function comes in with two overloads, one sync and one async, the async should be preferred in async contexts. But SE-0296 fails to recognize an async context in Task { ... }, despite the fact that the Task initializer accepts an async closure.

[qusc]

Thank you for your quick reply! Sorry, did not expect you to reply this quickly, I edited my comments, did not understand the issue itself fully when I posted.

Now that I understand that automatically choosing the synchronous version is considered harmful behavior, how can I make the compiler choose the async overload that I defined then? This is totally fine by me – however, adding await to this line does not resolve the issue, I still get the compiler error Ambiguous use of 'map' for this:

extension Optional {
    func map<U>(_ transform: (Wrapped) async throws -> U) async rethrows -> U? {
        if let self { return try await transform(self) }
        else { return nil }
    }
}

let optional: Void? = .none

Task {
    await asyncOperation(()) // Having this line causes a compiler error for the line below!
    await optional.map { print($0) }
}

func asyncOperation(_ parameter: Void) async { }

[qusc]

@groue

The presence of await asyncOperation makes the closure an async one, and this triggers the election the async overload of map.

Exactly this does not seem to be the case, the compiler error does not indicate that there is a missing await but that instead the use of the map function is ambiguous!

[groue]

Now that I understand that automatically choosing the synchronous version is considered harmful behavior

😅 Don't take this opinion of mine for granted! This issue was labelled "swift evolution proposal needed", which means that it's far from being universally acknowledged ;-)

however, adding await to this line does not resolve the issue, I still get the compiler error Ambiguous use of 'map' for this.

Now that's weird indeed! It's a distinct issue. I mean, they're related, but distinct. I would open a new GitHub issue, so that it is not merged with this one, and can be addressed independently.

[qusc]

@groue fyi: opened this issue: https://github.com/swiftlang/swift/issues/77021

[groue]

@groue fyi: opened this issue: https://github.com/swiftlang/swift/issues/77021

Looks good to me! Focused and clear :)