Possible `-whole-module-optimization` bug

[dabrahams]

Description

Tests fail in release mode unless -no-whole-module-optimization is passed.

Reproduction

Sorry, it's not a small repro, but then a WMO bug would tend not to be, right?

• Check out the possible-swift-wmo-bug tag of https://github.com/hylo-lang/hylo
• git submodule update --init
• Satisfy the prerequisites for building
• swift test --filter Execution to see a test pass
• swift test -Xswiftc -enable-testing -c release --filter Execution to see the failure (a fatalError call).

Apply this patch:

diff --git a/Package.swift b/Package.swift
index 36a041409..69f77b569 100644
--- a/Package.swift
+++ b/Package.swift
@@ -10,7 +10,7 @@ import PackageDescription

 /// Settings to be passed to swiftc for all targets.
 let allTargetsSwiftSettings: [SwiftSetting] = [
-  .unsafeFlags(["-warnings-as-errors", "-enable-testing"])
+  .unsafeFlags(["-warnings-as-errors", "-enable-testing", "-no-whole-module-optimization"])
 ]

 /// Dependencies for documentation extraction.

Repeat the command to see it not fail.

Alternatively, apply this patch

1 file changed, 1 insertion(+), 2 deletions(-)
StandardLibrary/StandardLibrary.swift | 3 +--

modified   StandardLibrary/StandardLibrary.swift
@@ -24,7 +24,6 @@ extension Utils.Host {
   /// platform.
   public static let hostedLibraryAST = Result {
     try AST(libraryRoot: hostedLibrarySourceRoot, for: CompilerConfiguration([]))
-      .roundTripSerialized()
   }

   /// An AST representing the freestanding core of standard library, conditionally compiled for
@@ -32,7 +31,7 @@ extension Utils.Host {
   public static let freestandingLibraryAST = Result {
     try AST(
       libraryRoot: freestandingLibrarySourceRoot, for: CompilerConfiguration(["freestanding"])
-    ).roundTripSerialized()
+    )
   }

 }

and repeat the command to see it not fail.

Expected behavior

Same behavior in debug and release builds; no observable change from turning on/off WMO; round-trip serialization makes no difference, or at least has the same effect in debug and release builds.

Environment

swift-driver version: 1.87.3 Apple Swift version 5.9.2 (swiftlang-5.9.2.2.56 clang-1500.1.0.2.5) Target: arm64-apple-macosx14.0

Additional information

No response

[dabrahams]

And of course, you can't do a clean build with the -no-whole-module-optimization patch applied, because of https://github.com/apple/swift-package-manager/issues/7274

[dabrahams]

Applying -no-whole-module-optimization just to the Core module is enough to suppress the fatalError.

[fumoboy007]

Hi! This may also be the cause of https://github.com/fumoboy007/msgpack-swift/issues/4. Hmm maybe not; this issue was for Swift 5.9.2 but my issue is only for Swift 5.10.

[fumoboy007]

Hmm maybe not; this issue was for Swift 5.9.2 but my issue is only for Swift 5.10.

Actually I was mistaken. https://github.com/fumoboy007/msgpack-swift/issues/4 does seem to be caused by some change in Swift 5.9.

I moved the relevant code to its own Swift package: OptimizationBugRepro.zip. The smaller package exhibits unexpected behavior with Swift 5.9.2 (release configuration, WMO enabled) but succeeds with Swift 5.8.1.

Using the prebuilt Swift development snapshots, I narrowed down the culprit to one of 85 commits. I was unable to narrow it down further because I couldn’t solve some issues with manually building the Swift compiler from source.

---

Sorry for hijacking this thread… hopefully my small reproduction test case (396 lines) can help a Swift compiler engineer easily find the culprit.

Here is main.swift:

var messageWriter = MessageWriter()
try EncodableMessagePackValue.nil.encode(to: &messageWriter)

// Should print "[192]".
print(Array(messageWriter.message))

The relevant stack looks like this:

1. main.swift calls encode(to:).
2. encode(to:) calls encodeNil(to:).
3. encodeNil(to:) calls messageWriter.write(byte: 0xC0)
4. write(byte:) calls Data.append(_:count:).

As you can see, the logic is quite straightforward. The rest of the code in the package unfortunately needs to be there; otherwise the issue disappears!