search

swiftlang / swift

The Swift Programming Language
https://swift.org
Apache License 2.0
67.65k stars 10.38k forks source link

Inconsistent Compilation Failure with Noncopyable Types in std::optional during Swift-C++ Interop #76368

Open bc-lee opened 2 months ago

bc-lee commented 2 months ago

Description

Swift 6.0 introduced support for using Noncopyable C++ types in Swift-C++ interop. While this feature works in many cases, I have encountered intermittent issues where it fails to compile. This problem occurs when a Noncopyable C++ type is wrapped in std::optional and returned to Swift.

The issue does not seem to be consistently tied to specific conditions, but it shows up more frequently when using C++20 or C++2b on macOS. I have created a minimal reproducible example that demonstrates the problem.

Reproduction

I created a two-module package, consisting of a C++ library and a Swift executable.

Tree structure of the package:

TestPackage
├── Package.swift
└── Sources
    ├── CppLib    
    │   ├── include
    │   │   └── cpp.h
    │   ├── cpp.cpp
    │   └── module.modulemap
    └── SwiftApp
        └── main.swift

Sources/CppLib/include/cpp.h

#ifndef CPPLIB_CPP_H_
#define CPPLIB_CPP_H_

#include <memory>
#include <optional>

class Noncopyable {
 public:
  explicit Noncopyable(int value);
  ~Noncopyable() = default;

  Noncopyable(const Noncopyable&) = delete;
  Noncopyable& operator=(const Noncopyable&) = delete;

  Noncopyable(Noncopyable&&) = default;
  Noncopyable& operator=(Noncopyable&&) = default;

  int value() const;

 private:
  std::unique_ptr<int> ptr_;
};

std::optional<Noncopyable> CreateNoncopyable(int value);

#endif  // CPPLIB_CPP_H_

Sources/CppLib/cpp.cpp

#include "cpp.h"

#include <cstdio>

Noncopyable::Noncopyable(int value) {
  ptr_ = std::make_unique<int>(value);
  //*ptr_ = value;
}

int Noncopyable::value() const {
  return *ptr_;
}

std::optional<Noncopyable> CreateNoncopyable(int value) {
  printf("Creating Noncopyable with value: %d, where C++ version is: %ld\n",
         value, __cplusplus);
  return Noncopyable(value);
}

Sources/CppLib/module.modulemap

module CppLib {
  header "cpp.h"
  export *
}

Sources/SwiftApp/main.swift

import CppLib

let noncopyable = CreateNoncopyable(42)
assert(noncopyable.__convertToBool())
print("Value: \(noncopyable.pointee.value())")

Package.swift

// swift-tools-version:6.0

import PackageDescription

let package = Package(
    name: "TestPackage",
    products: [
        .executable(name: "SwiftApp", targets: ["SwiftApp"])
    ],
    dependencies: [],
    targets: [
        .target(name: "CppLib"),
        .executableTarget(
            name: "SwiftApp",
            dependencies: ["CppLib"],
            swiftSettings: [.interoperabilityMode(.Cxx)]
        ),
    ]
)

// Modify the C++ language standard here to test different versions
package.cxxLanguageStandard = .cxx17
// package.cxxLanguageStandard = .cxx20
// package.cxxLanguageStandard = .cxx2b

The issue arises when running swift run --verbose to check compilation and linking steps, particularly on macOS with C++20 or C++2b. In these cases, the program fails to compile with the following error:

Click to see the error ``` /Applications/Xcode16.0.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/swift-frontend -frontend -emit-module -experimental-skip-non-inlinable-function-bodies-without-types /path/to/test-package/Sources/SwiftApp/main.swift -target arm64-apple-macosx10.13 -Xllvm -aarch64-use-tbi -enable-objc-interop -cxx-interoperability-mode=default -sdk /Applications/Xcode16.0.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX15.0.sdk -I /path/to/test-package/.build/arm64-apple-macosx/debug/Modules -I /Applications/Xcode16.0.app/Contents/Developer/Platforms/MacOSX.platform/Developer/usr/lib -F /Applications/Xcode16.0.app/Contents/Developer/Platforms/MacOSX.platform/Developer/Library/Frameworks -color-diagnostics -enable-testing -g -debug-info-format=dwarf -dwarf-version=4 -module-cache-path /path/to/test-package/.build/arm64-apple-macosx/debug/ModuleCache -swift-version 6 -Onone -D SWIFT_PACKAGE -D DEBUG -new-driver-path /Applications/Xcode16.0.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/swift-driver -entry-point-function-name SwiftApp_main -empty-abi-descriptor -resource-dir /Applications/Xcode16.0.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/swift -enable-anonymous-context-mangled-names -file-compilation-dir /path/to/test-package -Xcc -fmodule-map-file=/path/to/test-package/.build/arm64-apple-macosx/debug/CppLib.build/module.modulemap -Xcc -I -Xcc /path/to/test-package/Sources/CppLib/include -Xcc -std=c++20 -Xcc -isysroot -Xcc /Applications/Xcode16.0.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX15.0.sdk -Xcc -F -Xcc /Applications/Xcode16.0.app/Contents/Developer/Platforms/MacOSX.platform/Developer/Library/Frameworks -Xcc -fPIC -Xcc -g -module-name SwiftApp -package-name test_broken_cxx_interop -disable-clang-spi -target-sdk-version 15.0 -target-sdk-name macosx15.0 -external-plugin-path /Applications/Xcode16.0.app/Contents/Developer/Platforms/MacOSX.platform/Developer/usr/lib/swift/host/plugins#/Applications/Xcode16.0.app/Contents/Developer/Platforms/MacOSX.platform/Developer/usr/bin/swift-plugin-server -external-plugin-path /Applications/Xcode16.0.app/Contents/Developer/Platforms/MacOSX.platform/Developer/usr/local/lib/swift/host/plugins#/Applications/Xcode16.0.app/Contents/Developer/Platforms/MacOSX.platform/Developer/usr/bin/swift-plugin-server -plugin-path /Applications/Xcode16.0.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/swift/host/plugins -plugin-path /Applications/Xcode16.0.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/local/lib/swift/host/plugins -emit-module-doc-path /path/to/test-package/.build/arm64-apple-macosx/debug/Modules/SwiftApp.swiftdoc -emit-module-source-info-path /path/to/test-package/.build/arm64-apple-macosx/debug/Modules/SwiftApp.swiftsourceinfo -emit-dependencies-path /path/to/test-package/.build/arm64-apple-macosx/debug/SwiftApp.build/SwiftApp.emit-module.d -o /path/to/test-package/.build/arm64-apple-macosx/debug/Modules/SwiftApp.swiftmodule -emit-abi-descriptor-path /path/to/test-package/.build/arm64-apple-macosx/debug/Modules/SwiftApp.abi.json /Applications/Xcode16.0.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/swift-frontend -frontend -c -primary-file /path/to/test-package/Sources/SwiftApp/main.swift -emit-dependencies-path /path/to/test-package/.build/arm64-apple-macosx/debug/SwiftApp.build/main.d -emit-reference-dependencies-path /path/to/test-package/.build/arm64-apple-macosx/debug/SwiftApp.build/main.swiftdeps -target arm64-apple-macosx10.13 -Xllvm -aarch64-use-tbi -enable-objc-interop -cxx-interoperability-mode=default -sdk /Applications/Xcode16.0.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX15.0.sdk -I /path/to/test-package/.build/arm64-apple-macosx/debug/Modules -I /Applications/Xcode16.0.app/Contents/Developer/Platforms/MacOSX.platform/Developer/usr/lib -F /Applications/Xcode16.0.app/Contents/Developer/Platforms/MacOSX.platform/Developer/Library/Frameworks -color-diagnostics -enable-testing -g -debug-info-format=dwarf -dwarf-version=4 -module-cache-path /path/to/test-package/.build/arm64-apple-macosx/debug/ModuleCache -swift-version 6 -Onone -D SWIFT_PACKAGE -D DEBUG -new-driver-path /Applications/Xcode16.0.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/swift-driver -entry-point-function-name SwiftApp_main -empty-abi-descriptor -resource-dir /Applications/Xcode16.0.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/swift -enable-anonymous-context-mangled-names -file-compilation-dir /path/to/test-package -Xcc -fmodule-map-file=/path/to/test-package/.build/arm64-apple-macosx/debug/CppLib.build/module.modulemap -Xcc -I -Xcc /path/to/test-package/Sources/CppLib/include -Xcc -std=c++20 -Xcc -isysroot -Xcc /Applications/Xcode16.0.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX15.0.sdk -Xcc -F -Xcc /Applications/Xcode16.0.app/Contents/Developer/Platforms/MacOSX.platform/Developer/Library/Frameworks -Xcc -fPIC -Xcc -g -module-name SwiftApp -package-name test_broken_cxx_interop -disable-clang-spi -target-sdk-version 15.0 -target-sdk-name macosx15.0 -external-plugin-path /Applications/Xcode16.0.app/Contents/Developer/Platforms/MacOSX.platform/Developer/usr/lib/swift/host/plugins#/Applications/Xcode16.0.app/Contents/Developer/Platforms/MacOSX.platform/Developer/usr/bin/swift-plugin-server -external-plugin-path /Applications/Xcode16.0.app/Contents/Developer/Platforms/MacOSX.platform/Developer/usr/local/lib/swift/host/plugins#/Applications/Xcode16.0.app/Contents/Developer/Platforms/MacOSX.platform/Developer/usr/bin/swift-plugin-server -plugin-path /Applications/Xcode16.0.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/swift/host/plugins -plugin-path /Applications/Xcode16.0.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/local/lib/swift/host/plugins -o /path/to/test-package/.build/arm64-apple-macosx/debug/SwiftApp.build/main.swift.o -index-store-path /path/to/test-package/.build/arm64-apple-macosx/debug/index/store -index-system-modules error: emit-module command failed with exit code 1 (use -v to see invocation) /path/to/test-package/Sources/CppLib/include/cpp.h:5:2: note: in module 'std_optional' imported from /path/to/test-package/Sources/CppLib/include/cpp.h:5: 3 | 4 | #include 5 | #include | `- note: in module 'std_optional' imported from /path/to/test-package/Sources/CppLib/include/cpp.h:5: 6 | 7 | class Noncopyable { /Applications/Xcode16.0.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX15.0.sdk/usr/include/c++/v1/optional:369:5: error: no matching function for call to '__construct_at' 367 | _LIBCPP_HIDE_FROM_ABI _LIBCPP_CONSTEXPR_SINCE_CXX20 void __construct(_Args&&... __args) { 368 | _LIBCPP_ASSERT_INTERNAL(!has_value(), "__construct called for engaged __optional_storage"); 369 | std::__construct_at(std::addressof(this->__val_), std::forward<_Args>(__args)...); | `- error: no matching function for call to '__construct_at' 370 | this->__engaged_ = true; 371 | } : 374 | _LIBCPP_HIDE_FROM_ABI _LIBCPP_CONSTEXPR_SINCE_CXX20 void __construct_from(_That&& __opt) { 375 | if (__opt.has_value()) 376 | __construct(std::forward<_That>(__opt).__get()); | `- note: in instantiation of function template specialization 'std::__optional_storage_base::__construct' requested here 377 | } ... ```

Expected behavior

In all cases, the program should compile and run successfully. The program should print the value of the Noncopyable object like this:

Creating Noncopyable with value: 42, where C++ version is: 202302
Value: 42

Environment

macOS Environment: 14.6.1 Xcode 16.0 RC (swift-driver version: 1.115 Apple Swift version 6.0 (swiftlang-6.0.0.9.10 clang-1600.0.26.2)

Linux Environment: nightly-focal image from DockerHub Ubuntu 20.04.6 Swift version 6.1-dev (LLVM 6f0a61d8ba20c6a, Swift cf2af6809ecd52c) C++ Standard Library: libstdc++-9-dev(9.4.0-1ubuntu1~20.04.2)

Working cases: Linux Environment with C++17, C++20, and C++2b macOS Environment with C++17

Not working cases: macOS Environment with C++20 and C++2b

Additional information

Based on my investigation, this issue may be related to libc++ modules. There are potentially related issues:

https://github.com/swiftlang/swift/issues/76325 https://github.com/llvm/llvm-project/issues/98734

Although I suspect this could be a libc++ issue, I am filing this bug for the Swift team for completeness.

finagolfin commented 2 months ago

@bc-lee, does the Xcode libc++ module map use the rewrite, llvm/llvm-project@571178a21, from last summer? I wonder why they don't see the NDK 27 issues with a recent libc++ that I documented in #76325.

bc-lee commented 2 months ago

While Apple forked and uploaded the LLVM project to their own repository and made it public, this doesn't mean they use the same code in their internal repository. I once asked about this, and an Apple employee responded that Apple never guarantees the code in the public repository is identical to the one in their internal repository. Swift 6.0’s LLVM appears to use a branch based on stable/20230725 and LLVM 17 from upstream (I checked this), but as I mentioned, this doesn’t necessarily mean Apple uses the same code internally.

What I can confirm is that /Applications/Xcode16.0.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/c++/v1/module.modulemap has 2,076 lines and 1,000 modules. Additionally, the commit you mentioned shows no changes in the modulemap file. I can't comment on the NDK 27 issues as I don’t have any information about them.

finagolfin commented 2 months ago

this doesn’t necessarily mean Apple uses the same code internally.

I know. That's why I asked you, as I don't use macOS.

What I can confirm is that /Applications/Xcode16.0.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/c++/v1/module.modulemap has 2,076 lines and 1,000 modules.

Not sure why that's relevant here...

Additionally, the commit you mentioned shows no changes in the modulemap file.

Take a look at the sixth file in that commit, modulemap.in: it is used to generate the libc++ module map and shows extensive changes.

I can't comment on the NDK 27 issues as I don’t have any information about them.

That's fine, as I was only asking you about the macOS SDK, not the Android NDK.

I just went ahead and dumped the libc++ module maps from the stable Xcode 15.4 and some older 15.1/15.2 releases on the github CI. It appears that while the latest stable Xcode 15.4 uses that rewrite commit, the older Xcode versions don't, and the mac CI uses the older ones.

That seems to explain why my linking issue hasn't hit on macOS yet, but likely will soon.