swiftlang / swift-package-manager

The Package Manager for the Swift Programming Language
Apache License 2.0
9.75k stars 1.35k forks source link

Swift test targets using C targets don't seem to respect C language standard. #7997

Open Lancelotbronner opened 2 months ago

Lancelotbronner commented 2 months ago

Is it reproducible with SwiftPM command-line tools: swift build, swift test, swift package etc?

Description

Using C23 features will compile just fine (with cLanguageStandard: .c2x until you try to use them from a test target, the build will now fail on C23 features (bool without stdbool.h, digit separators 123'456, etc.).

Expected behavior

I would expect the C target to be compiled with the C23 language standard.

Actual behavior

Builds fail because the C target is built with the wrong standard.

Steps to reproduce

  1. You can use the following reproduction package: repro-c2x.zip
  2. Try to build the test target, both swift test and Xcode will fail to build with could not build Objective-C module 'crepro' because of Unknown type name 'bool'.

Swift Package Manager version/commit hash

see below

Swift & OS version (output of swift --version ; uname -a)

swift-driver version: 1.115 Apple Swift version 6.0 (swiftlang-6.0.0.9.10 clang-1600.0.26.2) Target: arm64-apple-macosx15.0 Darwin [redacted] 24.0.0 Darwin Kernel Version 24.0.0: Mon Aug 12 20:51:54 PDT 2024; root:xnu-11215.1.10~2/RELEASE_ARM64_T6000 arm64

Lancelotbronner commented 2 months ago

Tweaked my project a bit to avoid C23 features (temporarily) and now I get the following crash:

Stack dump:
0.  Program arguments: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/swift-frontend -frontend -c /Users/lancelot/Developer/System/KnightOS/Toolchain/kcc2/Tests/DeclarationTests.swift /Users/lancelot/Developer/System/KnightOS/Toolchain/kcc2/Tests/ExpressionTests.swift /Users/lancelot/Developer/System/KnightOS/Toolchain/kcc2/Tests/IntegerConstantTests.swift /Users/lancelot/Developer/System/KnightOS/Toolchain/kcc2/Tests/LexerTests.swift -primary-file /Users/lancelot/Developer/System/KnightOS/Toolchain/kcc2/Tests/XCCompilerTestCase.swift -emit-dependencies-path /Users/lancelot/Library/Developer/Xcode/DerivedData/kcc2-dezpkjaetpuntngfagfnikbnrlkb/Build/Intermediates.noindex/kcc2.build/Debug/KccTests.build/Objects-normal/arm64/XCCompilerTestCase.d -emit-const-values-path /Users/lancelot/Library/Developer/Xcode/DerivedData/kcc2-dezpkjaetpuntngfagfnikbnrlkb/Build/Intermediates.noindex/kcc2.build/Debug/KccTests.build/Objects-normal/arm64/XCCompilerTestCase.swiftconstvalues -emit-reference-dependencies-path /Users/lancelot/Library/Developer/Xcode/DerivedData/kcc2-dezpkjaetpuntngfagfnikbnrlkb/Build/Intermediates.noindex/kcc2.build/Debug/KccTests.build/Objects-normal/arm64/XCCompilerTestCase.swiftdeps -serialize-diagnostics-path /Users/lancelot/Library/Developer/Xcode/DerivedData/kcc2-dezpkjaetpuntngfagfnikbnrlkb/Build/Intermediates.noindex/kcc2.build/Debug/KccTests.build/Objects-normal/arm64/XCCompilerTestCase.dia -target arm64-apple-macos13.0 -Xllvm -aarch64-use-tbi -enable-objc-interop -stack-check -sdk /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX15.0.sdk -I /Users/lancelot/Library/Developer/Xcode/DerivedData/kcc2-dezpkjaetpuntngfagfnikbnrlkb/Build/Products/Debug -I /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/usr/lib -F /Users/lancelot/Library/Developer/Xcode/DerivedData/kcc2-dezpkjaetpuntngfagfnikbnrlkb/Build/Products/Debug/PackageFrameworks -F /Users/lancelot/Library/Developer/Xcode/DerivedData/kcc2-dezpkjaetpuntngfagfnikbnrlkb/Build/Products/Debug -F /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/Library/Frameworks -no-color-diagnostics -enable-testing -g -debug-info-format=dwarf -dwarf-version=4 -module-cache-path /Users/lancelot/Library/Developer/Xcode/DerivedData/ModuleCache.noindex -profile-generate -profile-coverage-mapping -swift-version 6 -enforce-exclusivity=checked -Onone -D SWIFT_PACKAGE -D DEBUG -D Xcode -serialize-debugging-options -const-gather-protocols-file /Users/lancelot/Library/Developer/Xcode/DerivedData/kcc2-dezpkjaetpuntngfagfnikbnrlkb/Build/Intermediates.noindex/kcc2.build/Debug/KccTests.build/Objects-normal/arm64/KccTests_const_extract_protocols.json -enable-experimental-feature DebugDescriptionMacro -enable-experimental-feature OpaqueTypeErasure -empty-abi-descriptor -plugin-path /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/swift/host/plugins/testing -validate-clang-modules-once -clang-build-session-file /Users/lancelot/Library/Developer/Xcode/DerivedData/ModuleCache.noindex/Session.modulevalidation -Xcc -working-directory -Xcc /Users/lancelot/Developer/System/KnightOS/Toolchain/kcc2/.swiftpm/xcode -resource-dir /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/swift -enable-anonymous-context-mangled-names -file-compilation-dir /Users/lancelot/Developer/System/KnightOS/Toolchain/kcc2/.swiftpm/xcode -Xcc -fmodule-map-file=/Users/lancelot/Library/Developer/Xcode/DerivedData/kcc2-dezpkjaetpuntngfagfnikbnrlkb/Build/Intermediates.noindex/GeneratedModuleMaps/libkcc.modulemap -Xcc -D_LIBCPP_HARDENING_MODE=_LIBCPP_HARDENING_MODE_DEBUG -Xcc -ivfsstatcache -Xcc /Users/lancelot/Library/Developer/Xcode/DerivedData/SDKStatCaches.noindex/macosx15.0-24A336-b74dfec4bc4e9d23bc944d435a01a688.sdkstatcache -Xcc -I/Users/lancelot/Developer/System/KnightOS/Toolchain/kcc2/Sources/libkcc/include -Xcc -I/Users/lancelot/Library/Developer/Xcode/DerivedData/kcc2-dezpkjaetpuntngfagfnikbnrlkb/Build/Products/Debug/include -Xcc -I/Users/lancelot/Library/Developer/Xcode/DerivedData/kcc2-dezpkjaetpuntngfagfnikbnrlkb/Build/Intermediates.noindex/kcc2.build/Debug/KccTests.build/DerivedSources-normal/arm64 -Xcc -I/Users/lancelot/Library/Developer/Xcode/DerivedData/kcc2-dezpkjaetpuntngfagfnikbnrlkb/Build/Intermediates.noindex/kcc2.build/Debug/KccTests.build/DerivedSources/arm64 -Xcc -I/Users/lancelot/Library/Developer/Xcode/DerivedData/kcc2-dezpkjaetpuntngfagfnikbnrlkb/Build/Intermediates.noindex/kcc2.build/Debug/KccTests.build/DerivedSources -Xcc -DSWIFT_PACKAGE -Xcc -DDEBUG=1 -module-name KccTests -package-name kcc2 -frontend-parseable-output -disable-clang-spi -target-sdk-version 15.0 -target-sdk-name macosx15.0 -external-plugin-path /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/usr/lib/swift/host/plugins#/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/usr/bin/swift-plugin-server -external-plugin-path /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/usr/local/lib/swift/host/plugins#/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/usr/bin/swift-plugin-server -plugin-path /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/swift/host/plugins -plugin-path /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/local/lib/swift/host/plugins -o /Users/lancelot/Library/Developer/Xcode/DerivedData/kcc2-dezpkjaetpuntngfagfnikbnrlkb/Build/Intermediates.noindex/kcc2.build/Debug/KccTests.build/Objects-normal/arm64/XCCompilerTestCase.o -index-unit-output-path /kcc2.build/Debug/KccTests.build/Objects-normal/arm64/XCCompilerTestCase.o -index-store-path /Users/lancelot/Library/Developer/Xcode/DerivedData/kcc2-dezpkjaetpuntngfagfnikbnrlkb/Index.noindex/DataStore -index-system-modules
1.  Apple Swift version 6.0 (swiftlang-6.0.0.9.10 clang-1600.0.26.2)
2.  Compiling with the current language version
3.  While evaluating request ExecuteSILPipelineRequest(Run pipelines { Mandatory Diagnostic Passes + Enabling Optimization Passes } on SIL for KccTests)
4.  While running pass #267 SILFunctionTransform "TransferNonSendable" on SILFunction "@$s8KccTests18XCCompilerTestCaseC5setUpyyF".
 for 'setUp()' (at /Users/lancelot/Developer/System/KnightOS/Toolchain/kcc2/Tests/XCCompilerTestCase.swift:22:11)
Stack dump without symbol names (ensure you have llvm-symbolizer in your PATH or set the environment var `LLVM_SYMBOLIZER_PATH` to point to it):
0  swift-frontend           0x000000010886f0fc llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) + 56
1  swift-frontend           0x000000010886d350 llvm::sys::RunSignalHandlers() + 112
2  swift-frontend           0x000000010886f6c8 SignalHandler(int) + 292
3  libsystem_platform.dylib 0x0000000186da4184 _sigtramp + 56
4  libsystem_pthread.dylib  0x0000000186d6ef70 pthread_kill + 288
5  libsystem_c.dylib        0x0000000186c7b908 abort + 128
6  swift-frontend           0x0000000104255ba8 swift::Partition::merge(swift::PartitionPrimitives::Element, swift::PartitionPrimitives::Element, bool) + 248
7  swift-frontend           0x0000000103ddcfdc swift::RegionAnalysisFunctionInfo::RegionAnalysisFunctionInfo(swift::SILFunction*, swift::PostOrderFunctionInfo*) + 14308
8  swift-frontend           0x0000000103ddff14 swift::RegionAnalysis::newFunctionAnalysis(swift::SILFunction*) + 56
9  swift-frontend           0x0000000104027040 (anonymous namespace)::TransferNonSendable::run() + 324
10 swift-frontend           0x000000010405b554 swift::SILPassManager::runFunctionPasses(unsigned int, unsigned int) + 3448
11 swift-frontend           0x0000000104057b38 swift::SILPassManager::executePassPipelinePlan(swift::SILPassPipelinePlan const&) + 11968
12 swift-frontend           0x0000000104091084 swift::SimpleRequest<swift::ExecuteSILPipelineRequest, std::__1::tuple<> (swift::SILPipelineExecutionDescriptor), (swift::RequestFlags)1>::evaluateRequest(swift::ExecuteSILPipelineRequest const&, swift::Evaluator&) + 56
13 swift-frontend           0x00000001040749e8 swift::ExecuteSILPipelineRequest::OutputType swift::Evaluator::getResultUncached<swift::ExecuteSILPipelineRequest, swift::ExecuteSILPipelineRequest::OutputType swift::evaluateOrFatal<swift::ExecuteSILPipelineRequest>(swift::Evaluator&, swift::ExecuteSILPipelineRequest)::'lambda'()>(swift::ExecuteSILPipelineRequest const&, swift::ExecuteSILPipelineRequest::OutputType swift::evaluateOrFatal<swift::ExecuteSILPipelineRequest>(swift::Evaluator&, swift::ExecuteSILPipelineRequest)::'lambda'()) + 412
14 swift-frontend           0x0000000104077164 swift::runSILDiagnosticPasses(swift::SILModule&) + 612
15 swift-frontend           0x00000001035eefc4 swift::CompilerInstance::performSILProcessing(swift::SILModule*) + 84
16 swift-frontend           0x0000000103219d68 performCompileStepsPostSILGen(swift::CompilerInstance&, std::__1::unique_ptr<swift::SILModule, std::__1::default_delete<swift::SILModule>>, llvm::PointerUnion<swift::ModuleDecl*, swift::SourceFile*>, swift::PrimarySpecificPaths const&, int&, swift::FrontendObserver*) + 1376
17 swift-frontend           0x0000000103218f74 swift::performCompileStepsPostSema(swift::CompilerInstance&, int&, swift::FrontendObserver*) + 984
18 swift-frontend           0x000000010321c228 performCompile(swift::CompilerInstance&, int&, swift::FrontendObserver*) + 1680
19 swift-frontend           0x000000010321af58 swift::performFrontend(llvm::ArrayRef<char const*>, char const*, void*, swift::FrontendObserver*) + 3572
20 swift-frontend           0x00000001031a201c swift::mainEntry(int, char const**) + 3680
21 dyld                     0x00000001869ec274 start + 2840
dschaefer2 commented 1 month ago

That's ugly :(. Could you raise the compiler crash against the compiler over at swiftlang/swift? We can look at the testing target issue here.

Lancelotbronner commented 1 month ago

Certainly, I'll try to get a small reproduction snippet for the compiler crash then open an issue over at swiftlang/swift.

Lancelotbronner commented 1 month ago

I've reduced the crash to a minimum reproduction case: https://github.com/swiftlang/swift/issues/76904

Both the SwiftPM and compiler issues are preventing me from testing in the project. Its a personal project so I can wait until the next Xcode release!

I haven't yet tested with Xcode 16.1 beta, it's my next step!

Lancelotbronner commented 1 month ago

Still doesn't seem to be on the right C language standard on Xcode 16.1 beta 3.

grynspan commented 1 month ago

tmp.zip I was able to replicate the issue with this sample package. Running the tests will print the value of __STDC_VERSION__ which should be at least 20220000 for C23, but I see 201112 indicating it's defaulting down to C11.

grynspan commented 1 month ago

@rauhul Any chance #7610 fixed this?

rauhul commented 1 month ago

I doubt it, I don't think I messed with the c language standard, only the cxx one.

grynspan commented 1 month ago

Looking at that diff, it's got a few tendrils, but if I'm reading it correctly it should be feasible to basically just insert the C equivalent of the C++ bits into the same spots.

Lancelotbronner commented 2 weeks ago

If I wanted to contribute to getting this fixed where could I start?

grynspan commented 2 weeks ago

I would start by looking at the PR I mentioned above and seeing where the cxxLanguageVersion property is used. The cLanguageVersion property ought to have similar tendrils.

Lancelotbronner commented 2 weeks ago

I believe this is the bit that propagates the C++ standard to test targets.

I'm not sure how to identify C targets. C++ can detect them using the -cxx-interoperability-mode flag.

Can the -Xcc -std=... flag's presence hurt? The C++ fix uses the value from the manifest and not anything from the module, I could just check if we have a C language standard and add the flag (when C++ interop is disabled, else I'm guessing they'll conflict).

// FIXME: this function should operate on a strongly typed buildSetting
// Move logic from PackageBuilder here.
/// Determines the arguments needed for cxx interop for this module.
func clangInteroperabilityArguments(
  // FIXME: Remove argument
  // This argument is added as a stop gap to support generating arguments
  // for tools which currently don't leverage "OTHER_SWIFT_FLAGS". In the
  // fullness of time this function should operate on a strongly typed
  // "interopMode" property of SwiftTargetBuildDescription instead of
  // digging through "OTHER_SWIFT_FLAGS" manually.
  propagateFromCurrentModuleOtherSwiftFlags: Bool
) throws -> [String] {
  func cxxInteroperabilityModeAndStandard(
      for module: ResolvedModule
  ) -> [String]? {
      let scope = self.buildParameters.createScope(for: module)
      let flags = scope.evaluate(.OTHER_SWIFT_FLAGS)
      let mode = flags.first { $0.hasPrefix("-cxx-interoperability-mode=") }
      guard let mode else { return nil }
      // FIXME: Use a stored self.cxxLanguageStandard property
      // It definitely should _never_ reach back into the manifest
      if let cxxStandard = self.package.manifest.cxxLanguageStandard {
          return [mode, "-Xcc", "-std=\(cxxStandard)"]
      } else {
          return [mode]
      }
  }

  if propagateFromCurrentModuleOtherSwiftFlags {
      // Look for cxx interop mode in the current module, if set exit early,
      // the flag is already present.
      if let args = cxxInteroperabilityModeAndStandard(for: self.target) {
          return args
      }
  }

  // Implicitly propagate cxx interop flags for generated test targets.
  // If the current module doesn't have cxx interop mode set, search
  // through the module's dependencies looking for the a module that
  // enables cxx interop and copy it's flag.
  switch self.testTargetRole {
  case .discovery, .entryPoint:
      for module in try self.target.recursiveModuleDependencies() {
          if let args = cxxInteroperabilityModeAndStandard(for: module) {
              return args
          }
      }
  default: break
  }
  return []
}