swiftlang / swift

The Swift Programming Language
https://swift.org
Apache License 2.0
67.66k stars 10.38k forks source link

A recent reorganization of the libc++ module map that shipped in NDK 27 broke building the Swift compiler and other C++ Interop #76325

Open finagolfin opened 2 months ago

finagolfin commented 2 months ago

Description

I just tried building the Swift 6.1 trunk Sep.4 source snapshot natively on Android 13 AArch64 with the new Android NDK 27 sysroot and it failed with this linker error:

/data/data/com.termux/files/usr/bin/clang++ -Wno-unknown-warning-option -Werror=unguarded-availability-new -fno-stack-protector -fPIC -fno-semantic-interposition -fvisibility-inlines-hidden -Werror=date-time -Werror=unguarded-availability-new -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wmissing-field-initializers -Wimplicit-fallthrough -Wcovered-switch-default -Wno-class-memaccess -Wno-noexcept-type -Wnon-virtual-dtor -Wdelete-non-virtual-dtor -Wsuggest-override -Wstring-conversion -Wmisleading-indentation -Wctad-maybe-unsupported -fdiagnostics-color -ffunction-sections -fdata-sections -O3 -DNDEBUG -Wl,--color-diagnostics    -target aarch64-unknown-linux-android24 -fno-termux-rpath -fuse-ld=lld -Xlinker --build-id=sha1 -Xlinker -z -Xlinker nostart-stop-gc tools/driver/CMakeFiles/swift-frontend.dir/driver.cpp.o -o bin/swift-frontend -L/data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/./lib   -L/data/data/com.termux/files/home/build/aug6/toolchain-android-aarch64/usr/lib/swift/android   -L/data/data/com.termux/files/home/build/aug6/toolchain-android-aarch64/usr/lib/swift/android/aarch64   -L/data/data/com.termux/files/home/build/Ninja-Release/swift-android-aarch64/lib -Wl,-rpath,"\$ORIGIN/../lib/swift/android:/data/data/com.termux/files/usr/lib:\$ORIGIN/../lib/swift/host/compiler"  -ldl  -llog  -lm  /data/data/com.termux/files/usr/lib/libc++_shared.so  /data/data/com.termux/files/home/build/aug6/toolchain-android-aarch64/usr/lib/swift/android/aarch64/swiftrt.o  -lswiftCore  lib/libswiftDriverTool.a  lib/libswiftCompilerModules.a  -lswiftCore  lib/libswiftFrontendTool.a  lib/libswiftDependencyScan.a  lib/libswiftDependencyScanImpl.a  lib/libswiftImmediate.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libLLVMMCJIT.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libLLVMOrcJIT.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libLLVMExecutionEngine.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libLLVMRuntimeDyld.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libLLVMJITLink.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libLLVMOrcTargetProcess.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libLLVMOrcShared.a  lib/libswiftIndex.a  lib/libswiftMigrator.a  lib/libswiftPrintAsClang.a  lib/libswiftFrontend.a  lib/libswiftAPIDigester.a  lib/libswiftConstExtract.a  lib/libswiftLocalization.a  lib/libswiftIRGen.a  lib/libswiftLLVMPasses.a  lib/libswiftSILGen.a  lib/libswiftSILOptimizer.a  lib/libswiftThreading.a  lib/libswiftSymbolGraphGen.a  lib/libswiftIDE.a  lib/libswiftDriver.a  lib/libswiftSerialization.a  lib/libswiftSIL.a  lib/libswiftSema.a  lib/libswiftSerialization.a  lib/libswiftSIL.a  lib/libswiftSema.a  lib/libswiftOption.a  lib/libswiftClangImporter.a  lib/libswiftParse.a  lib/libswiftIDEUtilsBridging.a  lib/libswiftASTGen.a  lib/libswiftAST.a  lib/libswiftBasic.a  lib/libswiftDemangling.a  /data/data/com.termux/files/usr/lib/libuuid.so  lib/libswiftMarkup.a  -llog  /data/data/com.termux/files/usr/lib/libc++_shared.so  /data/data/com.termux/files/home/build/Ninja-Release/cmark-android-aarch64/src/libcmark-gfm.a  -pthread  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libclangFrontendTool.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libclangExtractAPI.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libclangARCMigrate.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libclangStaticAnalyzerFrontend.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libclangStaticAnalyzerCheckers.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libclangStaticAnalyzerCore.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libclangCrossTU.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libclangDependencyScanning.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libLLVMX86CodeGen.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libLLVMX86AsmParser.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libLLVMX86Desc.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libLLVMX86Disassembler.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libLLVMX86Info.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libLLVMARMCodeGen.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libLLVMARMAsmParser.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libLLVMARMDisassembler.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libLLVMARMDesc.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libLLVMARMInfo.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libLLVMARMUtils.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libLLVMAArch64CodeGen.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libLLVMAsmPrinter.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libLLVMCFGuard.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libLLVMGlobalISel.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libLLVMSelectionDAG.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libLLVMAArch64AsmParser.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libLLVMAArch64Disassembler.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libLLVMAArch64Desc.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libLLVMMCDisassembler.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libLLVMAArch64Info.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libLLVMAArch64Utils.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libclangCodeGen.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libLLVMCoverage.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libLLVMLTO.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libLLVMExtensions.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libLLVMPasses.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libLLVMCoroutines.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libLLVMipo.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libLLVMInstrumentation.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libLLVMLinker.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libLLVMIRPrinter.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libLLVMCodeGen.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libLLVMBitWriter.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libLLVMObjCARCOpts.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libLLVMTarget.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libLLVMCodeGenTypes.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libLLVMVectorize.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libclangTooling.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libclangToolingRefactor.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libclangToolingRefactoring.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libclangIndex.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libclangFormat.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libclangToolingInclusions.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libclangToolingSyntax.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libclangToolingCore.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libclangRewriteFrontend.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libclangFrontend.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libclangDriver.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libLLVMWindowsDriver.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libLLVMOption.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libclangParse.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libLLVMRemoteCachingService.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libLLVMRemoteNullService.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libclangCAS.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libLLVMMCCAS.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libclangSerialization.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libclangSema.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libclangAPINotes.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libclangAnalysis.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libclangASTMatchers.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libLLVMFrontendHLSL.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libclangSupport.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libclangEdit.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libclangRewrite.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libclangAST.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libLLVMFrontendOpenMP.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libLLVMScalarOpts.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libLLVMAggressiveInstCombine.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libLLVMInstCombine.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libLLVMTransformUtils.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libLLVMAnalysis.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libLLVMProfileData.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libLLVMSymbolize.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libLLVMDebugInfoDWARF.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libLLVMDebugInfoPDB.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libLLVMDebugInfoMSF.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libLLVMDebugInfoBTF.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libLLVMObject.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libLLVMIRReader.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libLLVMBitReader.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libLLVMAsmParser.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libLLVMMCParser.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libLLVMMC.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libLLVMCASUtil.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libLLVMCAS.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libLLVMDebugInfoCodeView.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libLLVMTextAPI.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libLLVMCore.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libLLVMRemarks.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libLLVMBitstreamReader.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libLLVMBinaryFormat.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libclangLex.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libclangBasic.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libLLVMTargetParser.a  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libLLVMSupport.a  -ldl  -lm  /data/data/com.termux/files/home/build/Ninja-Release/llvm-android-aarch64/lib/libLLVMDemangle.a  -lrt  -lpthread  /data/data/com.termux/files/usr/lib/libz.so  /data/data/com.termux/files/usr/lib/libzstd.so  /data/data/com.termux/files/usr/lib/libtinfo.so  lib/swift/host/compiler/lib_CompilerSwiftIfConfig.so  lib/swift/host/compiler/lib_CompilerSwiftCompilerPluginMessageHandling.so  lib/swift/host/compiler/lib_CompilerSwiftSyntaxMacroExpansion.so  lib/swift/host/compiler/lib_CompilerSwiftOperators.so  lib/swift/host/compiler/lib_CompilerSwiftSyntaxMacros.so  lib/swift/host/compiler/lib_CompilerSwiftSyntaxBuilder.so  lib/swift/host/compiler/lib_CompilerSwiftParserDiagnostics.so  lib/swift/host/compiler/lib_CompilerSwiftBasicFormat.so  lib/swift/host/compiler/lib_CompilerSwiftIDEUtils.so  lib/swift/host/compiler/lib_CompilerSwiftParser.so  lib/swift/host/compiler/lib_CompilerSwiftDiagnostics.so  lib/swift/host/compiler/lib_CompilerSwiftSyntax.so  lib/lib_CompilerRegexParser.a && cd /data/data/com.termux/files/home/build/Ninja-Release/swift-android-aarch64/bin && /data/data/com.termux/files/usr/bin/cmake -E create_symlink swift-driver swift && cd /data/data/com.termux/files/home/build/Ninja-Release/swift-android-aarch64/bin && /data/data/com.termux/files/usr/bin/cmake -E create_symlink swift-driver swiftc && cd /data/data/com.termux/files/home/build/Ninja-Release/swift-android-aarch64/bin && /data/data/com.termux/files/usr/bin/cmake -E create_symlink swift-frontend swift-legacy-driver && cd /data/data/com.termux/files/home/build/Ninja-Release/swift-android-aarch64/bin && /data/data/com.termux/files/usr/bin/cmake -E create_symlink swift-frontend swiftc-legacy-driver && cd /data/data/com.termux/files/home/build/Ninja-Release/swift-android-aarch64/bin && /data/data/com.termux/files/usr/bin/cmake -E create_symlink swift-frontend sil-opt && cd /data/data/com.termux/files/home/build/Ninja-Release/swift-android-aarch64/bin && /data/data/com.termux/files/usr/bin/cmake -E create_symlink swift-frontend sil-func-extractor && cd /data/data/com.termux/files/home/build/Ninja-Release/swift-android-aarch64/bin && /data/data/com.termux/files/usr/bin/cmake -E create_symlink swift-frontend sil-nm && cd /data/data/com.termux/files/home/build/Ninja-Release/swift-android-aarch64/bin && /data/data/com.termux/files/usr/bin/cmake -E create_symlink swift-frontend sil-llvm-gen && cd /data/data/com.termux/files/home/build/Ninja-Release/swift-android-aarch64/bin && /data/data/com.termux/files/usr/bin/cmake -E create_symlink swift-frontend sil-passpipeline-dumper && cd /data/data/com.termux/files/home/build/Ninja-Release/swift-android-aarch64/bin && /data/data/com.termux/files/usr/bin/cmake -E create_symlink swift-frontend swift-dependency-tool && cd /data/data/com.termux/files/home/build/Ninja-Release/swift-android-aarch64/bin && /data/data/com.termux/files/usr/bin/cmake -E create_symlink swift-frontend swift-llvm-opt && cd /data/data/com.termux/files/home/build/Ninja-Release/swift-android-aarch64/bin && /data/data/com.termux/files/usr/bin/cmake -E create_symlink swift-frontend swift-indent && cd /data/data/com.termux/files/home/build/Ninja-Release/swift-android-aarch64/bin && /data/data/com.termux/files/usr/bin/cmake -E create_symlink swift-frontend swift-symbolgraph-extract && cd /data/data/com.termux/files/home/build/Ninja-Release/swift-android-aarch64/bin && /data/data/com.termux/files/usr/bin/cmake -E create_symlink swift-frontend swift-autolink-extract && cd /data/data/com.termux/files/home/build/Ninja-Release/swift-android-aarch64/bin && /data/data/com.termux/files/usr/bin/cmake -E create_symlink swift-frontend swift-api-digester && cd /data/data/com.termux/files/home/build/Ninja-Release/swift-android-aarch64/bin && /data/data/com.termux/files/usr/bin/cmake -E create_symlink swift-frontend swift-cache-tool && cd /data/data/com.termux/files/home/build/Ninja-Release/swift-android-aarch64/bin && /data/data/com.termux/files/usr/bin/cmake -E create_symlink swift-frontend swift-parse-test                     
ld.lld: error: undefined hidden symbol: void* std::__ndk1::__voidify[abi:ne180000]<swift::SILLocation>(swift::SILLocation&)                                        
>>> referenced by Optimizer.o
>>>               Optimizer.o:(swift::SILBuilder::createAllocStack(swift::SILLocation, swift::SILType, std::__ndk1::optional<swift::SILDebugVariable>, swift::HasDynamicLifetime_t, swift::IsLexical_t, swift::IsFromVarDecl_t, swift::UsesMoveableValueDebugInfo_t, bool)) in archive lib/libswiftCompilerModules.a                  
>>> referenced by SIL.o
>>>               SIL.o:(swift::DebugValueInst::getVarInfo(bool) const) in archive lib/libswiftCompilerModules.a                                                   
>>> referenced by SIL.o
>>>               SIL.o:(swift::DebugValueInst::getVarInfo(bool) const) in archive lib/libswiftCompilerModules.a
>>> referenced 2 more times

ld.lld: error: undefined hidden symbol: void* std::__ndk1::__voidify[abi:ne180000]<swift::SILDebugVariable>(swift::SILDebugVariable&)
>>> referenced by Optimizer.o                                                                                                                                      
>>>               Optimizer.o:(void std::__ndk1::__optional_storage_base<swift::SILDebugVariable, false>::__construct_from[abi:ne180000]<std::__ndk1::__optional_copy_base<swift::SILDebugVariable, false> const&>(std::__ndk1::__optional_copy_base<swift::SILDebugVariable, false> const&)) in archive lib/libswiftCompilerModules.a
>>> referenced by SIL.o
>>>               SIL.o:(void std::__ndk1::__optional_storage_base<swift::SILDebugVariable, false>::__assign_from[abi:ne180000]<std::__ndk1::__optional_copy_assign_base<swift::SILDebugVariable, false> const&>(std::__ndk1::__optional_copy_assign_base<swift::SILDebugVariable, false> const&)) in archive lib/libswiftCompilerModules.a

ld.lld: error: undefined hidden symbol: void* std::__ndk1::__voidify[abi:ne180000]<swift::FixedOperandList<1u>>(swift::FixedOperandList<1u>&)
>>> referenced by SIL.o
>>>               SIL.o:(swift::FixedOperandList<1u>* std::__ndk1::__construct_at[abi:ne180000]<swift::FixedOperandList<1u>, swift::EnumInst*, swift::SILValue&, swift::FixedOperandList<1u>*>(swift::FixedOperandList<1u>*, swift::EnumInst*&&, swift::SILValue&)) in archive lib/libswiftCompilerModules.a

ld.lld: error: undefined hidden symbol: void* std::__ndk1::__voidify[abi:ne180000]<swift::SILType>(swift::SILType&)
>>> referenced by SIL.o
>>>               SIL.o:(swift::DebugValueInst::getVarInfo(bool) const) in archive lib/libswiftCompilerModules.a
>>> referenced by SIL.o
>>>               SIL.o:(swift::AllocStackInst::getVarInfo(bool) const) in archive lib/libswiftCompilerModules.a                                                   
>>> referenced by SIL.o                                                                                                                                            
>>>               SIL.o:(swift::AllocStackInst::getVarInfo(bool) const) in archive lib/libswiftCompilerModules.a                                                   clang++: error: linker command failed with exit code 1 (use -v to see invocation)

Modifying the libc++ module map like this got it to build again:

--- a/module.modulemap    2024-09-07 08:30:49.099247434 +0530
+++ b/module.modulemap     2024-09-06 10:11:19.151172662 +0530
@@ -1537,6 +1537,7 @@
 module std_private_memory_uninitialized_algorithms        [system] {
   header "__memory/uninitialized_algorithms.h"
   export std_private_algorithm_copy
+  export std_private_memory_voidify
 }
 module std_private_memory_unique_ptr                      [system] {
                header "__memory/unique_ptr.h"

This worked fine when building the Aug. 6 trunk source snapshot with the NDK 26b sysroot last month, so I looked into it and that NDK did not ship with @ian-twilightcoder's reorganization of the libc++ module map from last summer, llvm/llvm-project@571178a21. NDK 27 ships with a lightly-patched version of that upstream libc++ module map, but I don't see anything when comparing their changes to the version in swiftlang/llvm-project that would cause this error.

I also see 11 newly failing C++ Interop tests from the Swift compiler validation suite, which all worked fine last month with NDK 26b:

<testcase classname="Swift(android-aarch64).Interop/Cxx/stdlib" name="use-swift-concurrency.swift" time="3.45">
  <failure><![CDATA[Script:
--
: 'RUN: at line 1';   rm -rf "/data/data/com.termux/files/home/build/Ninja-Release/swift-android-aarch64/test-android-aarch64/Interop/Cxx/stdlib/Output/use-swift-concurrency.swift.tmp" && mkdir -p "/data/data/com.termux/files/home/build/Ninja-Release/swift-android-aarch64/test-android-aarch64/Interop/Cxx/stdlib/Output/use-swift-concurrency.swift.tmp" && /data/data/com.termux/files/home/build/Ninja-Release/swift-android-aarch64/bin/swiftc -target aarch64-unknown-linux-android -toolchain-stdlib-rpath -Xclang-linker -fno-termux-rpath -Xlinker -rpath=/data/data/com.termux/files/usr/lib  -module-cache-path /data/data/com.termux/files/home/build/Ninja-Release/swift-android-aarch64/swift-test-results/aarch64-unknown-linux-android/clang-module-cache -swift-version 4  -Xfrontend -define-availability -Xfrontend 'SwiftStdlib 9999:macOS 9999, iOS 9999, watchOS 9999, tvOS 9999' -Xfrontend -define-availability -Xfrontend 'SwiftStdlib 5.0:macOS 10.14.4, iOS 12.2, watchOS 5.2, tvOS 12.2' -Xfrontend -define-availability -Xfrontend 'SwiftStdlib 5.1:macOS 10.15, iOS 13.0, watchOS 6.0, tvOS 13.0' -Xfrontend -define-availability -Xfrontend 'SwiftStdlib 5.2:macOS 10.15.4, iOS 13.4, watchOS 6.2, tvOS 13.4' -Xfrontend -define-availability -Xfrontend 'SwiftStdlib 5.3:macOS 11.0, iOS 14.0, watchOS 7.0, tvOS 14.0' -Xfrontend -define-availability -Xfrontend 'SwiftStdlib 5.4:macOS 11.3, iOS 14.5, watchOS 7.4, tvOS 14.5' -Xfrontend -define-availability -Xfrontend 'SwiftStdlib 5.5:macOS 12.0, iOS 15.0, watchOS 8.0, tvOS 15.0' -Xfrontend -define-availability -Xfrontend 'SwiftStdlib 5.6:macOS 12.3, iOS 15.4, watchOS 8.5, tvOS 15.4' -Xfrontend -define-availability -Xfrontend 'SwiftStdlib 5.7:macOS 13.0, iOS 16.0, watchOS 9.0, tvOS 16.0' -Xfrontend -define-availability -Xfrontend 'SwiftStdlib 5.8:macOS 13.3, iOS 16.4, watchOS 9.4, tvOS 16.4' -Xfrontend -define-availability -Xfrontend 'SwiftStdlib 5.9:macOS 14.0, iOS 17.0, watchOS 10.0, tvOS 17.0' -Xfrontend -define-availability -Xfrontend 'SwiftStdlib 5.10:macOS 14.4, iOS 17.4, watchOS 10.4, tvOS 17.4, visionOS 1.1' -Xfrontend -define-availability -Xfrontend 'SwiftStdlib 6.0:macOS 15.0, iOS 18.0, watchOS 11.0, tvOS 18.0, visionOS 2.0' -Xfrontend -define-availability -Xfrontend 'SwiftStdlib 6.1:macOS 9999, iOS 9999, watchOS 9999, tvOS 9999, visionOS 9999'  -module-cache-path /data/data/com.termux/files/home/build/Ninja-Release/swift-android-aarch64/swift-test-results/aarch64-unknown-linux-android/clang-module-cache /data/data/com.termux/files/home/swift/test/Interop/Cxx/stdlib/use-swift-concurrency.swift -I /data/data/com.termux/files/home/swift/test/Interop/Cxx/stdlib/Inputs -Xfrontend -enable-experimental-cxx-interop -Xfrontend -disable-availability-checking -I /data/data/com.termux/files/home/swift-corelibs-libdispatch -I /data/data/com.termux/files/home/build/Ninja-Release/libdispatch-android-aarch64/src/swift/swift -L /data/data/com.termux/files/home/build/Ninja-Release/libdispatch-android-aarch64 -vfsoverlay /data/data/com.termux/files/home/build/Ninja-Release/libdispatch-android-aarch64/dispatch-vfs-overlay.yaml -parse-as-library -o /data/data/com.termux/files/home/build/Ninja-Release/swift-android-aarch64/test-android-aarch64/Interop/Cxx/stdlib/Output/use-swift-concurrency.swift.tmp/a.out -module-name main  && echo /data/data/com.termux/files/home/build/Ninja-Release/swift-android-aarch64/test-android-aarch64/Interop/Cxx/stdlib/Output/use-swift-concurrency.swift.tmp/a.out && /bin/env LD_LIBRARY_PATH='/data/data/com.termux/files/home/build/Ninja-Release/swift-android-aarch64/lib/swift/android:/data/data/com.termux/files/home/build/Ninja-Release/libdispatch-android-aarch64'  /data/data/com.termux/files/home/build/Ninja-Release/swift-android-aarch64/test-android-aarch64/Interop/Cxx/stdlib/Output/use-swift-concurrency.swift.tmp/a.out
--
Exit Code: 1

Command Output (stderr):
--
<unknown>:0: warning: the '-enable-experimental-cxx-interop' flag is deprecated; please pass '-cxx-interoperability-mode=' instead
<unknown>:0: note: Swift will maintain source compatibility for imported APIs based on the selected compatibility mode, so updating the Swift compiler will not change how APIs are imported
/data/data/com.termux/files/home/swift/test/Interop/Cxx/stdlib/use-swift-concurrency.swift:18:9: warning: variable 'ConcurrencyTestSuite' was never mutated; consider changing to 'let' constant
16 | @main struct Main {
17 |   static func main() async {
18 |     var ConcurrencyTestSuite = TestSuite("Concurrency")
   |         `- warning: variable 'ConcurrencyTestSuite' was never mutated; consider changing to 'let' constant
19 | 
20 |     ConcurrencyTestSuite.test("Task.sleep") {
ld.lld: error: undefined symbol: std::__ndk1::basic_string<char, std::__ndk1::char_traits<char>, std::__ndk1::allocator<char>>::~basic_string()
>>> referenced by use-swift-concurrency-df1f81.o
>>>               /data/data/com.termux/files/usr/tmp/lit-tmp-2ey2ry3w/use-swift-concurrency-df1f81.o:($s4main4MainVAAyyYaFZyyYacfU0_TY3_)
>>> referenced by use-swift-concurrency-df1f81.o
>>>               /data/data/com.termux/files/usr/tmp/lit-tmp-2ey2ry3w/use-swift-concurrency-df1f81.o:($s4main4MainVAAyyYaFZyyYacfU0_TY3_)
>>> referenced by use-swift-concurrency-df1f81.o
>>>               /data/data/com.termux/files/usr/tmp/lit-tmp-2ey2ry3w/use-swift-concurrency-df1f81.o:($sSo3stdO6__ndk1O0071basic_stringCCharchar_traitsCCharallocatorCChar_mHGHsqaGJcraCCfsaqChraaVwxx)
>>> referenced 4 more times

ld.lld: error: undefined symbol: std::__ndk1::basic_string<char, std::__ndk1::char_traits<char>, std::__ndk1::allocator<char>>::basic_string(std::__ndk1::basic_string<char, std::__ndk1::char_traits<char>, std::__ndk1::allocator<char>> const&)
>>> referenced by use-swift-concurrency-df1f81.o
>>>               /data/data/com.termux/files/usr/tmp/lit-tmp-2ey2ry3w/use-swift-concurrency-df1f81.o:($sSo3stdO6__ndk1O0071basic_stringCCharchar_traitsCCharallocatorCChar_mHGHsqaGJcraCCfsaqChraaVwcp)
>>> referenced by use-swift-concurrency-df1f81.o
>>>               /data/data/com.termux/files/usr/tmp/lit-tmp-2ey2ry3w/use-swift-concurrency-df1f81.o:($sSo3stdO6__ndk1O0071basic_stringCCharchar_traitsCCharallocatorCChar_mHGHsqaGJcraCCfsaqChraaVwca)

ld.lld: error: undefined symbol: std::__ndk1::basic_string<char, std::__ndk1::char_traits<char>, std::__ndk1::allocator<char>>::~basic_string()
>>> referenced by CxxStdlib.o
>>>               CxxStdlib.o:(.text+0xD60) in archive /data/data/com.termux/files/home/build/Ninja-Release/swift-android-aarch64/lib/swift/android/libswiftCxxStdlib.a
>>> referenced by CxxStdlib.o
>>>               CxxStdlib.o:($sSo3stdO6__ndk1O0071basic_stringCCharchar_traitsCCharallocatorCChar_mHGHsqaGJcraCCfsaqChraaV9CxxStdlibE16debugDescriptionSSvg) in archive /data/data/com.termux/files/home/build/Ninja-Release/swift-android-aarch64/lib/swift/android/libswiftCxxStdlib.a
>>> referenced by CxxStdlib.o
>>>               CxxStdlib.o:($sSo3stdO6__ndk1O0071basic_stringCCharchar_traitsCCharallocatorCChar_mHGHsqaGJcraCCfsaqChraaV9CxxStdlibE11descriptionSSvg) in archive /data/data/com.termux/files/home/build/Ninja-Release/swift-android-aarch64/lib/swift/android/libswiftCxxStdlib.a

ld.lld: error: undefined symbol: std::__ndk1::basic_string<char, std::__ndk1::char_traits<char>, std::__ndk1::allocator<char>>::append(char const*, unsigned long)
>>> referenced by CxxStdlib.o
>>>               CxxStdlib.o:($sSo3stdO6__ndk1O0071basic_stringCCharchar_traitsCCharallocatorCChar_mHGHsqaGJcraCCfsaqChraaV9CxxStdlibE2peoiyyAFz_AFtFZ) in archive /data/data/com.termux/files/home/build/Ninja-Release/swift-android-aarch64/lib/swift/android/libswiftCxxStdlib.a
>>> referenced by CxxStdlib.o
>>>               CxxStdlib.o:($sSo3stdO6__ndk1O0071basic_stringCCharchar_traitsCCharallocatorCChar_mHGHsqaGJcraCCfsaqChraaV9CxxStdlibE6appendyyAFF) in archive /data/data/com.termux/files/home/build/Ninja-Release/swift-android-aarch64/lib/swift/android/libswiftCxxStdlib.a
>>> referenced by CxxStdlib.o
>>>               CxxStdlib.o:($sSo3stdO6__ndk1O0071basic_stringCCharchar_traitsCCharallocatorCChar_mHGHsqaGJcraCCfsaqChraaV9CxxStdlibE1poiyA2F_AFtFZ) in archive /data/data/com.termux/files/home/build/Ninja-Release/swift-android-aarch64/lib/swift/android/libswiftCxxStdlib.a
clang: error: linker command failed with exit code 1 (use -v to see invocation)
<unknown>:0: error: link command failed with exit code 1 (use -v to see invocation)

--
]]></failure>
</testcase>

Since the upstream libc++ reorg shipped with the LLVM 17 release last fall and there don't appear to be any fixes merged for this issue since, I assume this doesn't affect clang/C++, only Swift->C++ Interop, so I filed the issue here.

Reproduction

See commands pasted above

Expected behavior

No linker errors

Environment

Android 13 AArch64 in the Termux app

Additional information

I'll see what other module map changes get those 11 failing C++ Interop tests to build again. In the meantime, aren't there other environments using Swift with libc++ also that fail in this way?

@egorzhdan or @ldionne, in addition to Ian, let me know what you think.

ldionne commented 2 months ago

We are in the process of significantly rewriting our modulemap in libc++: https://github.com/llvm/llvm-project/pull/107638

This will yield significant compilation time speedups when using modules. IMO what we should do here (as far as trunk libc++ is concerned) is to keep moving towards that change and make sure that it doesn't break interop (in fact ideally it would fix the issues you're seeing). If we need to backport a partial fix to an already-released version of libc++, that's another story and I would tackle that separately.

Do you see problems when using libc++ as it shipped with LLVM 19? Perhaps the NDK doesn't ship such a recent version yet?

finagolfin commented 2 months ago

Do you see problems when using libc++ as it shipped with LLVM 19?

I have not tried it, but I see no differences in LLVM 19 using the voidify.h header, so I think it would still have this problem.

Perhaps the NDK doesn't ship such a recent version yet?

It doesn't.

I saw your rewrite pull above soon after I opened this issue: I hope it fixes this.

However, you all don't see this issue now when building the Swift toolchain with libc++ on other platforms? I don't see why this would only affect libc++ on Android.

ldionne commented 2 months ago

However, you all don't see this issue when building the Swift toolchain with libc++ on other platforms? I don't see why this would only affect libc++ on Android.

I will let Swift / interop folks answer this since I do not build the Swift toolchain myself.

finagolfin commented 2 months ago

I just dumped some libc++ module maps from recent Xcode versions on the github CI, and it appears that this libc++ reorg commit was first shipped with Xcode 15.3 six months ago. Checking the macOS CI for which Xcode it uses to build this Optimizer.o file, it shows these flags used, /Applications/Xcode-beta.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/swiftc -c -o /Users/ec2-user/jenkins/workspace/swift-PR-macos/branch-main/build/buildbot_incremental/swift-macosx-x86_64/SwiftCompilerSources/Optimizer.o -sdk /Applications/Xcode-beta.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX14.2.sdk, which implies Xcode 15.1 or 15.2, neither of which use Ian's reorganized libc++ module map.

This suggests this has not hit macOS yet only because the CI is using an older libc++ module map, but once the CI updates to Xcode 15.3 or later, building the Swift compiler and C++ interop is going to break on macOS too.

@shahmishal, just letting you know, perhaps you've already seen this problem.

michael-yuji commented 4 days ago

In the meantime, aren't there other environments using Swift with libc++ also that fail in this way?

Can confirm also hitting this exact issue on FreeBSD 14.1, which ships clang/libc++ 18.1.5

ld.lld: error: undefined hidden symbol: void* std::__1::__voidify[abi:se180100]<swift::SILLocation>(swift::SILLocation&)
>>> referenced by SIL.o
>>>               SIL.o:(swift::SILBuilder::createAllocStack(swift::SILLocation, swift::SILType, std::__1::optional<swift::SILDebugVariable>, swift::HasDynamicLifetime_t, swift::IsLexical_t, swift::IsFromVarDecl_t, swift::UsesMoveableValueDebugInfo_t, bool)) in archive lib/libswiftCompilerModules.a
>>> referenced by SIL.o
>>>               SIL.o:(swift::DebugValueInst::getVarInfo(bool) const) in archive lib/libswiftCompilerModules.a
>>> referenced by SIL.o
>>>               SIL.o:(swift::DebugValueInst::getVarInfo(bool) const) in archive lib/libswiftCompilerModules.a
>>> referenced 2 more times

ld.lld: error: undefined hidden symbol: void* std::__1::__voidify[abi:se180100]<swift::SILDebugVariable>(swift::SILDebugVariable&)
>>> referenced by SIL.o
>>>               SIL.o:(void std::__1::__optional_storage_base<swift::SILDebugVariable, false>::__construct_from[abi:se180100]<std::__1::__optional_copy_base<swift::SILDebugVariable, false> const&>(std::__1::__optional_copy_base<swift::SILDebugVariable, false> const&)) in archive lib/libswiftCompilerModules.a
>>> referenced by SIL.o
>>>               SIL.o:(void std::__1::__optional_storage_base<swift::SILDebugVariable, false>::__assign_from[abi:se180100]<std::__1::__optional_copy_assign_base<swift::SILDebugVariable, false> const&>(std::__1::__optional_copy_assign_base<swift::SILDebugVariable, false> const&)) in archive lib/libswiftCompilerModules.a

ld.lld: error: undefined hidden symbol: void* std::__1::__voidify[abi:se180100]<swift::FixedOperandList<1u>>(swift::FixedOperandList<1u>&)
>>> referenced by SIL.o
>>>               SIL.o:(swift::FixedOperandList<1u>* std::__1::__construct_at[abi:se180100]<swift::FixedOperandList<1u>, swift::EnumInst*, swift::SILValue&, swift::FixedOperandList<1u>*>(swift::FixedOperandList<1u>*, swift::EnumInst*&&, swift::SILValue&)) in archive lib/libswiftCompilerModules.a

ld.lld: error: undefined hidden symbol: void* std::__1::__voidify[abi:se180100]<swift::SILType>(swift::SILType&)
>>> referenced by SIL.o
>>>               SIL.o:(swift::DebugValueInst::getVarInfo(bool) const) in archive lib/libswiftCompilerModules.a
>>> referenced by SIL.o
>>>               SIL.o:(swift::AllocStackInst::getVarInfo(bool) const) in archive lib/libswiftCompilerModules.a
>>> referenced by SIL.o
>>>               SIL.o:(swift::AllocStackInst::getVarInfo(bool) const) in archive lib/libswiftCompilerModules.a

I'm also able to work around this issue by applying the same patch to libc++ modulemap (at /usr/include/c++/module.modulemap)