search

ReactiveX / RxSwift

Reactive Programming in Swift
MIT License
24.41k stars 4.17k forks source link

Linking multiple SPM dynamic libraries towards the same target ends up in failure. #2424

Closed nikita-leonov closed 2 years ago

nikita-leonov commented 2 years ago

Short description of the issue: When the package depends on multiple dynamic targets from RxSwift package it fails to compile with ld error.

Expected outcome: I would expect the package to compile.

What actually happens: Building of swift-dynamic-test provided in the example below will fail with Command Ld failed with a nonzero exit code, and the actual reason underneath will be that ld uses a path /Build/Products/Debug-iphoneos/PackageFrameworks/RxSwift-Dynamic.framework/RxSwift-Dynamic while there is no RxSwift-Dynamic in RxSwift-Dynamic.framework. Meanwhile, there is RxSwift in the folder RxSwift.framework folder next to it. Moreover manually moving and renaming the file to the place where ld tries to find it results in a successful build. Note: There is no use of RxSwift product library in the package file and it is unclear how the whole RxSwift.framework is being built as well as why RxSwift-Dynamic.framework exists on a disk but RxSwift-Dynamic is not there, while other things are.

Errors details:

Ld /Users/nleonov/Library/Developer/Xcode/DerivedData/swift-dynamic-test-hjhlavumahezasfjpkyaquybqqke/Build/Products/Debug-iphoneos/swift-dynamic-testTests.xctest/swift-dynamic-testTests normal (in target 'swift-dynamic-testTests' from project 'swift-dynamic-test')
    cd /Users/nleonov/projects/Temp/swift-dynamic-test
    /Applications/Xcode13.3.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang -target arm64-apple-ios9.0 -bundle -isysroot /Applications/Xcode13.3.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS15.4.sdk -L/Users/nleonov/Library/Developer/Xcode/DerivedData/swift-dynamic-test-hjhlavumahezasfjpkyaquybqqke/Build/Products/Debug-iphoneos -L/Applications/Xcode13.3.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/usr/lib -F/Users/nleonov/Library/Developer/Xcode/DerivedData/swift-dynamic-test-hjhlavumahezasfjpkyaquybqqke/Build/Products/Debug-iphoneos/PackageFrameworks -F/Users/nleonov/Library/Developer/Xcode/DerivedData/swift-dynamic-test-hjhlavumahezasfjpkyaquybqqke/Build/Products/Debug-iphoneos/PackageFrameworks -F/Users/nleonov/Library/Developer/Xcode/DerivedData/swift-dynamic-test-hjhlavumahezasfjpkyaquybqqke/Build/Products/Debug-iphoneos/PackageFrameworks -F/Users/nleonov/Library/Developer/Xcode/DerivedData/swift-dynamic-test-hjhlavumahezasfjpkyaquybqqke/Build/Products/Debug-iphoneos/PackageFrameworks -F/Users/nleonov/Library/Developer/Xcode/DerivedData/swift-dynamic-test-hjhlavumahezasfjpkyaquybqqke/Build/Products/Debug-iphoneos/PackageFrameworks -F/Users/nleonov/Library/Developer/Xcode/DerivedData/swift-dynamic-test-hjhlavumahezasfjpkyaquybqqke/Build/Products/Debug-iphoneos -F/Applications/Xcode13.3.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/Library/Frameworks -iframework /Applications/Xcode13.3.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/Library/Frameworks -iframework /Applications/Xcode13.3.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS15.4.sdk/Developer/Library/Frameworks -filelist /Users/nleonov/Library/Developer/Xcode/DerivedData/swift-dynamic-test-hjhlavumahezasfjpkyaquybqqke/Build/Intermediates.noindex/swift-dynamic-test.build/Debug-iphoneos/swift-dynamic-testTests.build/Objects-normal/arm64/swift-dynamic-testTests.LinkFileList -Xlinker -rpath -Xlinker /usr/lib/swift -Xlinker -rpath -Xlinker @loader_path/Frameworks -Xlinker -rpath -Xlinker @loader_path/../Frameworks -Xlinker -rpath -Xlinker @loader_path/Frameworks -dead_strip -Xlinker -object_path_lto -Xlinker /Users/nleonov/Library/Developer/Xcode/DerivedData/swift-dynamic-test-hjhlavumahezasfjpkyaquybqqke/Build/Intermediates.noindex/swift-dynamic-test.build/Debug-iphoneos/swift-dynamic-testTests.build/Objects-normal/arm64/swift-dynamic-testTests_lto.o -Xlinker -export_dynamic -Xlinker -no_deduplicate -fobjc-link-runtime -L/Applications/Xcode13.3.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/swift/iphoneos -L/usr/lib/swift -Xlinker -add_ast_path -Xlinker /Users/nleonov/Library/Developer/Xcode/DerivedData/swift-dynamic-test-hjhlavumahezasfjpkyaquybqqke/Build/Intermediates.noindex/swift-dynamic-test.build/Debug-iphoneos/swift-dynamic-testTests.build/Objects-normal/arm64/swift_dynamic_testTests.swiftmodule -framework XCTest /Users/nleonov/Library/Developer/Xcode/DerivedData/swift-dynamic-test-hjhlavumahezasfjpkyaquybqqke/Build/Products/Debug-iphoneos/PackageFrameworks/RxSwift-Dynamic.framework/RxSwift-Dynamic /Users/nleonov/Library/Developer/Xcode/DerivedData/swift-dynamic-test-hjhlavumahezasfjpkyaquybqqke/Build/Products/Debug-iphoneos/PackageFrameworks/RxCocoa-Dynamic.framework/RxCocoa-Dynamic -Xlinker -dependency_info -Xlinker /Users/nleonov/Library/Developer/Xcode/DerivedData/swift-dynamic-test-hjhlavumahezasfjpkyaquybqqke/Build/Intermediates.noindex/swift-dynamic-test.build/Debug-iphoneos/swift-dynamic-testTests.build/Objects-normal/arm64/swift-dynamic-testTests_dependency_info.dat -o /Users/nleonov/Library/Developer/Xcode/DerivedData/swift-dynamic-test-hjhlavumahezasfjpkyaquybqqke/Build/Products/Debug-iphoneos/swift-dynamic-testTests.xctest/swift-dynamic-testTests -Xlinker -add_ast_path -Xlinker /Users/nleonov/Library/Developer/Xcode/DerivedData/swift-dynamic-test-hjhlavumahezasfjpkyaquybqqke/Build/Intermediates.noindex/swift-dynamic-test.build/Debug-iphoneos/swift-dynamic-test.build/Objects-normal/arm64/swift_dynamic_test.swiftmodule

clang: error: no such file or directory: '/Users/nleonov/Library/Developer/Xcode/DerivedData/swift-dynamic-test-hjhlavumahezasfjpkyaquybqqke/Build/Products/Debug-iphoneos/PackageFrameworks/RxSwift-Dynamic.framework/RxSwift-Dynamic'
Command Ld failed with a nonzero exit code

Self-contained code example that reproduces the issue:

A library within a package depends on RxSwift-Dynamic and RxCocoa-Dynamic which are defined as dynamic in the RxSwift package.

let package = Package(
    name: "swift-dynamic-test",
    products: [
        .library(
            name: "swift-dynamic-test",
            targets: ["swift-dynamic-test"]),
    ],
    dependencies: [
        .package(url: "https://github.com/ReactiveX/RxSwift.git", from: "6.5.0"),
    ],
    targets: [
        .target(
            name: "swift-dynamic-test",
            dependencies: [
                .product(name: "RxSwift-Dynamic", package: "RxSwift"),
                .product(name: "RxCocoa-Dynamic", package: "RxSwift")
            ]
        )
    ]
)

Platform/Environment

How easy is it to reproduce? (chances of successful reproduce after running the self-contained code)

Xcode version:

13.3.1

Installation method:

I have multiple versions of Xcode installed: (so we can know if this is a potential cause of your issue)

Level of RxSwift knowledge: (this is so we can understand your level of knowledge and formulate the response in an appropriate manner)

nikita-leonov commented 2 years ago

The issue reported on Swift Forums too, but without much of activities so far — https://forums.swift.org/t/linking-multiple-dynamic-libraries-that-depend-on-the-same-target-fails-with-ld-failing-due-to-missing-files/57382/2

freak4pc commented 2 years ago

Thanks for reporting @nikita-leonov :) That's pretty frustrating, but I think it's an SPM linking issue. I'm not entirely sure what we can do in the project to resolve it, but if you have any thoughts I'm happy to further discuss.

nikita-leonov commented 2 years ago

@freak4pc indeed seems like an SPM issue. Although I think there is an opportunity to improve the current products layout that may indirectly fix the issue. It seems like all dynamic products within the package depend on the same RxSwift target. This will likely end up in symbol duplication if you link multiple dynamic libraries such as RxCocoa-Dynamic and RxRelay-Dynamic into the single end product. To fix issues like this common dependency products should be not within the same package, but in separate packages. This means RxSwift likely needs to be defined as a separate package and in this way, it can be reused as dynamic across multiple Dynamic products within RxSwift infrastructure itself. This is definitely a suboptimal way of doing things, but the current state of SPM is likely the only experiment to try that may unblock this issue. This fix is simple to validate, but I do not want to invest in it much unless we both agree it worth trying and adopting in case if fixes the problem.

nikita-leonov commented 2 years ago

I validated the setup where there is a separate package RxSwift with RxSwift-Dynamic target and where the top layer package where RxCocoa and others refer to the RxSwift. It does solve the problem above, as well as a general issue with RxSwift symbols linked into multiple dynamic libraries.

freak4pc commented 2 years ago

I validated the setup where there is a separate package RxSwift with RxSwift-Dynamic target and where the top layer package where RxCocoa and others refer to the RxSwift. It does solve the problem above, as well as a general issue with RxSwift symbols linked into multiple dynamic libraries.

Thanks for looking into it. Can you share how that looks ?

freak4pc commented 2 years ago

@nikita-leonov ping

nikita-leonov commented 2 years ago

@freak4pc Hmm, so I have a mixed set of news. I did update the post here https://forums.swift.org/t/linking-multiple-dynamic-libraries-that-depend-on-the-same-target-fails-with-ld-failing-due-to-missing-files/57382/2 it seems the issue I experienced was not SPM specific, but probably some environment issue. After some change in the environment that I was unable to trace yet the issue is not reproducible anymore. I still believe that libs layout where multiple products could be static and dynamic, meanwhile all of them depend on RxSwift static target underneath may end up in the symbols duplication, but it is probably a discussion for a separate ticket. RxSwift is an important dependency in our product, and we are working through SPM migration now, so be sure if there anything poping up I will follow up here :) Although I do think we can close this particular ticket for now unless I will find what actually happened :O

freak4pc commented 2 years ago

Happy to circle back to this. We also recently migrated everything over to SPM & Tuist and experienced no issues as well, so we're happy to hear anything you pop into.

Closing this for the time being, but feel free to comment or open a new issue if you find anything. Thanks!