search

tuist / tuist

Tuist's CLI
https://tuist.dev
MIT License
4.65k stars 566 forks source link

SPM external dependency SDWebImageSwiftUI fails to build #3762

Closed kikeenrique closed 2 years ago

kikeenrique commented 3 years ago

Describe the bug I created a dummy project depending on SDWebImageSwiftUI using SPM interface, and when I try build it, it fails.

Error logs are like this:

❌ /Users/user/Library/Developer/Xcode/DerivedData/TuistSpmSdwebimageswiftuiFails-fqkgsqzjasoywvdcjzesgeldrsdg/Build/Products/Debug-iphonesimulator/SDWebImage.framework/Headers/SDWebImage.h:10:9: include of non-modular header inside framework module 'SDWebImage': '/Users/user/Developer/tools/tuist-spm-sdwebimageswiftui-fails/Tuist/Dependencies/SwiftPackageManager/.build/checkouts/SDWebImage/SDWebImage/include/SDWebImage/SDWebImageCompat.h'
#import <SDWebImage/SDWebImageCompat.h>
        ^

To Reproduce Steps to reproduce the behavior:

I've created a repo to reproduce the behaviour, also with CI added, but it's just needed to add SDWebImageSwiftUI in Dependencies.swift . https://github.com/kikeenrique/tuist-spm-sdwebimageswiftui-fails

In this job you could see the full compilation logs. https://github.com/kikeenrique/tuist-spm-sdwebimageswiftui-fails/runs/4365023568

Expected behavior SPM package compiles without failure.

Desktop:

Additional context I've also used a dummy project with SDWebImageSwiftUI as SPM dependency and it builds properly. I've traced down xcodebuild to a single compilation file to compare and spot differences on xcodebuild arguments. There are no much differences, but there is one that caught my eyes :

-Xcc

-fmodule-map-file\=/Users/user/Library/Developer/Xcode/DerivedData/myDerivedDatamy/Build/Intermediates.noindex/GeneratedModuleMaps-iphonesimulator/SDWebImage.modulemap

This is the DerivedData for Xcode SPM variant:

 git:(develop) ✗  ls -la /Users/user/Library/Developer/Xcode/DerivedData/Testsdwebimage-dckoyuwquqxbxhcuhfgvjfrcsdvx/Build/Products/Debug-iphonesimulator/
total 5304
drwxr-xr-x@ 8 user  staff      256 Nov 25 21:06 .
drwxr-xr-x@ 3 user  staff       96 Nov 25 21:06 ..
drwxr-xr-x@ 2 user  staff       64 Nov 25 21:06 PackageFrameworks
-rw-r--r--  1 user  staff  1406408 Nov 25 21:06 SDWebImage.o
-rw-r--r--  1 user  staff  1306464 Nov 25 21:06 SDWebImageSwiftUI.o
drwxr-xr-x  7 user  staff      224 Nov 25 21:06 SDWebImageSwiftUI.swiftmodule
drwxr-xr-x  6 user  staff      192 Nov 25 21:06 Testsdwebimage.app
drwxr-xr-x  7 user  staff      224 Nov 25 21:06 Testsdwebimage.swiftmodule

git:(develop) ✗  ls -la /Users/user/Library/Developer/Xcode/DerivedData/Testsdwebimage-dckoyuwquqxbxhcuhfgvjfrcsdvx/Build/Intermediates.noindex/GeneratedModuleMaps-iphonesimulator
total 40
drwxr-xr-x  5 user  staff   160 Nov 25 21:06 .
drwxr-xr-x@ 7 user  staff   224 Nov 25 21:06 ..
-rw-r--r--  1 user  staff   221 Nov 25 21:06 SDWebImage.modulemap
-rw-r--r--  1 user  staff  9403 Nov 25 21:06 SDWebImageSwiftUI-Swift.h
-rw-r--r--  1 user  staff    72 Nov 25 21:06 SDWebImageSwiftUI.modulemap

 git:(develop) ✗  cat /Users/user/Library/Developer/Xcode/DerivedData/Testsdwebimage-dckoyuwquqxbxhcuhfgvjfrcsdvx/Build/Intermediates.noindex/GeneratedModuleMaps-iphonesimulator/SDWebImage.modulemap
module SDWebImage {
umbrella header "/Users/user/Library/Developer/Xcode/DerivedData/Testsdwebimage-dckoyuwquqxbxhcuhfgvjfrcsdvx/SourcePackages/checkouts/SDWebImage/SDWebImage/include/SDWebImage/SDWebImage.h"
export *
}%

 git:(develop) ✗  cat /Users/user/Library/Developer/Xcode/DerivedData/Testsdwebimage-dckoyuwquqxbxhcuhfgvjfrcsdvx/Build/Intermediates.noindex/GeneratedModuleMaps-iphonesimulator/SDWebImageSwiftUI.modulemap
module SDWebImageSwiftUI {
header "SDWebImageSwiftUI-Swift.h"
export *
}%

This is the DerivedData for Tuist:

git:(develop) ✗  ls -la /Users/user/Library/Developer/Xcode/DerivedData/Workspace-egfgfeihvgnlcxdwkslotvzjreap/Build/Products/Debug-iphonesimulator
total 0
drwxr-xr-x@ 4 user  staff  128 Nov 25 21:11 .
drwxr-xr-x@ 3 user  staff   96 Nov 25 21:11 ..
drwxr-xr-x  7 user  staff  224 Nov 25 21:11 SDWebImage.framework
drwxr-xr-x  5 user  staff  160 Nov 25 21:11 SDWebImageSwiftUI.framework

 git:(develop) ✗  ls -la /Users/user/Library/Developer/Xcode/DerivedData/Workspace-egfgfeihvgnlcxdwkslotvzjreap/Build/Intermediates.noindex/
total 0
drwxr-xr-x@ 5 user  staff  160 Nov 25 21:11 .
drwxr-xr-x  4 user  staff  128 Nov 25 21:11 ..
drwxr-xr-x  3 user  staff   96 Nov 25 21:11 SDWebImage.build
drwxr-xr-x  3 user  staff   96 Nov 25 21:11 SDWebImageSwiftUI.build
drwxr-xr-x  8 user  staff  256 Nov 25 21:11 XCBuildData
mflknr commented 2 years ago

Hello,

i have the exact same issue. Any news on this one? Thanks

kikeenrique commented 2 years ago

I can just update report info. I've updated the example repo to tuist 3.0 and Xcode 13.2.1 and stills fails to build. Logs in https://github.com/kikeenrique/tuist-spm-sdwebimageswiftui-fails/runs/5536077748

danieleformichelli commented 2 years ago

Thanks for the update, I think there is some problem when an Objc target depends on an SPM dependency with modulemap, as I suspect that the -fmodule-map-file only works on swift targets 🤔 We would need to investigate it more and find a solution 🙏

mflknr commented 2 years ago

For everyone who wants to implement the workaround mentioned in #4255 you need to add "HEADER_SEARCH_PATHS": "$(inherited) $(SRCROOT)/SDWebImage/include/SDWebImage $(SRCROOT)/SDWebImage/Private $(SRCROOT)/SDWebImage/Core $(SRCROOT)/WebImage" to the SDWebImage dependency target settings and everything works fine (for now and Xcode 13.2.1)

kikeenrique commented 2 years ago

I don't really get where exactly should I add the HEADER_SEARCH_PATHS definition using the reported example repository https://github.com/kikeenrique/tuist-spm-sdwebimageswiftui-fails Any help?

mflknr commented 2 years ago

Hey @kikeenrique, we have multiple options when adding settings to targets. You can do it at project level, but in this case this won't help with external dependencies. But tuist provides an option to set settings for external dependencies. My current setup is summed up by this example:

let dependencies = Dependencies(
    swiftPackageManager: SwiftPackageManagerDependencies(
        [
            .remote(url: "https://github.com/SDWebImage/SDWebImageSwiftUI.git", requirement: .exact("2.0.2"))
            // all your other dependencies
        ],
        baseSettings: makeFrameworkSettings(),
        targetSettings: makeTargetSettings()
    )
)

public static func makeTargetSettings() -> [String: SettingsDictionary] {
    var settings: [String: SettingsDictionary] = [:]
    let allYourDependencyNames  = ["SDWebImage"] // add all other dependency names here, this is at least what I am doing
    allYourDependencyNames.forEach { dependency in
        settings[dependency] = makeBaseSettings()
        settings[dependency] = ["HEADER_SEARCH_PATHS": "$(inherited) $(SRCROOT)/SDWebImage/include/SDWebImage $(SRCROOT)/SDWebImage/Private $(SRCROOT)/SDWebImage/Core $(SRCROOT)/WebImage"]
    }

    return settings
}

static func makeFrameworkSettings() -> Settings {
    .settings(
        base: makeFrameworkBaseSettings(),
        configurations: [
            // just an example
            .debug(name: "Debug"),
            .release(name: "AdHoc"),
            .release(name: "Release")
        ]
    )
}

// this is a workaround for named configuration that are different to "Debug" and "Release".
static func makeFrameworkBaseSettings() -> SettingsDictionary {
    [
        "FRAMEWORK_SEARCH_PATHS": "$(inherited) $(SYMROOT)/Release$(EFFECTIVE_PLATFORM_NAME)"
    ]
}

static func makeBaseSettings() -> SettingsDictionary {
        SettingsDictionary()
            .marketingVersion(C.version)
            .currentProjectVersion(C.currentBuild)
            .swiftVersion(C.swiftVersion)
            .swiftOptimizeObjectLifetimes(false)
            .bitcodeEnabled(true)
            .merging([
                "OTHER_LDFLAGS": "-ObjC",
                "PUSH_URL_SCHEME": "fressnapfapp",
                "ALWAYS_SEARCH_USER_PATHS": "NO",
                .... you other settings here
            ])
}

So, currently I set project settings for every project + set explicits settings for every external dependency. Additionally just for SDWebImage I add the HEADER_SEARCH_PATHS. I hope that helps!

Banck commented 2 years ago

Is there any progress? The same issue with GRPC (CNIOBoringSSL)

The latest library in our project which has to be integrated via spm instead of dependency.swift

makleso6 commented 2 years ago

Same issue with CNIOBoringSSL

kikeenrique commented 2 years ago

Today I've reviewing the original issue. And... it's been fixed in some release (currently 3.8.0 in CI jobs). I've two CI jobs that have passed build+test using both approaches

func makeTargetSettings() -> [String: SettingsDictionary] { var settings: [String: SettingsDictionary] = [:] let allYourDependencyNames = ["SDWebImage"] // add all other dependency names here, this is at least what I am doing allYourDependencyNames.forEach { dependency in settings[dependency] = makeBaseSettings() settings[dependency] = ["HEADER_SEARCH_PATHS": "$(inherited) $(SRCROOT)/SDWebImage/include/SDWebImage $(SRCROOT)/SDWebImage/Private $(SRCROOT)/SDWebImage/Core $(SRCROOT)/WebImage"] }

return settings

}

func makeFrameworkSettings() -> Settings { .settings( base: makeFrameworkBaseSettings() ) }

// this is a workaround for named configuration that are different to "Debug" and "Release". func makeFrameworkBaseSettings() -> SettingsDictionary { [ "FRAMEWORK_SEARCH_PATHS": "$(inherited) $(SYMROOT)" ] }

let dependencies = Dependencies( swiftPackageManager: .init( [.remote(url: "https://github.com/SDWebImage/SDWebImageSwiftUI/", requirement: .upToNextMajor(from: "2.0.2") ) ], baseSettings: makeFrameworkSettings(), targetSettings: makeTargetSettings()) )

danieleformichelli commented 2 years ago

Thanks for reporting @kikeenrique !

sanghyeok-kim commented 6 months ago

@pepicrft Is there still nothing resolved in Tuist regarding this? Is setting HEADER_SEARCH_PATHS for headers of all Objc libraries like this is the only solution for now?

pepicrft commented 6 months ago

@sanghyeok-kim some packages that contain Swift protocols or Objective-C categories might require that upstream targets include the -ObjC OTHER_LDFLAGS to prevent the linker from stripping too much. Unfortunately, we can't know with certainty how to reliably apply the flags ourselves. Projects like Bazel and CocoaPods did so and ended up reverting it. However, note that the package authors can make some changes in their packages for the integration to work without requiring you to use the flag. If you tell me which package is causing issues, I can open a PR in it.

sanghyeok-kim commented 6 months ago

@sanghyeok-kim some packages that contain Swift protocols or Objective-C categories might require that upstream targets include the -ObjC OTHER_LDFLAGS to prevent the linker from stripping too much. Unfortunately, we can't know with certainty how to reliably apply the flags ourselves. Projects like Bazel and CocoaPods did so and ended up reverting it. However, note that the package authors can make some changes in their packages for the integration to work without requiring you to use the flag. If you tell me which package is causing issues, I can open a PR in it.

I found the relevant history, and now I can understand what you've described. Thank you :)