search

grpc / grpc-swift

The Swift language implementation of gRPC.
Apache License 2.0
2.01k stars 414 forks source link

Carthage not working for 0.5.1 & 0.6.0 #329

Closed sbarow closed 5 years ago

sbarow commented 5 years ago

Carthage: 0.31.2 Tried with: github "grpc/grpc-swift" "23a0ebdee9613f615f2f2469ed3e700df5856417" github "grpc/grpc-swift" "0.6.0" github "grpc/grpc-swift" "0.5.1"

Originally posted by @sbarow in https://github.com/grpc/grpc-swift/issues/13#issuecomment-436341804

MrMage commented 5 years ago

Please include additional details such as error messages and what exactly you tried. I'm not an expert for the Carthage support, maybe @JonasVautherin can help you.

/cc @sbarow

sbarow commented 5 years ago

Just created a Cartfile with github "grpc/grpc-swift" and ran carthage update. This is the output:

*** Fetching grpc-swift
*** Checking out grpc-swift at "0.6.0"
*** xcodebuild output can be found in /var/folders/cs/08bq_hfn2kz5vcspfqr3dc380000gn/T/carthage-xcodebuild.PaQANO.log
*** Building scheme "SwiftGRPC-Carthage-Package" in SwiftGRPC-Carthage.xcodeproj
Build Failed
    Task failed with exit code 65:
    /usr/bin/xcrun xcodebuild -project /Users/cammc/Desktop/protobug/Carthage/Checkouts/grpc-swift/SwiftGRPC-Carthage.xcodeproj -scheme SwiftGRPC-Carthage-Package -configuration Release -derivedDataPath /Users/cammc/Library/Caches/org.carthage.CarthageKit/DerivedData/10.0_10A255/grpc-swift/0.6.0 -sdk appletvos ONLY_ACTIVE_ARCH=NO CODE_SIGNING_REQUIRED=NO CODE_SIGN_IDENTITY= CARTHAGE=YES archive -archivePath /var/folders/cs/08bq_hfn2kz5vcspfqr3dc380000gn/T/grpc-swift SKIP_INSTALL=YES GCC_INSTRUMENT_PROGRAM_FLOW_ARCS=NO CLANG_ENABLE_CODE_COVERAGE=NO STRIP_INSTALLED_PRODUCT=NO (launched in /Users/cammc/Desktop/protobug/Carthage/Checkouts/grpc-swift)

This usually indicates that project itself failed to compile. Please check the xcodebuild log for more details: /var/folders/cs/08bq_hfn2kz5vcspfqr3dc380000gn/T/carthage-xcodebuild.PaQANO.log

In the log file:

error: Build input files cannot be found: 'protobug/Carthage/Checkouts/grpc-swift/.build/checkouts/swift-protobuf.git--7219529775138357838/Sources/SwiftProtobuf/NameMap.swift', 'protobug/Carthage/Checkouts/grpc-swift/.build/checkouts/swift-protobuf.git--7219529775138357838/Sources/SwiftProtobuf/TextFormatEncodingVisitor.swift', 'protobug/Carthage/Checkouts/grpc-swift/.build/checkouts/swift-protobuf.git--7219529775138357838/Sources/SwiftProtobuf/ZigZag.swift', 'protobug/Carthage/Checkouts/grpc-swift/.build/checkouts/swift-protobuf.git--7219529775138357838/Sources/SwiftProtobuf/BinaryEncodingSizeVisitor.swift', 'protobug/Carthage/Checkouts/grpc-swift/.build/checkouts/swift-protobuf.git--7219529775138357838/Sources/SwiftProtobuf/Version.swift', 'protobug/Carthage/Checkouts/grpc-swift/.build/checkouts/swift-protobuf.git--7219529775138357838/Sources/SwiftProtobuf/CustomJSONCodable.swift', 'protobug/Carthage/Checkouts/grpc-swift/.build/checkouts/swift-protobuf.git--7219529775138357838/Sources/SwiftProtobuf/Google_Protobuf_FieldMask+Extensions.swift', 'protobug/Carthage/Checkouts/grpc-swift/.build/checkouts/swift-protobuf.git--7219529775138357838/Sources/SwiftProtobuf/Message+BinaryAdditions.swift', 'protobug/Carthage/Checkouts/grpc-swift/.build/checkouts/swift-protobuf.git--7219529775138357838/Sources/SwiftProtobuf/duration.pb.swift', 'protobug/Carthage/Checkouts/grpc-swift/.build/checkouts/swift-protobuf.git--7219529775138357838/Sources/SwiftProtobuf/BinaryEncoder.swift', 'protobug/Carthage/Checkouts/grpc-swift/.build/checkouts/swift-protobuf.git--7219529775138357838/Sources/SwiftProtobuf/Google_Protobuf_Wrappers+Extensions.swift', 'protobug/Carthage/Checkouts/grpc-swift/.build/checkouts/swift-protobuf.git--7219529775138357838/Sources/SwiftProtobuf/Message+JSONAdditions.swift', 'protobug/Carthage/Checkouts/grpc-swift/.build/checkouts/swift-protobuf.git--7219529775138357838/Sources/SwiftProtobuf/BinaryDecodingOptions.swift', 'protobug/Carthage/Checkouts/grpc-swift/.build/checkouts/swift-protobuf.git--7219529775138357838/Sources/SwiftProtobuf/type.pb.swift', 'protobug/Carthage/Checkouts/grpc-swift/.build/checkouts/swift-protobuf.git--7219529775138357838/Sources/SwiftProtobuf/FieldTypes.swift', 'protobug/Carthage/Checkouts/grpc-swift/.build/checkouts/swift-protobuf.git--7219529775138357838/Sources/SwiftProtobuf/Varint.swift', 'protobug/Carthage/Checkouts/grpc-swift/.build/checkouts/swift-protobuf.git--7219529775138357838/Sources/SwiftProtobuf/AnyUnpackError.swift', 'protobug/Carthage/Checkouts/grpc-swift/.build/checkouts/swift-protobuf.git--7219529775138357838/Sources/SwiftProtobuf/any.pb.swift', 'protobug/Carthage/Checkouts/grpc-swift/.build/checkouts/swift-protobuf.git--7219529775138357838/Sources/SwiftProtobuf/BinaryEncodingVisitor.swift', 'protobug/Carthage/Checkouts/grpc-swift/.build/checkouts/swift-protobuf.git--7219529775138357838/Sources/SwiftProtobuf/BinaryDelimited.swift', 'protobug/Carthage/Checkouts/grpc-swift/.build/checkouts/swift-protobuf.git--7219529775138357838/Sources/SwiftProtobuf/JSONScanner.swift', 'protobug/Carthage/Checkouts/grpc-swift/.build/checkouts/swift-protobuf.git--7219529775138357838/Sources/SwiftProtobuf/Message+JSONArrayAdditions.swift', 'protobug/Carthage/Checkouts/grpc-swift/.build/checkouts/swift-protobuf.git--7219529775138357838/Sources/SwiftProtobuf/TextFormatEncoder.swift', 'protobug/Carthage/Checkouts/grpc-swift/.build/checkouts/swift-protobuf.git--7219529775138357838/Sources/SwiftProtobuf/TextFormatScanner.swift', 'protobug/Carthage/Checkouts/grpc-swift/.build/checkouts/swift-protobuf.git--7219529775138357838/Sources/SwiftProtobuf/ProtoNameProviding.swift', 'protobug/Carthage/Checkouts/grpc-swift/.build/checkouts/swift-protobuf.git--7219529775138357838/Sources/SwiftProtobuf/BinaryDecoder.swift', 'protobug/Carthage/Checkouts/grpc-swift/.build/checkouts/swift-protobuf.git--7219529775138357838/Sources/SwiftProtobuf/JSONEncodingVisitor.swift', 'protobug/Carthage/Checkouts/grpc-swift/.build/checkouts/swift-protobuf.git--7219529775138357838/Sources/SwiftProtobuf/ExtensionMap.swift', 'protobug/Carthage/Checkouts/grpc-swift/.build/checkouts/swift-protobuf.git--7219529775138357838/Sources/SwiftProtobuf/ExtensibleMessage.swift', 'protobug/Carthage/Checkouts/grpc-swift/.build/checkouts/swift-protobuf.git--7219529775138357838/Sources/SwiftProtobuf/empty.pb.swift', 'protobug/Carthage/Checkouts/grpc-swift/.build/checkouts/swift-protobuf.git--7219529775138357838/Sources/SwiftProtobuf/Visitor.swift', 'protobug/Carthage/Checkouts/grpc-swift/.build/checkouts/swift-protobuf.git--7219529775138357838/Sources/SwiftProtobuf/MathUtils.swift', 'protobug/Carthage/Checkouts/grpc-swift/.build/checkouts/swift-protobuf.git--7219529775138357838/Sources/SwiftProtobuf/Enum.swift', 'protobug/Carthage/Checkouts/grpc-swift/.build/checkouts/swift-protobuf.git--7219529775138357838/Sources/SwiftProtobuf/Google_Protobuf_Value+Extensions.swift', 'protobug/Carthage/Checkouts/grpc-swift/.build/checkouts/swift-protobuf.git--7219529775138357838/Sources/SwiftProtobuf/JSONMapEncodingVisitor.swift', 'protobug/Carthage/Checkouts/grpc-swift/.build/checkouts/swift-protobuf.git--7219529775138357838/Sources/SwiftProtobuf/struct.pb.swift', 'protobug/Carthage/Checkouts/grpc-swift/.build/checkouts/swift-protobuf.git--7219529775138357838/Sources/SwiftProtobuf/JSONDecoder.swift', 'protobug/Carthage/Checkouts/grpc-swift/.build/checkouts/swift-protobuf.git--7219529775138357838/Sources/SwiftProtobuf/Message+TextFormatAdditions.swift', 'protobug/Carthage/Checkouts/grpc-swift/.build/checkouts/swift-protobuf.git--7219529775138357838/Sources/SwiftProtobuf/FieldTag.swift', 'protobug/Carthage/Checkouts/grpc-swift/.build/checkouts/swift-protobuf.git--7219529775138357838/Sources/SwiftProtobuf/TextFormatDecodingError.swift', 'protobug/Carthage/Checkouts/grpc-swift/.build/checkouts/swift-protobuf.git--7219529775138357838/Sources/SwiftProtobuf/timestamp.pb.swift', 'protobug/Carthage/Checkouts/grpc-swift/.build/checkouts/swift-protobuf.git--7219529775138357838/Sources/SwiftProtobuf/TimeUtils.swift', 'protobug/Carthage/Checkouts/grpc-swift/.build/checkouts/swift-protobuf.git--7219529775138357838/Sources/SwiftProtobuf/JSONEncoder.swift', 'protobug/Carthage/Checkouts/grpc-swift/.build/checkouts/swift-protobuf.git--7219529775138357838/Sources/SwiftProtobuf/ProtobufAPIVersionCheck.swift', 'protobug/Carthage/Checkouts/grpc-swift/.build/checkouts/swift-protobuf.git--7219529775138357838/Sources/SwiftProtobuf/Message+AnyAdditions.swift', 'protobug/Carthage/Checkouts/grpc-swift/.build/checkouts/swift-protobuf.git--7219529775138357838/Sources/SwiftProtobuf/Google_Protobuf_Any+Registry.swift', 'protobug/Carthage/Checkouts/grpc-swift/.build/checkouts/swift-protobuf.git--7219529775138357838/Sources/SwiftProtobuf/Google_Protobuf_Duration+Extensions.swift', 'protobug/Carthage/Checkouts/grpc-swift/.build/checkouts/swift-protobuf.git--7219529775138357838/Sources/SwiftProtobuf/DoubleFormatter.swift', 'protobug/Carthage/Checkouts/grpc-swift/.build/checkouts/swift-protobuf.git--7219529775138357838/Sources/SwiftProtobuf/Google_Protobuf_Timestamp+Extensions.swift', 'protobug/Carthage/Checkouts/grpc-swift/.build/checkouts/swift-protobuf.git--7219529775138357838/Sources/SwiftProtobuf/Internal.swift', 'protobug/Carthage/Checkouts/grpc-swift/.build/checkouts/swift-protobuf.git--7219529775138357838/Sources/SwiftProtobuf/SimpleExtensionMap.swift', 'protobug/Carthage/Checkouts/grpc-swift/.build/checkouts/swift-protobuf.git--7219529775138357838/Sources/SwiftProtobuf/BinaryDecodingError.swift', 'protobug/Carthage/Checkouts/grpc-swift/.build/checkouts/swift-protobuf.git--7219529775138357838/Sources/SwiftProtobuf/WireFormat.swift', 'protobug/Carthage/Checkouts/grpc-swift/.build/checkouts/swift-protobuf.git--7219529775138357838/Sources/SwiftProtobuf/source_context.pb.swift', 'protobug/Carthage/Checkouts/grpc-swift/.build/checkouts/swift-protobuf.git--7219529775138357838/Sources/SwiftProtobuf/Message.swift', 'protobug/Carthage/Checkouts/grpc-swift/.build/checkouts/swift-protobuf.git--7219529775138357838/Sources/SwiftProtobuf/ProtobufMap.swift', 'protobug/Carthage/Checkouts/grpc-swift/.build/checkouts/swift-protobuf.git--7219529775138357838/Sources/SwiftProtobuf/TextFormatDecoder.swift', 'protobug/Carthage/Checkouts/grpc-swift/.build/checkouts/swift-protobuf.git--7219529775138357838/Sources/SwiftProtobuf/JSONDecodingError.swift', 'protobug/Carthage/Checkouts/grpc-swift/.build/checkouts/swift-protobuf.git--7219529775138357838/Sources/SwiftProtobuf/api.pb.swift', 'protobug/Carthage/Checkouts/grpc-swift/.build/checkouts/swift-protobuf.git--7219529775138357838/Sources/SwiftProtobuf/Decoder.swift', 'protobug/Carthage/Checkouts/grpc-swift/.build/checkouts/swift-protobuf.git--7219529775138357838/Sources/SwiftProtobuf/ExtensionFields.swift', 'protobug/Carthage/Checkouts/grpc-swift/.build/checkouts/swift-protobuf.git--7219529775138357838/Sources/SwiftProtobuf/Google_Protobuf_Struct+Extensions.swift', 'protobug/Carthage/Checkouts/grpc-swift/.build/checkouts/swift-protobuf.git--7219529775138357838/Sources/SwiftProtobuf/HashVisitor.swift', 'protobug/Carthage/Checkouts/grpc-swift/.build/checkouts/swift-protobuf.git--7219529775138357838/Sources/SwiftProtobuf/SelectiveVisitor.swift', 'protobug/Carthage/Checkouts/grpc-swift/.build/checkouts/swift-protobuf.git--7219529775138357838/Sources/SwiftProtobuf/field_mask.pb.swift', 'protobug/Carthage/Checkouts/grpc-swift/.build/checkouts/swift-protobuf.git--7219529775138357838/Sources/SwiftProtobuf/ExtensionFieldValueSet.swift', 'protobug/Carthage/Checkouts/grpc-swift/.build/checkouts/swift-protobuf.git--7219529775138357838/Sources/SwiftProtobuf/StringUtils.swift', 'protobug/Carthage/Checkouts/grpc-swift/.build/checkouts/swift-protobuf.git--7219529775138357838/Sources/SwiftProtobuf/JSONEncodingError.swift', 'protobug/Carthage/Checkouts/grpc-swift/.build/checkouts/swift-protobuf.git--7219529775138357838/Sources/SwiftProtobuf/UnknownStorage.swift', 'protobug/Carthage/Checkouts/grpc-swift/.build/checkouts/swift-protobuf.git--7219529775138357838/Sources/SwiftProtobuf/Google_Protobuf_ListValue+Extensions.swift', 'protobug/Carthage/Checkouts/grpc-swift/.build/checkouts/swift-protobuf.git--7219529775138357838/Sources/SwiftProtobuf/AnyMessageStorage.swift', 'protobug/Carthage/Checkouts/grpc-swift/.build/checkouts/swift-protobuf.git--7219529775138357838/Sources/SwiftProtobuf/wrappers.pb.swift', 'protobug/Carthage/Checkouts/grpc-swift/.build/checkouts/swift-protobuf.git--7219529775138357838/Sources/SwiftProtobuf/JSONDecodingOptions.swift', 'protobug/Carthage/Checkouts/grpc-swift/.build/checkouts/swift-protobuf.git--7219529775138357838/Sources/SwiftProtobuf/BinaryEncodingError.swift', 'protobug/Carthage/Checkouts/grpc-swift/.build/checkouts/swift-protobuf.git--7219529775138357838/Sources/SwiftProtobuf/Google_Protobuf_Any+Extensions.swift', 'protobug/Carthage/Checkouts/grpc-swift/.build/checkouts/swift-protobuf.git--7219529775138357838/Sources/SwiftProtobuf/MessageExtension.swift'

** ARCHIVE FAILED **

The following build commands failed:
    CompileSwiftSources normal arm64 com.apple.xcode.tools.swift.compiler
(1 failure)
sbarow commented 5 years ago

Following up on this running make build-carthage & carthage build --no-skip-current succeeds if run in the project directory.

Would it make sense to distribute pre-built frameworks Archive Prebuilt Frameworks?

JonasVautherin commented 5 years ago

Oh, I answered on the other issue.

@sbarow are you on Xcode 10? I think there is something related to that.

sbarow commented 5 years ago

@JonasVautherin yeah I was trying this on Xcode 10.

JonasVautherin commented 5 years ago

@MrMage: Let's continue on this open issue.

I was saying:

Somehow it doesn't build with Xcode 10, and I haven't understood why yet. Xcode 9.4.1 works, though. I'm not sure what Xcode 10 is doing but it somehow changes the path = ".build/checkouts/swift-protobuf.git- in the project file and then doesn't find it -_-.

And @MrMage answered:

could it be that the paths generated by swift resolve are different for Xcode 10 vs. 9.4? Or maybe Xcode 10/Swift 4.2 simply doesn't generate deterministic paths for the dependencies, while 9.4/Swift 4.1 does? That would also explain why I never was able to create a working Carthage project for the NIO branch.

I believe I started testing that, and now I don't remember what was my conclusion. I'll do it again tomorrow or on Monday. Let's hope it's the first one (deterministic but different between Xcode 10 and Xcode 9). @sbarow @MrMage let me know if you get further information about that =).

sbarow commented 5 years ago

Looks like the issue is the paths being output in the .build folder. When the Carthage project is generated it would have something like this:

grpc-swift⁩/.build⁩/⁨checkouts⁩/⁨swift-protobuf.git-5652066567309619809⁩

While the carthage update will end up generating:

grpc-swift⁩/.build⁩/checkouts⁩/swift-protobuf.git-442897717507903005⁩

Which is why you get a whole bunch of errors like 

error: Build input files cannot be found: '.../grpc-swift/.build/checkouts/swift-protobuf.git-5652066567309619809/Sources/SwiftProtobuf/TextFormatEncodingVisitor.swift',

Is it possible to not output the swift-protobuf.git-XXX/... in the checkouts and rather just have something like swift-protobuf/...

JonasVautherin commented 5 years ago

If you run it multiple times, does it always end up being 442897717507903005⁩? If it does, it means that right now, grpc-swift supports carthage for Xcode 9, but at some point it could move to Xcode 10 (or even we could think about having a tag for one of them, I don't know).

Is it possible to not output the swift-protobuf.git-XXX/... in the checkouts and rather just have something like swift-protobuf/...

I don't know how it works, actually. I just know that it comes from the SwiftPM when we run swift package resolve. Maybe we should ask the SwiftPM people if they would have an elegant solution for that (e.g. what you suggest, if that's acceptable for SwiftPM).

sbarow commented 5 years ago

I don't think its the resolve, it actually happens on the make project-carthage which internally calls swift package generate-xcodeproj. That does the checkout for SPM hence it has a reference to the swift-protobuf.git-XXX files.

If there is no .build folder that's when it looks like those names are assigned, so each clean update would have different names which makes sense why a carthage update would break.

MrMage commented 5 years ago

For what it’s worth, this is exactly the problem I have been trying to describe in https://github.com/grpc/grpc-swift/pull/282#issuecomment-429210956 and the following comments.

@sbarow when downloading the project with Carthage, swift package generate-xcodeproj is not called, but the Carthage project contains an extra build step that calls swift package resolve. Is that what you mean?

JonasVautherin commented 5 years ago

For what it’s worth, this is exactly the problem I have been trying to describe in

282 (comment) and the following comments.

Right, I remember. Back then I was just observing that it was deterministic on Xcode 9. I'll definitely check that for Xcode 10.

sbarow commented 5 years ago

@MrMage what I meant is when a "new release" is done in the grpc-swift repo and you run make project-carthage then internally swift package generate-xcodeproj is run which will do checkouts etc. The commit then contains the reference to file paths of .../swift-protobuf.git-XXX/.... You then do carthage update in your own repo which would have the swift package resolve build step run and it will then create a new .build directory with a different .../swift-protobuf.git-YYY/...

MrMage commented 5 years ago

Ah, thanks for the explanation. That's indeed the case, see my previous comments.

JonasVautherin commented 5 years ago

@sbarow: But I do believe that the file paths generated by swift package resolve are deterministic (at least before Xcode 10 / Swift 4.2).

I have been using grpc-swift from Carthage for weeks in Xcode 9.4.1, and so have at least 4 other developers. So it really seems to me like it works with Xcode 9.4.1 (if you refer to the right commit in your Cartfile, which we do here).

So I see two things here:

Now, what I don't get is how this works: because swift package resolve doesn't modify the xcodeproj (I guess?), how can the xcodeproj know that path before running swift package resolve in the first place?

MrMage commented 5 years ago

Now, what I don't get is how this works: because swift package resolve doesn't modify the xcodeproj (I guess?), how can the xcodeproj know that path before running swift package resolve in the first place?

The xcodeproj is generated by the maintainers of this project via make project-carthage and uses whatever .build/ paths are currently used by the maintainer's local checkout. swift resolve on the other hand is called on the user's Mac when using Carthage, which might have different paths as you described.

MrMage commented 5 years ago

Looks like Xcode 10's dependency paths are no longer deterministic, I'm afraid:

daniel@Daniels-iMac ~/Desktop $ git clone git@github.com:grpc/grpc-swift.git
Cloning into 'grpc-swift'...
remote: Enumerating objects: 10, done.
remote: Counting objects: 100% (10/10), done.
remote: Compressing objects: 100% (10/10), done.
remote: Total 11135 (delta 0), reused 0 (delta 0), pack-reused 11125
Receiving objects: 100% (11135/11135), 8.08 MiB | 5.16 MiB/s, done.
Resolving deltas: 100% (6931/6931), done.
daniel@Daniels-iMac ~/Desktop $ cd grpc-swift/
daniel@Daniels-iMac ~/Desktop/grpc-swift $ swift package resolve
Fetching https://github.com/apple/swift-protobuf.git
Fetching https://github.com/kylef/Commander.git
Fetching https://github.com/apple/swift-nio-zlib-support.git
Fetching https://github.com/kylef/Spectre.git
Completed resolution in 7.76s
Cloning https://github.com/kylef/Commander.git
Resolving https://github.com/kylef/Commander.git at 0.8.0
Cloning https://github.com/kylef/Spectre.git
Resolving https://github.com/kylef/Spectre.git at 0.9.0
Cloning https://github.com/apple/swift-protobuf.git
Resolving https://github.com/apple/swift-protobuf.git at 1.1.2
Cloning https://github.com/apple/swift-nio-zlib-support.git
Resolving https://github.com/apple/swift-nio-zlib-support.git at 1.0.0
daniel@Daniels-iMac ~/Desktop/grpc-swift $ ls -la .build/checkouts/
total 0
drwxr-x---   6 daniel  staff  192 Nov  8 15:55 .
drwxr-x---   5 daniel  staff  160 Nov  8 15:55 ..
drwxr-xr-x  15 daniel  staff  480 Nov  8 15:55 Commander.git-4775119199020034144
drwxr-xr-x  16 daniel  staff  512 Nov  8 15:55 Spectre.git-3013504057989409880
drwxr-xr-x   8 daniel  staff  256 Nov  8 15:55 swift-nio-zlib-support.git--4283068970447824954
drwxr-xr-x  21 daniel  staff  672 Nov  8 15:55 swift-protobuf.git--8570329225063932037
daniel@Daniels-iMac ~/Desktop/grpc-swift $ rm -rf .build/ Package.resolved 
daniel@Daniels-iMac ~/Desktop/grpc-swift $ swift package resolve
Fetching https://github.com/apple/swift-protobuf.git
Fetching https://github.com/kylef/Commander.git
Fetching https://github.com/apple/swift-nio-zlib-support.git
Fetching https://github.com/kylef/Spectre.git
Completed resolution in 9.95s
Cloning https://github.com/apple/swift-protobuf.git
Resolving https://github.com/apple/swift-protobuf.git at 1.1.2
Cloning https://github.com/kylef/Commander.git
Resolving https://github.com/kylef/Commander.git at 0.8.0
Cloning https://github.com/apple/swift-nio-zlib-support.git
Resolving https://github.com/apple/swift-nio-zlib-support.git at 1.0.0
Cloning https://github.com/kylef/Spectre.git
Resolving https://github.com/kylef/Spectre.git at 0.9.0
daniel@Daniels-iMac ~/Desktop/grpc-swift $ ls -la .build/checkouts/
total 0
drwxr-x---   6 daniel  staff  192 Nov  8 15:55 .
drwxr-x---   5 daniel  staff  160 Nov  8 15:55 ..
drwxr-xr-x  15 daniel  staff  480 Nov  8 15:55 Commander.git--3197549877292877740
drwxr-xr-x  16 daniel  staff  512 Nov  8 15:55 Spectre.git--2968694622967762684
drwxr-xr-x   8 daniel  staff  256 Nov  8 15:55 swift-nio-zlib-support.git-5230833211210166877
drwxr-xr-x  21 daniel  staff  672 Nov  8 15:55 swift-protobuf.git-1432166696668694164
JonasVautherin commented 5 years ago

I'm asking on the Swift Forums about that, maybe they can help.

I hope there is a solution, because otherwise, it would look like Xcode 10 / Swift 4.2 is fundamentally incompatible with Carthage, and that's pretty bad...

sbarow commented 5 years ago

Could a solution be vendoring pre-built frameworks?

MrMage commented 5 years ago

I don't think so, as Swift modules are not yet ABI-stable. Also, we do not want to pollute the repository with build artifacts like that.

sbarow commented 5 years ago

Don’t think ABI stability would matter in this case. You can have a project that has Swift 4 & 4.2 modules. But I do get not wanting to pollute the repo.

ishkawa commented 5 years ago

I think we can support Carthage on Xcode 10 in other way like below:

  1. Remove SwiftProtobuf target, which is not deterministic, from SwiftGRPC-Carthage.xcodeproj.
  2. Add Cartfile which contains github "apple/swift-protobuf".
  3. Create .xcworkspace file that contains SwiftGRPC-Carthage.xcodeproj and Carthage/Checkouts/swift-protobuf/SwiftProtobuf.xcodeproj.
  4. Let SwiftGRPC target to link product of Carthage/Checkouts/swift-protobuf/SwiftProtobuf.xcodeproj.

2, 3 and 4 are standard steps to support Carthage in project which has dependencies. For example, ReactiveSwift follows this way. To maintenance as simple as current project, these steps should be done by patch-carthage-project.rb.

A while ago, I tried similar way here and succeeded to support Carthage for iOS project. This only supports iOS project because it references Carthage/Build/iOS directly. I guess it can support multiple platforms if we introduce xcworkspace and reference product directory. You can try it by github "ishkawa/grpc-swift" "0.6.0-xcode10-carthage-ios".

JonasVautherin commented 5 years ago

@ishkawa Thanks a lot for your input! If I understand correctly, you suggest that the issue is coming from SwiftProtobuf. What if grpc-swift added more dependencies? What I like in the current Carthage support (that unfortunatly doesn't work for Xcode 10 anymore) is that it doesn't require to maintain the dependencies (even though there is this constraint of running make project-carthage for each release).

However, if you say it is standard in the Carthage world, I can totally believe you (I am new to Carthage, even as a user).

On the swift forums, I received a suggestion saying:

Do in a pre-build phase:

swift package resolve
swift package edit <dep_1_name>
swift package edit <dep_2_name>

Then in a post-build phase, do:

swift package unedit <dep_1_name>
swift package unedit <dep_2_name>

You could write a script to parse out information from the Package.resolved file so you know what dependencies you have. The issue is that then the xcodeproj doesn't know about them and won't compile or link against them. So you will likely need to manually maintain them in the xcodeproj. You will just want to put dependencies into edit mode when you add them to the project file.

What do you think about that?

JonasVautherin commented 5 years ago

@ishkawa Thinking about it, why not just removing the SwiftPM dependencies and using a Cartfile for that? @MrMage would that be an option?

I see that SwiftPM and Cocoapods are already maintained in parallel, so I guess the dependencies could also be maintained in a Cartfile, couldn't they?

JonasVautherin commented 5 years ago

Note: I can confirm that swift package resolve is deterministic in swift 4.1.2 but not in swift 4.2.1.

I kind of liked how it worked with swift 4.1.2, where swift package resolve was called as a pre-build phase in the xcodeproj. So I asked for clarification here on the Swift forums.

But I guess the next step is really to implement @ishkawa's suggestion above.

JonasVautherin commented 5 years ago

(Sorry for spamming the issue, but I've got news):

From Aciid comment's here, it sounds like it is a bad idea to generate the xcodeproj from SwiftPM for Carthage.

From stephencelis comment's here, it seems like we could generate an xcode project using XcodeGen, and share that one on the repo. Which means that Cocoapods, Carthage and SwiftPM would be maintained separately, but it seems like it is the only truly correct way...

sbarow commented 5 years ago

@JonasVautherin nice, I can take a look at the XcodeGen stuff if we are okay with doing that. Have been using it for a while on some projects.

MrMage commented 5 years ago

I personally like the idea of generating the project like that; it feels similar to what CocoaPods is doing (just in a less automated fashion).

However, I assume this would require us to implement this approach in all dependency projects (at least SwiftProtobuf) as well? I.e. they would need a working Carthage project in a similar fashion?

On 14. Nov 2018, at 11:49, Cameron Mc Gorian notifications@github.com wrote:

@JonasVautherin nice, I can take a look at the XcodeGen stuff if we are okay with doing that. Have been using it for a while on some projects.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

ishkawa commented 5 years ago

@JonasVautherin

If I understand correctly, you suggest that the issue is coming from SwiftProtobuf. What if grpc-swift added more dependencies?

Same as SwiftProtobuf, because the issue is not SwiftProtobuf-specific. We can add a new dependency by following steps:

  1. Add a line to Cartfile.
  2. Modify patch-carthage-project.rb to add the new framework to xcworkspace.
  3. Run make project-carthage.

Thinking about it, why not just removing the SwiftPM dependencies and using a Cartfile for that?

Purpose of 3 and 4 in my previous comment are to let Xcode to know:

Without the steps, xcodebuild run by Carthage will not build SwiftProtobuf.

Actually, it is a little bit complicated to generate xcodeproj compatible with Carthage. Especially, when a project has dependencies. With XcodeGen, we can do the same things in more straightforward way. It would be a good choice.

MrMage commented 5 years ago

But how are we supposed to pull in the dependencies (ie SwiftProtobuf currently) in a reproducible fashion?

ishkawa commented 5 years ago

Carthage has Cartfile.resolved to record checked out version. The file is expected to be committed in this repository. When we run carthage bootstrap on root of this project, Carthage references the file and pull the same dependencies.

MrMage commented 5 years ago

But for that to work, wouldn't the dependencies need to support Carthage first as well?

JonasVautherin commented 5 years ago

nice, I can take a look at the XcodeGen stuff if we are okay with doing that. Have been using it for a while on some projects.

@sbarow: That would be awesome! And I am obviously wanting to help you if I can, as I need Carthage support myself :-). I think that @MrMage and @ishkawa agree that using XcodeGen would be an elegant solution (please correct me if I'm wrong, but that's what I understood from the comments), so I would be in favor of trying that.

But for that to work, wouldn't the dependencies need to support Carthage first as well?

@MrMage: SwiftProtobuf does support Carthage. Would that solve this issue?

Looking at the grpc-swift podspec, the dependencies for CocoaPods are SwiftProtobuf, gRPC-Core and BoringSSL. But the last two are built as part of gRPC-Swift, right?

MrMage commented 5 years ago

@sbarow: That would be awesome! And I am obviously wanting to help you if I can, as I need Carthage support myself :-). I think that @MrMage and @ishkawa agree that using XcodeGen would be an elegant solution (please correct me if I'm wrong, but that's what I understood from the comments), so I would be in favor of trying that.

Indeed!

But for that to work, wouldn't the dependencies need to support Carthage first as well?

@MrMage: SwiftProtobuf does support Carthage. Would that solve this issue?

Sorry, I was not aware of that. Looking at the grpc-swift podspec, the dependencies for CocoaPods are SwiftProtobuf, gRPC-Core and BoringSSL. But the last two are built as part of gRPC-Swift, right?

At least for CocoaPods, we are using the BoringSSL Pod that is hosted inside the core grpc/grpc repo. I guess we would either need to make that one Carthage-supporting (if it isn't), or find a different, existing, Carthage- supporting OpenSSL-like (eg BoringSSL) repo. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

sbarow commented 5 years ago

I have started working on an XcodeGen version. It will contain 3 targets BoringSSL, CgRPC, SwiftGRPC regarding SwiftProtobuf we will get with Carthage and add as a dependency in a Cartfile

I'll try have a WIP diff up soon, getting the build settings correct for BoringSSL & CgRPC will be a bit tricky.

MrMage commented 5 years ago

Thanks for the heads-up! Please disregard my comment about BoringSSL; I forgot that we are shipping the source code of that in the repository. Sorry again! (I've only been able to use my phone for most of the past few days, so can't look into the code too much.)

On 15. Nov 2018, at 07:56, Cameron Mc Gorian notifications@github.com wrote:

I have started working on an XcodeGen version. It will contain 3 targets BoringSSL, CgRPC, SwiftGRPC regarding SwiftProtobuf we will get with Carthage and add as a dependency in a Cartfile

I'll try have a WIP diff up soon, getting the build settings correct for BoringSSL & CgRPC will be a bit tricky.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

JonasVautherin commented 5 years ago

I'll try have a WIP diff up soon, getting the build settings correct for BoringSSL & CgRPC will be a bit tricky.

Note that we are doing stuff here: I had to fix the xcodeproj generated by SwiftPM for Carthage. I was also removing targets here.

Well, I guess you may just be interested in looking at this, which is what we do to generate the Carthage xcodeproj (and this works with swift 4.1).

JonasVautherin commented 5 years ago

@sbarow Did you start doing something with Xcodegen? I'm having trouble compiling for Xcode 10 so I would love to have a look at that.

sbarow commented 5 years ago

@JonasVautherin I have been bogged down with stuff at work, I'll only be able to look at this again next week.

JonasVautherin commented 5 years ago

@sbarow Did you have a chance to have a look at it? I failed miserably on #336... Probably because I don't know xcodegen. I was hoping you would have more success there.

MrMage commented 5 years ago

Hopefully fixed by #360; thanks to @byuarus and @JonasVautherin for making this happen!

sbarow commented 5 years ago

Amazing, thank you @byuarus & @JonasVautherin

sbarow commented 5 years ago

@MrMage side note, this issue might be of interest to watch Carthage: Add support for Swift Package Manager only projects

MrMage commented 5 years ago

Indeed! Closing this for now.