grpc / grpc-swift

The Swift language implementation of gRPC.
Apache License 2.0
2.05k stars 420 forks source link

"undefined symbols" error when building for release. #1369

Open pacu opened 2 years ago

pacu commented 2 years ago

What are you trying to achieve?

I'm the maintainer of ZcashLightClientKit. I'm trying to release a new version of the SDK through Cocoapods and I'm having problems with SwiftNIO (it's a dependency of grpc-swift).

What have you tried so far?

Building for release fails. :( whereas debug builds seem to work fine (probably because the active architecture of my mac is intel 64-bit)

The error after pod trunk push --skip-tests --allow-warnings verbose

    Undefined symbols for architecture arm64:
      "_catmc_nio_atomic__Bool_compare_and_exchange", referenced from:
          NIOPosix.PendingDatagramWritesManager.add(envelope: NIOCore.AddressedEnvelope<NIOCore.ByteBuffer>, promise: NIOCore.EventLoopPromise<()>?) -> Swift.Bool in PendingDatagramWritesManager.o
          NIOPosix.PendingStreamWritesManager.add(data: NIOCore.IOData, promise: NIOCore.EventLoopPromise<()>?) -> Swift.Bool in PendingWritesManager.o
      "_catmc_nio_atomic__Bool_create_with_existing_storage", referenced from:
          NIOPosix.PendingDatagramWritesManager.init(msgs: Swift.UnsafeMutableBufferPointer<__C.CNIODarwin_mmsghdr>, iovecs: Swift.UnsafeMutableBufferPointer<__C.iovec>, addresses: Swift.UnsafeMutableBufferPointer<__C.sockaddr_storage>, storageRefs: Swift.UnsafeMutableBufferPointer<Swift.Unmanaged<Swift.AnyObject>>, controlMessageStorage: NIOPosix.UnsafeControlMessageStorage) -> NIOPosix.PendingDatagramWritesManager in PendingDatagramWritesManager.o
          NIOPosix.PendingStreamWritesManager.init(iovecs: Swift.UnsafeMutableBufferPointer<__C.iovec>, storageRefs: Swift.UnsafeMutableBufferPointer<Swift.Unmanaged<Swift.AnyObject>>) -> NIOPosix.PendingStreamWritesManager in PendingWritesManager.o
          function signature specialization <Arg[1] = Owned To Guaranteed, Arg[2] = Owned To Guaranteed> of generic specialization <NIOPosix.ServerSocket> of NIOPosix.BaseSocketChannel.init(socket: A, parent: NIOCore.Channel?, eventLoop: NIOPosix.SelectableEventLoop, recvAllocator: NIOCore.RecvByteBufferAllocator) throws -> NIOPosix.BaseSocketChannel<A> in SocketChannel.o
          function signature specialization <Arg[1] = Owned To Guaranteed, Arg[2] = Owned To Guaranteed> of generic specialization <NIOPosix.PipePair> of NIOPosix.BaseSocketChannel.init(socket: A, parent: NIOCore.Channel?, eventLoop: NIOPosix.SelectableEventLoop, recvAllocator: NIOCore.RecvByteBufferAllocator) throws -> NIOPosix.BaseSocketChannel<A> in SocketChannel.o
      "_catmc_nio_atomic_long_create_with_existing_storage", referenced from:
          variable initialization expression of NIOPosix.MultiThreadedEventLoopGroup.(index in _C2B1528F4FBA68A3DBFA89DBAEBE9D4D) : NIOConcurrencyHelpers.NIOAtomic<Swift.Int> in BaseSocket.o
          one-time initialization function for (nextEventLoopGroupID in _C2B1528F4FBA68A3DBFA89DBAEBE9D4D) in MultiThreadedEventLoopGroup.o
          function signature specialization <Arg[0] = Owned To Guaranteed, Arg[1] = Owned To Guaranteed> of NIOPosix.MultiThreadedEventLoopGroup.init(threadInitializers: [(NIOPosix.NIOThread) -> ()], selectorFactory: () throws -> NIOPosix.Selector<NIOPosix.NIORegistration>) -> NIOPosix.MultiThreadedEventLoopGroup in MultiThreadedEventLoopGroup.o
      "_catmc_nio_atomic__Bool_store", referenced from:
          NIOPosix.BaseSocketChannel.close0(error: Swift.Error, mode: NIOCore.CloseMode, promise: NIOCore.EventLoopPromise<()>?) -> () in BaseSocketChannel.o
          NIOPosix.BaseSocketChannel.becomeActive0(promise: NIOCore.EventLoopPromise<()>?) -> () in BaseSocketChannel.o
          NIOPosix.PendingDatagramWritesManager.(didWrite in _DAB463F912477C8697B1F8AEBF8BD15A)(_: NIOPosix.IOResult<Swift.Int>, messages: Swift.UnsafeMutableBufferPointer<__C.CNIODarwin_mmsghdr>?) -> NIOPosix.OneWriteOperationResult in PendingDatagramWritesManager.o
          function signature specialization <Arg[0] = [Closure Propagated : closure #1 (NIOPosix.WriteMechanism) throws -> NIOPosix.OneWriteOperationResult in NIOPosix.PendingStreamWritesManager.triggerAppropriateWriteOperations(scalarBufferWriteOperation: (Swift.UnsafeRawBufferPointer) throws -> NIOPosix.IOResult<Swift.Int>, vectorBufferWriteOperation: (Swift.UnsafeBufferPointer<__C.iovec>) throws -> NIOPosix.IOResult<Swift.Int>, scalarFileWriteOperation: (Swift.Int32, Swift.Int, Swift.Int) throws -> NIOPosix.IOResult<Swift.Int>) throws -> NIOPosix.OverallWriteResult, Argument Types : [NIOPosix.PendingStreamWritesManager@callee_guaranteed (@unowned Swift.UnsafeRawBufferPointer) -> (@unowned NIOPosix.IOResult<Swift.Int>, @error @owned Swift.Error)@callee_guaranteed (@unowned Swift.UnsafeBufferPointer<__C.iovec>) -> (@unowned NIOPosix.IOResult<Swift.Int>, @error @owned Swift.Error)@callee_guaranteed (@unowned Swift.Int32, @unowned Swift.Int, @unowned Swift.Int) -> (@unowned NIOPosix.IOResult<Swift.Int>, @error @owned Swift.Error)]> of generic specialization <NIOPosix.PendingStreamWritesManager> of (extension in NIOPosix):NIOPosix.PendingWritesManager.triggerWriteOperations(triggerOneWriteOperation: (NIOPosix.WriteMechanism) throws -> NIOPosix.OneWriteOperationResult) throws -> NIOPosix.OverallWriteResult in PendingWritesManager.o
          NIOPosix.PendingStreamWritesManager.(didWrite in _E8AD929129EAA2971226E80CBC6171E1)(itemCount: Swift.Int, result: NIOPosix.IOResult<Swift.Int>) -> NIOPosix.OneWriteOperationResult in PendingWritesManager.o
          generic specialization <NIOPosix.ServerSocket> of NIOPosix.BaseSocketChannel.close0(error: Swift.Error, mode: NIOCore.CloseMode, promise: NIOCore.EventLoopPromise<()>?) -> () in SocketChannel.o
          generic specialization <NIOPosix.Socket> of NIOPosix.BaseSocketChannel.close0(error: Swift.Error, mode: NIOCore.CloseMode, promise: NIOCore.EventLoopPromise<()>?) -> () in SocketChannel.o
          ...
      "_catmc_nio_atomic__Bool_load", referenced from:
          NIOPosix.BaseSocketChannel.isActive.getter : Swift.Bool in BaseSocketChannel.o
          NIOPosix.BaseStreamSocketChannel.isWritable.getter : Swift.Bool in BaseStreamSocketChannel.o
          function signature specialization <Arg[0] = [Closure Propagated : closure #1 (NIOPosix.WriteMechanism) throws -> NIOPosix.OneWriteOperationResult in NIOPosix.PendingDatagramWritesManager.triggerAppropriateWriteOperations(scalarWriteOperation: (Swift.UnsafeRawBufferPointer, Swift.UnsafePointer<__C.sockaddr>, Swift.UInt32, Swift.Optional<NIOCore.AddressedEnvelope<NIOCore.ByteBuffer>.Metadata>) throws -> NIOPosix.IOResult<Swift.Int>, vectorWriteOperation: (Swift.UnsafeMutableBufferPointer<__C.CNIODarwin_mmsghdr>) throws -> NIOPosix.IOResult<Swift.Int>) throws -> NIOPosix.OverallWriteResult, Argument Types : [NIOPosix.PendingDatagramWritesManager@callee_guaranteed (@unowned Swift.UnsafeRawBufferPointer, @unowned Swift.UnsafePointer<__C.sockaddr>, @unowned Swift.UInt32, @guaranteed NIOCore.AddressedEnvelope<NIOCore.ByteBuffer>.Metadata?) -> (@unowned NIOPosix.IOResult<Swift.Int>, @error @owned Swift.Error)@callee_guaranteed (@unowned Swift.UnsafeMutableBufferPointer<__C.CNIODarwin_mmsghdr>) -> (@unowned NIOPosix.IOResult<Swift.Int>, @error @owned Swift.Error)]> of generic specialization <NIOPosix.PendingDatagramWritesManager> of (extension in NIOPosix):NIOPosix.PendingWritesManager.triggerWriteOperations(triggerOneWriteOperation: (NIOPosix.WriteMechanism) throws -> NIOPosix.OneWriteOperationResult) throws -> NIOPosix.OverallWriteResult in PendingWritesManager.o
          function signature specialization <Arg[0] = [Closure Propagated : closure #1 (NIOPosix.WriteMechanism) throws -> NIOPosix.OneWriteOperationResult in NIOPosix.PendingStreamWritesManager.triggerAppropriateWriteOperations(scalarBufferWriteOperation: (Swift.UnsafeRawBufferPointer) throws -> NIOPosix.IOResult<Swift.Int>, vectorBufferWriteOperation: (Swift.UnsafeBufferPointer<__C.iovec>) throws -> NIOPosix.IOResult<Swift.Int>, scalarFileWriteOperation: (Swift.Int32, Swift.Int, Swift.Int) throws -> NIOPosix.IOResult<Swift.Int>) throws -> NIOPosix.OverallWriteResult, Argument Types : [NIOPosix.PendingStreamWritesManager@callee_guaranteed (@unowned Swift.UnsafeRawBufferPointer) -> (@unowned NIOPosix.IOResult<Swift.Int>, @error @owned Swift.Error)@callee_guaranteed (@unowned Swift.UnsafeBufferPointer<__C.iovec>) -> (@unowned NIOPosix.IOResult<Swift.Int>, @error @owned Swift.Error)@callee_guaranteed (@unowned Swift.Int32, @unowned Swift.Int, @unowned Swift.Int) -> (@unowned NIOPosix.IOResult<Swift.Int>, @error @owned Swift.Error)]> of generic specialization <NIOPosix.PendingStreamWritesManager> of (extension in NIOPosix):NIOPosix.PendingWritesManager.triggerWriteOperations(triggerOneWriteOperation: (NIOPosix.WriteMechanism) throws -> NIOPosix.OneWriteOperationResult) throws -> NIOPosix.OverallWriteResult in PendingWritesManager.o
          NIOPosix.PendingStreamWritesManager.description.getter : Swift.String in PendingWritesManager.o
          NIOPosix.PipeChannel.description.getter : Swift.String in PipeChannel.o
          NIOPosix.DatagramChannel.isWritable.getter : Swift.Bool in SocketChannel.o
          ...
      "_catmc_nio_atomic_unsigned_long_long_create_with_existing_storage", referenced from:
          variable initialization expression of NIOPosix.SelectableEventLoop.scheduledTaskCounter : NIOConcurrencyHelpers.NIOAtomic<Swift.UInt64> in BaseSocket.o
          function signature specialization <Arg[0] = Owned To Guaranteed, Arg[1] = Owned To Guaranteed, Arg[2] = Owned To Guaranteed> of NIOPosix.SelectableEventLoop.init(thread: NIOPosix.NIOThread, parentGroup: NIOPosix.MultiThreadedEventLoopGroup?, selector: NIOPosix.Selector<NIOPosix.NIORegistration>, canBeShutdownIndividually: Swift.Bool) -> NIOPosix.SelectableEventLoop in SelectableEventLoop.o
      "_catmc_nio_atomic_long_add", referenced from:
          NIOPosix.MultiThreadedEventLoopGroup.next() -> NIOCore.EventLoop in MultiThreadedEventLoopGroup.o
          function signature specialization <Arg[0] = Owned To Guaranteed, Arg[1] = Owned To Guaranteed> of NIOPosix.MultiThreadedEventLoopGroup.init(threadInitializers: [(NIOPosix.NIOThread) -> ()], selectorFactory: () throws -> NIOPosix.Selector<NIOPosix.NIORegistration>) -> NIOPosix.MultiThreadedEventLoopGroup in MultiThreadedEventLoopGroup.o
      "_catmc_nio_atomic_unsigned_long_long_add", referenced from:
          merged generic specialization <NIOPosix.SelectableEventLoop, NIOCore.EventLoopFuture<()>> of (extension in NIOCore):NIOCore.EventLoop.submit<A>(() throws -> A1) -> NIOCore.EventLoopFuture<A1> in Bootstrap.o
          NIOPosix.SelectableEventLoop.initiateClose(queue: __C.OS_dispatch_queue, completionHandler: (Swift.Result<(), Swift.Error>) -> ()) -> () in SelectableEventLoop.o
          NIOPosix.SelectableEventLoop.scheduleTask<A>(deadline: NIOCore.NIODeadline, _: () throws -> A) -> NIOCore.Scheduled<A> in SelectableEventLoop.o
          NIOPosix.SelectableEventLoop.execute(() -> ()) -> () in SelectableEventLoop.o
          closure #2 (Swift.Result<(), Swift.Error>) -> () in doClose #1 () -> () in NIOPosix.SelectableEventLoop.initiateClose(queue: __C.OS_dispatch_queue, completionHandler: (Swift.Result<(), Swift.Error>) -> ()) -> () in SelectableEventLoop.o
          generic specialization <NIOPosix.ServerSocket> of NIOPosix.BaseSocketChannel.close0(error: Swift.Error, mode: NIOCore.CloseMode, promise: NIOCore.EventLoopPromise<()>?) -> () in SocketChannel.o
          generic specialization <NIOPosix.Socket> of NIOPosix.BaseSocketChannel.close0(error: Swift.Error, mode: NIOCore.CloseMode, promise: NIOCore.EventLoopPromise<()>?) -> () in SocketChannel.o
          ...
    ld: symbol(s) not found for architecture arm64
    clang: error: linker command failed with exit code 1 (use -v to see invocation)
Lukasa commented 2 years ago

This seems to be a build or linkage issue. Can you provide the full compile output, or some indication of what modules got built and linked?

pacu commented 2 years ago

Yes sure. I apologize. I got anxious and forgot to add version of the package I'm using. 😅 which is 1.0.0 This is the output of the pod trunk push command log.txt

FranzBusch commented 2 years ago

I see from your podspec that you fixed the gRPC version to 1.0.0. This is pretty old, and many fixes have been released since. Any reason for this hard version requirement? Could you try upping this to the latest version?

steve-ham commented 2 years ago

I'm having the same problem as as @pacu.

I did not specify a version s.dependency 'gRPC-Swift' so would have downloaded the latest version (1.7.1.), even looking at the log, it looks like it did.

Resolving dependencies of 
  CDN: trunk Relative path: CocoaPods-version.yml exists! Returning local because checking is only performed in repo update
  CDN: trunk Relative path: all_pods_versions_f_7_4.txt exists! Returning local because checking is only performed in repo update
  CDN: trunk Relative path: Specs/f/7/4/gRPC-Swift/1.7.1/gRPC-Swift.podspec.json exists! Returning local because checking is only performed in repo update

Error log:

Undefined symbols for architecture x86_64:
      "_catmc_nio_atomic__Bool_compare_and_exchange", referenced from:
          NIOPosix.PendingDatagramWritesManager.add(envelope: NIOCore.AddressedEnvelope<NIOCore.ByteBuffer>, promise: NIOCore.EventLoopPromise<()>?) -> Swift.Bool in PendingDatagramWritesManager.o
          NIOPosix.PendingStreamWritesManager.add(data: NIOCore.IOData, promise: NIOCore.EventLoopPromise<()>?) -> Swift.Bool in PendingWritesManager.o

Reproduction steps:

  1. Download MyLibrary: https://github.com/steve-ham/MyLibrary
  2. cd /Users/steve/Downloads/MyLibrary
  3. pod lib lint --verbose

Error log link: https://github.com/steve-ham/MyLibrary/blob/master/grpc-swift-error-log.txt

Hypothesis: PendingDatagramWritesManager.swift compareAndExchange function in line 430 is undefined. Import is not working correctly.

import NIOCore
import NIOConcurrencyHelpers
Lukasa commented 2 years ago

Here's the SwiftNIOPOSIX podspec: https://github.com/CocoaPods/Specs/blob/master/Specs/3/6/7/SwiftNIOPosix/2.38.0/SwiftNIOPosix.podspec.json. This appears to correctly express a dependency on SwiftNIOConcurrencyHelpers, which defines that symbol.

steve-ham commented 2 years ago

Yea, then my hypothesis is wrong but the error's still there. So maybe it's caused by some other reason.

Does "pod lib lint" work successfully with the project link that I provided for you guys? https://github.com/steve-ham/MyLibrary

Thanks for prompt response and looking into this.

pacu commented 2 years ago

@steve-ham thanks for putting this together. I cloned your repo and tried to do pod lib lint --verbose 2> error_log.txt 1> error_log.txt

this is the error I got. error_log.txt error_log.txt

mateuszfidos commented 2 years ago

Hey! Any update on this? I'm having the same issue. Different grpc-swift versions (1.7.1, 1.6.1, 1.6.0, 1.5.0) but the same outcome.

pacu commented 2 years ago

I don't know how I did it but it stopped happening in my case. Super weird

steve-ham commented 2 years ago

For me I'm still getting the same below error

Undefined symbols for architecture x86_64:
      "_catmc_nio_atomic__Bool_compare_and_exchange", referenced from:
          NIOPosix.PendingDatagramWritesManager.add(envelope: NIOCore.AddressedEnvelope<NIOCore.ByteBuffer>, promise: 
steve-ham commented 2 years ago

@Lukasa @FranzBusch It has been over a month since this issue opened (March 11th). Is someone looking into this? Does it look like this is a complicated problem and will take a long time? Is there anything I can help to accelerate debugging? Thanks.

Lukasa commented 2 years ago

Sorry, I'd missed that the pod lib --lint output contained a complete compile log.

The first failed compile step attempts to link NIOPosix, and the command line invocation is:

/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang
 -target x86_64-apple-ios10.0-simulator
 -dynamiclib
 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator15.2.sdk
 -L/Users/steve/Library/Developer/Xcode/DerivedData/App-cqtbcyakcqxmvtbzqvlbycstjulk/Build/Products/Release-iphonesimulator/SwiftNIOPosix
 -L/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/swift/iphonesimulator
 -L/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator15.2.sdk/usr/lib/swift
 -F/Users/steve/Library/Developer/Xcode/DerivedData/App-cqtbcyakcqxmvtbzqvlbycstjulk/Build/Products/Release-iphonesimulator/SwiftNIOPosix
 -F/Users/steve/Library/Developer/Xcode/DerivedData/App-cqtbcyakcqxmvtbzqvlbycstjulk/Build/Products/Release-iphonesimulator/CNIOAtomics
 -F/Users/steve/Library/Developer/Xcode/DerivedData/App-cqtbcyakcqxmvtbzqvlbycstjulk/Build/Products/Release-iphonesimulator/CNIODarwin
 -F/Users/steve/Library/Developer/Xcode/DerivedData/App-cqtbcyakcqxmvtbzqvlbycstjulk/Build/Products/Release-iphonesimulator/CNIOLinux
 -F/Users/steve/Library/Developer/Xcode/DerivedData/App-cqtbcyakcqxmvtbzqvlbycstjulk/Build/Products/Release-iphonesimulator/CNIOWindows
 -F/Users/steve/Library/Developer/Xcode/DerivedData/App-cqtbcyakcqxmvtbzqvlbycstjulk/Build/Products/Release-iphonesimulator/SwiftNIOConcurrencyHelpers
 -F/Users/steve/Library/Developer/Xcode/DerivedData/App-cqtbcyakcqxmvtbzqvlbycstjulk/Build/Products/Release-iphonesimulator/SwiftNIOCore
 -F/Users/steve/Library/Developer/Xcode/DerivedData/App-cqtbcyakcqxmvtbzqvlbycstjulk/Build/Products/Release-iphonesimulator/_NIODataStructures
 -filelist /Users/steve/Library/Developer/Xcode/DerivedData/App-cqtbcyakcqxmvtbzqvlbycstjulk/Build/Intermediates.noindex/Pods.build/Release-iphonesimulator/SwiftNIOPosix.build/Objects-normal/x86_64/NIOPosix.LinkFileList
 -install_name @rpath/NIOPosix.framework/NIOPosix
 -Xlinker -rpath
 -Xlinker /usr/lib/swift
 -Xlinker -rpath
 -Xlinker @executable_path/Frameworks
 -Xlinker -rpath
 -Xlinker @loader_path/Frameworks
 -dead_strip
 -Xlinker -object_path_lto
 -Xlinker /Users/steve/Library/Developer/Xcode/DerivedData/App-cqtbcyakcqxmvtbzqvlbycstjulk/Build/Intermediates.noindex/Pods.build/Release-iphonesimulator/SwiftNIOPosix.build/Objects-normal/x86_64/NIOPosix_lto.o
 -Xlinker -objc_abi_version
 -Xlinker 2
 -fobjc-arc
 -fobjc-link-runtime
 -L/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/swift/iphonesimulator
 -L/usr/lib/swift
 -Xlinker -add_ast_path
 -Xlinker /Users/steve/Library/Developer/Xcode/DerivedData/App-cqtbcyakcqxmvtbzqvlbycstjulk/Build/Intermediates.noindex/Pods.build/Release-iphonesimulator/SwiftNIOPosix.build/Objects-normal/x86_64/NIOPosix.swiftmodule
 -framework CNIODarwin
 -framework CNIOLinux
 -framework CNIOWindows
 -framework NIOConcurrencyHelpers
 -framework NIOCore
 -framework _NIODataStructures
 -framework Foundation
 -Xlinker -no_adhoc_codesign
 -compatibility_version 1
 -current_version 1
 -Xlinker -dependency_info
 -Xlinker /Users/steve/Library/Developer/Xcode/DerivedData/App-cqtbcyakcqxmvtbzqvlbycstjulk/Build/Intermediates.noindex/Pods.build/Release-iphonesimulator/SwiftNIOPosix.build/Objects-normal/x86_64/NIOPosix_dependency_info.dat
 -o /Users/steve/Library/Developer/Xcode/DerivedData/App-cqtbcyakcqxmvtbzqvlbycstjulk/Build/Intermediates.noindex/Pods.build/Release-iphonesimulator/SwiftNIOPosix.build/Objects-normal/x86_64/Binary/NIOPosix

This appears to correctly request NIOConcurrencyHelpers, but it does not request CNIOAtomics. This is unlike what happens in several of the compile steps, which do appropriately request to include CNIOAtomics. This is a problem, because CNIOAtomics is ultimately what defines this symbol.

Ultimately I am unfamiliar with the compilation mechanism of Cocoapods so it's not really easy for me to diagnose this, but it appears that the principal issue is that Cocoapods is not linking CNIOAtomics appropriately with things that transitively depend on it. My guess, however, is that this is caused by @inlinable annotations hitting the public API of a different module (that is, we've got an @inlinable attribute in NIOConcurrencyHelpers that exposes symbols from CNIOAtomics) but Cocoapods not appropriately linking the target. This is also probably related to the fact that Cocoapods appears to be producing frameworks, which are presumably dynamically linked, rather than the statically linked melange that SwiftPM produces.

I think we'd need someone expert in Cocoapods to be more confident, but my guess is that we probably have to rewrite the podspecs to express the full transitive linkage graph for each pod: that is, we need to tell Cocoapods to link not only every one of the actual module dependencies but also every one of the transitive set to guard against this kind of problem.

mateuszfidos commented 2 years ago

Thanks @Lukasa, that's a great overview of the issue. Cocoapods seem to be very nondeterministic in failing to link those libs properly. I have this framework workflow setup on CI and I always got to run it twice. First time it always fails and the second time the podspsecs get linted and uploaded properly. I'm assuming Cocoapods are caching something but still I don't understand how this affects linking and how linking can be nondeterministic, but again I'm lacking the compiler knowledge so I dunno...

I guess I gotta think about SPM support :)

steve-ham commented 2 years ago

One of the weird things is I was able to successfully install grpc-swift as dependency and use it on March 9, 2022 9:36 PM. But something changed since then and I am not able to install it anymore.

steve-ham commented 2 years ago

gRPC-ProtoRPC grpc objc gets installed as dependency successfully so I guess I'll use this until grpc-swift/cocoapods gets fixed.

Lukasa commented 2 years ago

Linking cannot be nondeterministic, but build order can affect the outcome. In particular it's possible to try to link a library that is not present because it hasn't been built yet.

pacu commented 2 years ago

The issue came back out of nowhere again.

      "_catmc_nio_atomic_long_add", referenced from:
          NIOPosix.MultiThreadedEventLoopGroup.next() -> NIOCore.EventLoop in MultiThreadedEventLoopGroup.o
          function signature specialization <Arg[0] = Owned To Guaranteed, Arg[1] = Owned To Guaranteed> of NIOPosix.MultiThreadedEventLoopGroup.init(threadInitializers: [(NIOPosix.NIOThread) -> ()], selectorFactory: () throws -> NIOPosix.Selector<NIOPosix.NIORegistration>) -> NIOPosix.MultiThreadedEventLoopGroup in MultiThreadedEventLoopGroup.o
      "_catmc_nio_atomic_unsigned_long_long_add", referenced from:
          merged generic specialization <NIOPosix.SelectableEventLoop, NIOCore.EventLoopFuture<()>> of (extension in NIOCore):NIOCore.EventLoop.submit<A>(() throws -> A1) -> NIOCore.EventLoopFuture<A1> in Bootstrap.o
          NIOPosix.SelectableEventLoop.initiateClose(queue: __C.OS_dispatch_queue, completionHandler: (Swift.Result<(), Swift.Error>) -> ()) -> () in SelectableEventLoop.o
          NIOPosix.SelectableEventLoop.scheduleTask<A>(deadline: NIOCore.NIODeadline, _: () throws -> A) -> NIOCore.Scheduled<A> in SelectableEventLoop.o
          NIOPosix.SelectableEventLoop.execute(() -> ()) -> () in SelectableEventLoop.o
          closure #2 (Swift.Result<(), Swift.Error>) -> () in doClose #1 () -> () in NIOPosix.SelectableEventLoop.initiateClose(queue: __C.OS_dispatch_queue, completionHandler: (Swift.Result<(), Swift.Error>) -> ()) -> () in SelectableEventLoop.o
          generic specialization <NIOPosix.Socket> of NIOPosix.BaseSocketChannel.close0(error: Swift.Error, mode: NIOCore.CloseMode, promise: NIOCore.EventLoopPromise<()>?) -> () in SocketChannel.o
          merged generic specialization <NIOPosix.ServerSocket> of NIOPosix.BaseSocketChannel.close0(error: Swift.Error, mode: NIOCore.CloseMode, promise: NIOCore.EventLoopPromise<()>?) -> () in SocketChannel.o
          ...
    ld: symbol(s) not found for architecture arm64
    clang: error: linker command failed with exit code 1 (use -v to see invocation)

    /var/folders/zd/zss_3kv112337jq_8hm28j540000gn/T/CocoaPods-Lint-20220506-4052-10mcqm-ZcashLightClientKit/Pods/Pods.xcodeproj: warning: The iOS Simulator deployment target 'IPHONEOS_DEPLOYMENT_TARGET' is set to 8.0, but the range of supported deployment target versions is 9.0 to 15.4.99. (in target 'SQLite.swift' from project 'Pods')
    /var/folders/zd/zss_3kv112337jq_8hm28j540000gn/T/CocoaPods-Lint-20220506-4052-10mcqm-ZcashLightClientKit/Pods/Pods.xcodeproj: warning: The iOS Simulator deployment target 'IPHONEOS_DEPLOYMENT_TARGET' is set to 8.0, but the range of supported deployment target versions is 9.0 to 15.4.99. (in target 'Logging' from project 'Pods')
    ** BUILD FAILED **

    The following build commands failed:
        CompileSwiftSources normal x86_64 com.apple.xcode.tools.swift.compiler (in target 'SwiftNIO' from project 'Pods')
        CompileSwiftSources normal arm64 com.apple.xcode.tools.swift.compiler (in target 'SwiftNIO' from project 'Pods')
        Ld /Users/pacu/Library/Developer/Xcode/DerivedData/App-cohvakelomzoyibjhnwigsiedzsv/Build/Intermediates.noindex/Pods.build/Release-iphonesimulator/SwiftNIOPosix.build/Objects-normal/x86_64/Binary/NIOPosix normal x86_64 (in target 'SwiftNIOPosix' from project 'Pods')
        Ld /Users/pacu/Library/Developer/Xcode/DerivedData/App-cohvakelomzoyibjhnwigsiedzsv/Build/Intermediates.noindex/Pods.build/Release-iphonesimulator/SwiftNIOPosix.build/Objects-normal/arm64/Binary/NIOPosix normal arm64 (in target 'SwiftNIOPosix' from project 'Pods')

Unfortunately I don't know what triggered it. I thought it was related to having the iPhone plugged in, but it makes no difference if it is or not.

pacu commented 2 years ago

The same code succeeded building when I ran it through CI. Bitrise does not give me information about the mac used being M1 or Intel so this could be related to the processor architecture.

Lukasa commented 2 years ago

I don't think the processor architecture is likely to be relevant here. What version of the cocoapods are you using?

mateuszfidos commented 2 years ago

@pacu I tried both intel and m1 with the same outcome so it seems processor architecture is not relevant here

pacu commented 2 years ago

I don't think the processor architecture is likely to be relevant here. What version of the cocoapods are you using?

1.11.3

Lukasa commented 2 years ago

Sorry, I meant of the grpc and NIO cocoapods.

pacu commented 2 years ago
Installing SwiftNIO (2.40.0)
Installing gRPC-Swift (1.7.3)
Lukasa commented 2 years ago

What build setup do you have? In particular, how are you building and for what devices?

pacu commented 2 years ago

I'm running pod lint lib on this Pod https://github.com/zcash/ZcashLightClientKit/

   /Applications/Xcode.app/Contents/Developer/usr/bin/xcodebuild clean build -workspace /var/folders/zd/zss_3kv112337jq_8hm28j540000gn/T/CocoaPods-Lint-20220511-6135-aqliu8-ZcashLightClientKit/App.xcworkspace -scheme App -configuration Release CODE_SIGN_IDENTITY=- -sdk iphonesimulator -destination id=C568B267-EEC1-45F1-B6D3-39DD9B0CB099

    User defaults from command line:
        IDEPackageSupportUseBuiltinSCM = YES

    Build settings from command line:
        CODE_SIGN_IDENTITY = -
        SDKROOT = iphonesimulator15.4

    note: Using new build system
    note: Using codesigning identity override: -
    note: Build preparation complete
  /Applications/Xcode.app/Contents/Developer/usr/bin/xcodebuild -version 
Xcode 13.3.1
Build version 13E500a
Lukasa commented 2 years ago

Can you provide the full output that includes all the compilation lines?

pacu commented 2 years ago

Can you provide the full output that includes all the compilation lines?

Guess which command started to work successfully out of nowhere 🥶