search

xmtp / xmtp-react-native

A package you can use to build with XMTP in a React Native or Expo app.
MIT License
41 stars 19 forks source link

Can not import latest XMTPiOS with LibXMTP #221

Closed cameronvoell closed 8 months ago

cameronvoell commented 8 months ago

Describe the bug

  pod "XMTP", :git => 'https://github.com/xmtp/xmtp-ios.git'
  pod "LibXMTP", :git => 'https://github.com/xmtp/libxmtp-swift'

With the above lines added to my Example app Podfile, I get an error during pod install

[!] Unable to install vendored xcframework LibXMTPRust for Pod LibXMTP because it contains dynamic libraries which are not supported. Use dynamic frameworks for dynamic linking instead.

This error is consistent with apple developer documentation on XCFrameworks (https://developer.apple.com/documentation/xcode/creating-a-multi-platform-binary-framework-bundle):

Avoid using dynamic library files (.dylib files) for dynamic linking. An XCFramework can include dynamic library files, but only macOS supports these libraries for dynamic linking. Dynamic linking on iOS, watchOS, and tvOS requires the XCFramework to contain .framework bundles.

Expected behavior

We normally need to be able to build xmtp-react-native sdk with the lastest xmtp-ios main branch so that we can integrate fixes and features into xmtp-react-native that require updates on xmtp-ios repo.

Steps to reproduce the bug

Test building xmtp-react-native against the latest xmtp-ios by adding the following to your Example app Podfile:

  pod "XMTP", :git => 'https://github.com/xmtp/xmtp-ios.git'
  pod "LibXMTP", :git => 'https://github.com/xmtp/libxmtp-swift'
neekolas commented 8 months ago

Does changing to the static library make this go away?

cameronvoell commented 8 months ago

Does changing to the static library make this go away?

This is fixed using the static lib branch, which can be tested with the Podfile branch update as seen below

pod "XMTP", :git => 'https://github.com/xmtp/xmtp-ios.git'
pod "LibXMTP", :git => 'https://github.com/xmtp/libxmtp-swift', :branch => 'cv/staticlib'

*Note you likely need to have git-lfs installed on your machine because of the large static binary sizes. Follow instructions from here: https://git-lfs.com/

brew install git-lfs
git lfs install
cameronvoell commented 8 months ago

@nakajima For my libxmtp-swift branch, I took binaries from this libxmtp PR (latest commit f8139fc5fca4291d5c7c1951f4a052a49774e5ee)

I'm not sure if that specific libxmtp commit has been tested against the xmtp-ios Example app yet, as I don't think the xmtp-ios example app worked against my libxmtp-swift branch when I tried earlier.

cameronvoell commented 8 months ago

Avoid using dynamic library files (.dylib files) for dynamic linking. An XCFramework can include dynamic library files, but only macOS supports these libraries for dynamic linking. Dynamic linking on iOS, watchOS, and tvOS requires the XCFramework to contain .framework bundles.

@neekolas also wanted to note that the static library files might not be the only way to fix this. In the line above it mentions ".framework bundles" for dynamic linking on iOS. It's been tricky to find documentation on .framework bundles, but I did find an example of one in a repo here => https://github.com/phrase/ios-sdk/tree/master/PhraseSDK.xcframework . The common theme seems to be directory structure with <library name>.framework folder that has a binary in it just named <library name> with no suffix. This stack overflow for example has one answer maybe with some guidance on how to generate that .framework bundle for ios support of a dynamic library. (though most comments seem to recommend building static)

All that being said, it still seems like the vast majority of examples and recommendations say use static libraries for iOS. Evidence for going the static route include:

  1. Uniffi docs "Note: You also need to add staticlib crate type if you target iOS.
  2. Matrix Crypto XCFramework uses static libraries (iOS binary is 99MB) - https://github.com/matrix-org/matrix-rust-sdk/releases
  3. Matrix Components Swift XCFramework uses static libraries (iOS binary is 339MB) - https://github.com/matrix-org/matrix-rust-components-swift/releases

My recommendation would be to try and get xmtp-ios, xmtp-react-native, and libxmtp-swift all working with the static libraries from your libxmtp staticlib branch at least for groups alpha. Perhaps for beta we can look into optimizing to decrease the static binary sizes or switching to iOS using .framework bundles for dynamic libraries (as opposed to the .dylib which doesnt seem to be supported on iOS).

neekolas commented 8 months ago

FWIW, we do export a .xcframework today. You should be able to generate one locally by running make framework in the bindings_ffi folder of libxmtp. That framework gets pushed into libxmtp-swift on every commit to main in libxmtp. The .dylib file is part of that framework.

cameronvoell commented 8 months ago

FWIW, we do export a .xcframework today. You should be able to generate one locally by running make framework in the bindings_ffi folder of libxmtp. That framework gets pushed into libxmtp-swift on every commit to main in libxmtp. The .dylib file is part of that framework.

@neekolas The terminology is a pretty confusing here. ".xcframework" (or XCFramework bundles) and the ".framework bundle" for bundling dynamic libraries are actually two different things.

Function of XCFrameworks is to specify different libraries targeting different architectures. .xcframeworks can bundle static libraries OR dynamic libraries for the different architectures.

".framework bundles" are a way specifically for packaging dynamic libraries, as opposed to just using a .dylib standalone file.

I think what became confusing about our LibCMTPRust.xcframework is that it is packaging .dylib files with a sibling headers directory, which is in incorrect way to package a ".framework bundle" (See anatomy of a framework bundle). I think part of the reason our .dylib files are not bundled correctly as ".framework bundles" is that when we updated from bundling ".a" static libraries to .dylibs (PR here), we never updated our xcodebuild -create-xcframework command from using -library to using -framework.

If you look at the section Generate the XCFramework bundle linked, it says to use the -library flag specifically for including static library (.a) files.

We could look into bundling dynamic libraries as proper ".framework bundles", but again, I think static libraries might be the preferred option anyway.

neekolas commented 8 months ago

The terminology is a pretty confusing here. ".xcframework" (or XCFramework bundles) and the ".framework bundle" for bundling dynamic libraries are actually two different things.

Thank you @cameronvoell. This was extremely informative. Appreciate you diving into the details here. It sounds like the best course of action is to move back to a staticlib.

Seeing as you have the most context, can I leave it to you and @nakajima to validate the change in xmtp-ios as best you can and make sure it doesn't break anything downstream. And feel free to make any changes to my PR as you two see fit.

Great work here

github-actions[bot] commented 8 months ago

:tada: This issue has been resolved in version 1.25.2 :tada:

The release is available on:

Your semantic-release bot :package::rocket: