any version 10.17.0 or higher: Error (Xcode): Undefined symbol

AliKales commented 10 months ago

After updating cloud_firestore to 4.13.0 and 10.16.0 to 10.17.0 and error start to show up. Please check below

flutter run -v

i tried flutter clean, removing Pods, podfile.lock PodFile then pod install --repo-update

my Podfile

# Uncomment this line to define a global platform for your project
platform :ios, '11.0'

# CocoaPods analytics sends network stats synchronously affecting flutter build latency.
ENV['COCOAPODS_DISABLE_STATS'] = 'true'

project 'Runner', {
  'Debug' => :debug,
  'Profile' => :release,
  'Release' => :release,
}

def flutter_root
  generated_xcode_build_settings_path = File.expand_path(File.join('..', 'Flutter', 'Generated.xcconfig'), __FILE__)
  unless File.exist?(generated_xcode_build_settings_path)
    raise "#{generated_xcode_build_settings_path} must exist. If you're running pod install manually, make sure flutter pub get is executed first"
  end

  File.foreach(generated_xcode_build_settings_path) do |line|
    matches = line.match(/FLUTTER_ROOT\=(.*)/)
    return matches[1].strip if matches
  end
  raise "FLUTTER_ROOT not found in #{generated_xcode_build_settings_path}. Try deleting Generated.xcconfig, then run flutter pub get"
end

require File.expand_path(File.join('packages', 'flutter_tools', 'bin', 'podhelper'), flutter_root)

flutter_ios_podfile_setup

target 'Runner' do
  use_frameworks!
  use_modular_headers!

  flutter_install_all_ios_pods File.dirname(File.realpath(__FILE__))

  pod 'FirebaseFirestore', :git => 'https://github.com/invertase/firestore-ios-sdk-frameworks.git', :tag => '10.17.0'
end

post_install do |installer|
  installer.pods_project.targets.each do |target|
    flutter_additional_ios_build_settings(target)
  end
end

flutter doctor -v


[✓] Flutter (Channel stable, 3.13.6, on macOS 13.4 22F66 darwin-arm64, locale tr-TR)
    • Flutter version 3.13.6 on channel stable at /Users/kales/Documents/code/flutter/flutter
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision ead455963c (7 weeks ago), 2023-09-26 18:28:17 -0700
    • Engine revision a794cf2681
    • Dart version 3.1.3
    • DevTools version 2.25.0

[✓] Android toolchain - develop for Android devices (Android SDK version 33.0.1)
    • Android SDK at /Users/kales/Library/Android/sdk
    • Platform android-33, build-tools 33.0.1
    • Java binary at: /Applications/Android Studio.app/Contents/jbr/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 11.0.15+0-b2043.56-8887301)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 14.2)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Build 14C18
    • CocoaPods version 1.11.3

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 2022.1)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 11.0.15+0-b2043.56-8887301)

[✓] VS Code (version 1.84.2)
    • VS Code at /Users/kales/Downloads/Visual Studio Code.app/Contents
    • Flutter extension version 3.76.0

[✓] Connected device (4 available)
    • sdk gphone64 arm64 (mobile) • emulator-5554                        • android-arm64  • Android
      13 (API 33) (emulator)
    • iPhone 14 Pro (mobile)      • 418D3F60-17D4-487D-AEB2-21D42173165D • ios            •
      com.apple.CoreSimulator.SimRuntime.iOS-16-2 (simulator)
    • macOS (desktop)             • macos                                • darwin-arm64   • macOS
      13.4 22F66 darwin-arm64
    • Chrome (web)                • chrome                               • web-javascript • Google
      Chrome 119.0.6045.123

[✓] Network resources
    • All expected network resources are available.

• No issues found!

Long story short, after updating to 10.17.0 it doesnt want to work

lukepighetti commented 9 months ago

Also seeing this issue. Broke our builds on iOS

AliKales commented 9 months ago

Also seeing this issue. Broke our builds on iOS

Yeah. I tried everything but it just won't work. Do we need to wait for an Update or will you tell me if you find a solution? Thanks

lukepighetti commented 9 months ago

My solution has been to downgrade and pin

mikehardy commented 9 months ago

I will look at this as I have time but that may be a while. This solution (where we take the upstream .zip file, unpack it, rip out the compiled firestore, and pack that up as a pod) is inherently fragile, and it is in a general sense just a performance optimization

So, if a performance optimization isn't working, the first thing you should do is disable it at least get things working, yes? So I would do that before I advise for a downgrade where you will now be stuck on older versions with other unrelated bugs that are already fixed.

Second thing you can do is look for other optimizations. This is a build time optimization, trying to optimize away compile time. There is another path forward here that is much more robust. much much more robust: ccache

My best advice is to stop using this fragile optimization and use the robust ccache optimization to speed compile times. It is what I actually do in CI and daily development, despite being a maintainer here!

This is how you do it (ignore that it is react-native, the instructions will work for Flutter too) https://reactnative.dev/docs/build-speed#use-a-compiler-cache

Now you have the standard upstream Pods (much better! less fragile) and you have a general compiler cache that will speed up compilation of your entire app. It's a win/win

AliKales commented 9 months ago

OK, thank you for your advises. For now it is not urgent for me to build for IOS i just wanted to let you know about the problem. I prefer to wait your solution for version 10.17.0. I'm not closing the issue for now. Thanks!

mikehardy commented 9 months ago

To be clear: I would not wait for a solution here - I have not even investigated it for feasibility - it may be that it is simply not feasible to perform this sort of build optimization anymore. And it will depend on my time, and I'm pretty overbooked. All of that is to say that with apologies, it may not happen or may take quite a long time. I do really advise you to check out ccache, it's a fabulous system.

brew install ccache on macos (assuming you have the brew package system?) - set up the ccache.conf to handle iOS builds with the right flags, one little thing added to your Podfile and things are really really fast. No waiting for me

AliKales commented 9 months ago

ok thank you for your time! i'm closing this.

russellwheatley commented 9 months ago

Hey @mikehardy I've spent quite a while looking into this issue. Unfortunately, I have not been able to solve it. I thought I'd report my findings:

The framework ships with FirebaseSharedSwift binary. It works fine until your project pulls in another dependency that has a transitive dependency on FirebaseSharedSwift.

For instance, this works fine:

dependencies:
  cloud_firestore: ^4.13.1
  firebase_core: ^2.22.0
  flutter:
    sdk: flutter

This produces the error:

dependencies:
  cloud_firestore: ^4.13.1
  firebase_core: ^2.22.0
  firebase_database: ^10.3.4
  flutter:
    sdk: flutter

This is because firebase_database creates a transitive dependency on FirebaseSharedSwift. Snippet from Podfile.lock:

  - FirebaseDatabase (10.18.0):  
    - FirebaseAppCheckInterop (~> 10.17)  
    - FirebaseCore (~> 10.0)  
    - FirebaseSharedSwift (~> 10.0)  
    - leveldb-library (~> 1.22)

Here is the first line of the erroneous build log from Xcode:

ld: Undefined symbols:
  enum case for FirebaseSharedSwift.FirebaseDataDecoder.DataDecodingStrategy.blob(FirebaseSharedSwift.FirebaseDataDecoder.DataDec

Full build log of exception

The exception points to missing objects as opposed to missing header files.The architecture is correct (I checked the framework binary architecture by running lipo -info FirebaseSharedSwift) which produced the correct - Non-fat file: FirebaseSharedSwift is architecture: arm64.

Edits to the `FirebaseFirestore.podspec` that failed

Removed FirebaseSharedSwift framework from the distribution if using firebase_database as a dependency, and use the FirebaseSharedSwift transitive dependency. Still produced the same exception.

Removed FirebaseSharedSwift binary from the distribution and made it a dependency instead. Still produced the same exception.
Updated some of the settings to match what is set in the podspec of FirebaseFirestore on the firebase-ios-sdk. Still produced the same exception.

Really not sure how to fix this problem. Have you experienced this on RNFB? or have any thoughts on a solution? 🤔

paulb777 commented 9 months ago

I'm not sure if this is helpful or not, but we solved a similar problem with the Swift Package Manager distribution and its binary Firestore target by wrapping Firestore in an "Internal" target and having an outer Firestore target depend on both FirestoreInternal and FirebaseSharedSwift - https://github.com/firebase/firebase-ios-sdk/blob/master/Package.swift#L1503

russellwheatley commented 9 months ago

Thanks for the heads up, @paulb777 👍 . We had a try implementing the above and still couldn't get beyond the exception. Do you mind taking a look at these changes to see if there is something we could do to fix it?

https://github.com/invertase/firestore-ios-sdk-frameworks/compare/main...test-fixing10#diff-722406d90405bff17f7e918babfa09df6eb0e10808c09f55b7349f15d1996c99R35

paulb777 commented 9 months ago

I think that FirebaseFirestoreInternal and FirebaseSharedSwift should both be plain dependencies of the Invertase FirebaseFirestore.podspec without the special handling. cc: @ncooke3.

ncooke3 commented 9 months ago

I don't have an immediate answer, but I'll add some more context to what changes happened in 10.17.0.

I'm not sure if this is helpful or not, but we solved a similar problem with the Swift Package Manager distribution and its binary Firestore target by wrapping Firestore in an "Internal" target and having an outer Firestore target depend on both FirestoreInternal and FirebaseSharedSwift - https://github.com/firebase/firebase-ios-sdk/blob/master/Package.swift#L1503

Adding on to the above comment, the outer Firestore target is a source target and so is the FirebaseSharedSwift target. So the dependency chain for Firestore's SwiftPM distribution looks like below. I'm excluding some other transitive dependencies like FirebaseCore, gRPC, etc.

- Firestore (source target composed of Swift sources):
  - FirestoreInternal (binary target)
  - SharedSwift (source target composed of Swift sources)

FirestoreInternal is composed of the sources that used to live in Firestore. And Firestore is now composed of the sources that used to live in FirestoreSwift. So FirestoreInternal is the big framework that we distributed as a binary to optimize build time.

The framework ships with FirebaseSharedSwift binary. It works fine until your project pulls in another dependency that has a transitive dependency on FirebaseSharedSwift.

This is the reason that SharedSwift remained a source target in the SwiftPM distribution. If we made it a binary target, then other top-level source targets that depend on SharedSwift (like Database) would not build since I had issues getting a source Swift module to depends on a binary Swift module. I think this is related to the problem we are seeing here.

Potential solution: Invertase's FirestoreFirestore.podspec wraps Firebase's FirestoreFirestore.podspec, but swaps out the Firebase FirestoreFirestoreInternal dependency with Invertase's. This would mirror what is done for SwiftPM in the Firebase repo.

@russellwheatley, in the issue's description, the undefined symbols were mangled:

Undefined symbol:
           _$s19FirebaseSharedSwift0A11DataDecoderC0D16DecodingStrategyO4blobyA2EmFWC

In the error logs you included, I noticed they are not:

ld: Undefined symbols:
  enum case for FirebaseSharedSwift.FirebaseDataDecoder.DataDecodingStrategy.blob(FirebaseSharedSwift.FirebaseDataDecoder.DataDec

Was this a result of you adding the FirebaseSharedSwift.xcframework in your branch?

I think that FirebaseFirestoreInternal and FirebaseSharedSwift should both be plain dependencies of the Invertase FirebaseFirestore.podspec without the special handling. cc: @ncooke3.

+1 for FirebaseFirestoreInternal always being there. There should be no other case it already may exist in the project (like with levelDB).

mikehardy commented 9 months ago

Commenting really briefly since I was tagged and asked a question but I have little to offer in comparison to the big knowledge drop from Nick Cooke!

Have you experienced this on RNFB? or have any thoughts on a solution?

I have not experienced this on RNFB - yet - because I converted RNFB to use ccache as it speeds up compile of the entire app / all firebase items vs just firestore as this optimization does. I typically only test react-native-firebase with this repo in response to bugs like this and I hadn't had time to do so yet.

Hopefully with the info from Paul + Nick things may be rearranged here so it works again. In react-native-firebase there is just a single line I think that is currently commented out and then it uses this podspec as a dep (possibly by git branch ref) vs the Firestore normal source podspec

josiahsrc commented 9 months ago

Installing ccache worked for my purposes. Thanks @mikehardy for the recommendation! It got my build without invertase down from 450 seconds to 50 seconds.

In case it's useful for anyone else, here's what I did on a mac M1, mostly taken from this article

Remove from podfile:

# remove this
pod 'FirebaseFirestore', :git => 'https://github.com/invertase/firestore-ios-sdk-frameworks.git', :tag => '10.12.0'

Install ccache

brew install ccache

Export these env vars, or else add them to ~/.zshrc

export CCACHE_SLOPPINESS=clang_index_store,file_stat_matches,include_file_ctime,include_file_mtime,ivfsoverlay,pch_defines,modules,system_headers,time_macros
export CCACHE_FILECLONE=true
export CCACHE_DEPEND=true
export CCACHE_INODECACHE=true

export PATH="/opt/homebrew/opt/ccache/libexec:$PATH"

Add ccache config to podfile

post_install do |installer|
  installer.pods_project.targets.each do |target|
    ...
    target.build_configurations.each do |config|

      # Add these
      config.build_settings["CC"] = "clang"
      config.build_settings["LD"] = "clang"
      config.build_settings["CXX"] = "clang++"
      config.build_settings["LDPLUSPLUS"] = "clang++"

      ...
    end
  end
end

Then try building. First build will be slow, subsequent should be faster.

mikehardy commented 9 months ago

Fantastic @josiahsrc - for anyone on an Intel Mac or using GitHub actions only difference is the Homebrew path varies from /opt/homebrew to /usr/local/homebrew if I recall correctly, but the incredible speedup across whole build is real, and in most CI environments it's trivial to cache the ccache directory across runs. React-native-firebase repository iOS e2e workflow has a good example ( caching section / config section )

YDA93 commented 9 months ago

@josiahsrc i carefully followed your comment and the links:

and for some reason ccache is not caching:

Cacheable calls:       0 / 2605 ( 0.00%)
  Hits:                0
    Direct:            0
    Preprocessed:      0
  Misses:              0
Uncacheable calls:  2605 / 2605 (100.0%)
Local storage:
  Cache size (GiB):  0.0 /  5.0 ( 0.00%)

Can you run ccache -s and show us the output?

josiahsrc commented 9 months ago

@YDA93 I didn't check the ccache output before, but I'm getting similar results to what you have there:

Cacheable calls:    2046 / 2046 (100.0%)
  Hits:                0 / 2046 ( 0.00%)
    Direct:            0
    Preprocessed:      0
  Misses:           2046 / 2046 (100.0%)
Local storage:
  Cache size (GiB):  0.9 /  5.0 (18.29%)
  Hits:                0 / 2046 ( 0.00%)
  Misses:           2046 / 2046 (100.0%)

What's interesting though, is that I just ran the build twice. The first build was 264.1s, but the second was 36.8s. Empirically, it looks like something is being cached, but I'm not 100% certain

YDA93 commented 9 months ago

@josiahsrc I do get similar results to your build time without ccache:

First attempt: 226.3s
Second attempt: 41.3s

And I believe if you uninstall ccache completely you would also get the results you mentioned earlier.

I'm not sure which cache mechanism is responsible of this, is it a new update to Flutter or Xcode or both.

josiahsrc commented 9 months ago

🤷‍♂️ Who knows. But if it's a shorter build time, I'll take it! 😄

YDA93 commented 9 months ago

@russellwheatley @lukepighetti @AliKales Can you share why we need to use invertase when we already get similar build time to what it's promoting for? Are we expecting less build time? Can you share your build time for the first and second attempts?

mikehardy commented 9 months ago

So, there can be a few layers of caching going on - it appears the results above were resulting in cache misses from the ccache system, so that has for some reason not been installed correctly in one instance (the one with zero cacheable calls) but has been installed correctly in the second instance (the one with lots of cacheable calls).

Then on a second build I believe Xcode and the xcodebuild system determined it simply did not need to rebuild things.

Even if ccache was not there, Xcode can pull unchanged items from it's own cache in ~/Library/Developer/Xcode/DerivedData/<Your app name>-<some semi-random string>

@YDA93 in answer to your direct question: because in CI environments, which do not have a lot of compute power typically and are always effectively a first time build, firestore compilation (which transitively compiles gRPC, abseil etc etc) was taking 10-15 minutes for people.

On a local machine it is not quite as important to optimize in this way, however if you do the optimization for CI, then you should run it locally as well so you don't have build breaks in CI that you don't reproduce locally.

All that said, ccache should work and does work for me. You can turn on logging for it with a config change to see why things are not cacheable or why there are misses.

This is what things look like when you have ccache running for a long time with lots of projects - all iOS stuff though and mostly firebase-related - it really does work:


mike@isabela:~ % ccache -s
Cacheable calls:   182296 / 183494 (99.35%)
  Hits:             84747 / 182296 (46.49%)
    Direct:         84735 /  84747 (99.99%)
    Preprocessed:      12 /  84747 ( 0.01%)
  Misses:           97549 / 182296 (53.51%)
Uncacheable calls:   1198 / 183494 ( 0.65%)
Local storage:
  Cache size (GB):   36.1 /   40.0 (90.17%)
  Cleanups:           190
  Hits:             84747 / 182296 (46.49%)
  Misses:           97549 / 182296 (53.51%)

YDA93 commented 9 months ago

@mikehardy Thank you so much for your detailed and insightful response! Now I am interested in using ccache. Except I still have no hits using local macOS m1:

Cacheable calls:     2605 / 18235 (14.29%)
  Hits:                 0 /  2605 ( 0.00%)
    Direct:             0
    Preprocessed:       0
  Misses:            2605 /  2605 (100.0%)
Uncacheable calls:  15630 / 18235 (85.71%)
Local storage:
  Cache size (GiB):   0.6 /   5.0 (12.55%)
  Hits:                 0 /  2605 ( 0.00%)
  Misses:            2605 /  2605 (100.0%)

But will definitely use it in CI and hopefully will get this settled. Thank you!

wer-mathurin commented 9 months ago

The cache mechanism is cool, but not sure this will work on codemagic(CI), because each build is like having a new build machine....so like a first build for ccache....or maybe I'm missing something....or need to use a distributed cache?

mikehardy commented 9 months ago

@wer-mathurin as far as I understand, all cloud CI providers use ephemeral build machines such that each CI run is like a new build machine. But for reasons of speed they also all provide a way to cache things between runs.

codemagic does as well from a quick look at their docs: https://docs.codemagic.io/knowledge-codemagic/caching/

So you would want to cache $HOME/.ccache (that's the default at least) I suppose along with whatever else you are caching?

jkgatt commented 9 months ago

The above is also an issue on 10.18.0.

mikehardy commented 9 months ago

@jkgatt and anyone else - this will be an issue on any firebase-ios-sdk >= 10.17.0 - it is not going to self-resolve until there is some fix here - I will update the issue title.

jkgatt commented 9 months ago

Okay thanks @mikehardy!

Has anyone tried any of the fixes in PRs?

mikehardy commented 9 months ago

The comment from Nick Cooke here: https://github.com/invertase/firestore-ios-sdk-frameworks/issues/79#issuecomment-1817042949

...is the current "state of the knowledge". I do not think there has been further work on exploring those thoughts since it has been posted, but @russellwheatley may have more info there? I personally have not done any exploration

kNoAPP commented 9 months ago

Fantastic @josiahsrc - for anyone on an Intel Mac or using GitHub actions only difference is the Homebrew path varies from /opt/homebrew to /usr/local/homebrew if I recall correctly, but the incredible speedup across whole build is real, and in most CI environments it's trivial to cache the ccache directory across runs. React-native-firebase repository iOS e2e workflow has a good example ( caching section / config section )

Since the time these links were posted, the e2e build code has changed on the referenced branch. The highlights on these links now highlight the wrong sections of code. Please see these updated links for setting up GitHub Actions ( caching section / config section )

ben-milanko commented 9 months ago

Has anyone else ran into this issue while trying to migrate?

Error (Xcode): Framework 'BoringSSL-GRPC' not found

Running into this after removing the invertase precompiled pod and can't seem to find a working fix for it.

josiahsrc commented 9 months ago

@ben-milanko That looks like a pods issue where some old artifacts from a previous build are beingcached. If you're using flutter, try this before building again:

cd ios
rm -rf Pods
rm -rf .symlinks
flutter clean
flutter pub get

You may also want to delete and regenerate Podfile.lock

ben-milanko commented 9 months ago

Thanks for the suggestions @josiahsrc , I've tried these and a few other things but can't seem to get it working. I've pinned the cloud_firestore release so we can keep using the invertase pod for the moment.

Update: Turns out the issue had to do with the Other linker flags in build settings. Deleting the 3 instances of OTHER_LDFLAGS in project.pbxproj resolved it (I didn't find out which framework specifically was causing the issue).

lukemmtt commented 9 months ago

@ben-milanko You're saying you did indeed get ccache working? Can you share details about your setup and process to get there? I tried for quite a while with the tips above, to no avail.

mikehardy commented 9 months ago

I still believe ccache is an overrall superior solution than this pod and I encourage people to try it, though we will try to fix this pod

However discussion of ccache is really unrelated to this issue and will clutter up the issue making it hard to track real progress here.

All ccache discussion should go in this new discussion I just started in the hope it helps people: https://github.com/invertase/firestore-ios-sdk-frameworks/discussions/82

(I really hope that helps - ccache can be a little touchy and accumulating ccache discussion in one spot has the potential to really help people over time)

jims57 commented 9 months ago

OK, thank you for your advises. For now it is not urgent for me to build for IOS i just wanted to let you know about the problem. I prefer to wait your solution for version 10.17.0. I'm not closing the issue for now. Thanks!

I have done some experience about this issue, I am using Firestore 10.18.0, and Cocoapod 1.14.3. These are what my findings:

a. If you want to only add cloud_firestore: ^4.13.3 in pubspec.yaml, and you have to change pod FirebaseFirestore to pod FirebaseFirestore/WithLeveldb in my case. This is mentioned in this thread. I built sucessfully after I changed it, here is my example:

change this code in ios/Podfile

pod 'FirebaseFirestore', :git => 'https://github.com/invertase/firestore-ios-sdk-frameworks.git', :tag => '10.18.0'

to this code in ios/Podfile

pod 'FirebaseFirestore/WithLeveldb', :git => 'https://github.com/invertase/firestore-ios-sdk-frameworks.git', :tag => '10.18.0'

b. But it fails to build iOS again, saying Error (Xcode): Undefined symbol if I also add Realtime DB package(firebase_database: ^10.3.6) in pubspec.yaml, even I did the solution mentoned in item a here. I found this version cannot include both of them, you have to either use Firestore or Realtime DB in pubspec.yaml. And it is also only related with Realtime DB, becuase, I found Firestore can work alongside other Firebase packages, such as firebase_storage, firebase_auth etc.

lukepighetti commented 8 months ago

Does anyone have a list of dependency versions we can use to pin this to 10.16.0? Something shifted and our builds are broken and the 6 minute builds without these pre-bundled sdk frameworks are non-viable

btanev commented 8 months ago

Here is a list of firebase dependencies we use in one of our projects. Pinned for iOS SDK 10.16.0, released up to Oct 31, 2023:

  # firebase
  cloud_firestore: 4.12.2 # https://pub.dev/packages/cloud_firestore
  cloud_functions: 4.5.3 # https://pub.dev/packages/cloud_functionsomzomx
  firebase_auth: 4.12.1 # https://pub.dev/packages/firebase_auth
  firebase_core: 2.21.0 # https://pub.dev/packages/firebase_core
  firebase_crashlytics: 3.4.3 # https://pub.dev/packages/firebase_crashlytics
  firebase_dynamic_links: 5.4.3 # https://pub.dev/packages/firebase_dynamic_links
  firebase_storage: 11.4.1 # https://pub.dev/packages/firebase_storage

tenhobi commented 8 months ago

TL;DR other packages:

firebase_analytics: 10.6.3 # https://pub.dev/packages/firebase_analytics
firebase_messaging: 14.7.3 # https://pub.dev/packages/firebase_messaging
firebase_performance: 0.9.3+3 # https://pub.dev/packages/firebase_performance
firebase_remote_config: 4.3.3 # https://pub.dev/packages/firebase_remote_config

When you pin your dependencies correctly, but it still does not work, read this:

In our project, we use

firebase_analytics
firebase_core
firebase_crashlytics
firebase_messaging
firebase_performance

the other packages not mentioned by @btanev can be pinned as

firebase_analytics: 10.6.3
firebase_messaging: 14.7.3
firebase_performance: 0.9.3+3

And... that did not work. After time I discovered that in Podfile.lock there still is a dependency on FirebaseRemoteConfig from FirebasePerformance. In the lock it was like this:

  - FirebaseRemoteConfig (10.19.0):
    - FirebaseABTesting (~> 10.0)
    - FirebaseCore (~> 10.0)
    - FirebaseInstallations (~> 10.0)
    - FirebaseSharedSwift (~> 10.0) # you DO NOT want this here
    - GoogleUtilities/Environment (~> 7.8)
    - "GoogleUtilities/NSData+zlib (~> 7.8)"

meaning that if anyone of you have any package with FirebaseSharedSwift, you must replace it somehow. Therefore I added firebase_remote_config to our Pubspec, even when we do not use it. So we force Podfile to use it in 1.16.0:

firebase_remote_config: 4.3.3

That worked and Podfile.lock now has this without that dependency

  - FirebaseRemoteConfig (10.16.0):
    - FirebaseABTesting (~> 10.0)
    - FirebaseCore (~> 10.0)
    - FirebaseInstallations (~> 10.0)
    - GoogleUtilities/Environment (~> 7.8)
    - "GoogleUtilities/NSData+zlib (~> 7.8)"

Then just don't forget to remove every cache etc. and it worked for us.

cd ios
rm -rf .symlinks
rm -rf Pods
rm -rf Podfile.lock
pod cache clean --all

cd ..
flutter clean

Hope this helps someone.

scopendo commented 8 months ago

@btanev / @tenhobi – when you refer to pinning the Firebase iOS SDK to 10.16.0 do you mean adding the following edit to Podfile?

target 'Runner' do
  use_frameworks!
  use_modular_headers!

  flutter_install_all_ios_pods File.dirname(File.realpath(__FILE__))

  # >>>>> BEGIN workaround for symbol not found error
  pod 'FirebaseCore', :git => 'https://github.com/firebase/firebase-ios-sdk.git', :tag => '10.16.0'
  # <<<<< END workaround for symbol not found error
end

tenhobi commented 8 months ago

@scopendo ~~Yes~~ No, we use FirebaseFirestore like this:

target 'Runner' do
  use_frameworks!
  use_modular_headers!

  flutter_install_all_ios_pods File.dirname(File.realpath(__FILE__))

  # Use precompiled Firebase SDK for faster builds
  pod 'FirebaseFirestore', :git => 'https://github.com/invertase/firestore-ios-sdk-frameworks.git', :tag => '10.16.0'
end

jkgatt commented 8 months ago

I still believe ccache is an overrall superior solution than this pod and I encourage people to try it, though we will try to fix this pod

However discussion of ccache is really unrelated to this issue and will clutter up the issue making it hard to track real progress here.

All ccache discussion should go in this new discussion I just started in the hope it helps people: #82

(I really hope that helps - ccache can be a little touchy and accumulating ccache discussion in one spot has the potential to really help people over time)

I've moved onto this and honestly its much better than depending on the ios sdk framework.

Everyone should do this too as you're going to just be stuck with all the old versions of any firebase component.

costacoz commented 8 months ago

I desperately need to use this pre-built dependency. I have kernel panic while trying to build for iOS on macbook M1 Pro with cloud_firestore dependency. #12090 issue It happens on "Running Xcode build..." step. So the only way I overcame this problem is to use this pre-built pod.

So @mikehardy , for me using ccache is not an option, because I can't even get to the point of putting this build into the cache. I hope for your understanding.

mikehardy commented 8 months ago

A kernel panic indicates something deeply wrong with the hardware or operating system and the prebuilt dep would just be covering it. I'd be afraid to use anything built on a machine like that as it may be corrupt silently. You need to get service for that machine from Apple

russellwheatley commented 8 months ago

Hey everyone, I've been trying to get this to work without much success. To clear up any misunderstanding, we want to continue to support this repo as we believe it is an easier solution to using ccache for developers who do not have much experience with native, and this issue is a big impediment to the adoption of Firebase on Flutter & React Native for Apple platforms.

@ncooke3 - following on from this comment: https://github.com/invertase/firestore-ios-sdk-frameworks/issues/79#issuecomment-1817042949

I've tried to follow the Swift Package Manager strategy that you outlined.

Making `FirebaseSharedSwift` a source dependency

Branch: https://github.com/invertase/firestore-ios-sdk-frameworks/pull/86/files

We removed the precompiled frameworks from the base subspec: https://github.com/invertase/firestore-ios-sdk-frameworks/pull/86/files#diff-722406d90405bff17f7e918babfa09df6eb0e10808c09f55b7349f15d1996c99R51-R59

and made them plain dependencies on source: https://github.com/invertase/firestore-ios-sdk-frameworks/pull/86/files#diff-722406d90405bff17f7e918babfa09df6eb0e10808c09f55b7349f15d1996c99R67-R70

We also followed the advice for creating a wrapper around FirebaseFirestoreInternalWrapper https://github.com/invertase/firestore-ios-sdk-frameworks/pull/86/files#diff-722406d90405bff17f7e918babfa09df6eb0e10808c09f55b7349f15d1996c99R38-R44

and added it as a dependency for base subspec here: https://github.com/invertase/firestore-ios-sdk-frameworks/pull/86/files#diff-722406d90405bff17f7e918babfa09df6eb0e10808c09f55b7349f15d1996c99R72

This resulted in the usual error in the Xcode build logs:

This error occurred when depending on firestore only.

Podfile.lock

```yml PODS: - cloud_firestore (4.14.0): - Firebase/Firestore (= 10.19.0) - firebase_core - Flutter - nanopb (< 2.30910.0, >= 2.30908.0) - Firebase/CoreOnly (10.19.0): - FirebaseCore (= 10.19.0) - Firebase/Firestore (10.19.0): - Firebase/CoreOnly - FirebaseFirestore (~> 10.19.0) - firebase_core (2.24.2): - Firebase/CoreOnly (= 10.19.0) - Flutter - FirebaseAppCheckInterop (10.19.0) - FirebaseCore (10.19.0): - FirebaseCoreInternal (~> 10.0) - GoogleUtilities/Environment (~> 7.12) - GoogleUtilities/Logger (~> 7.12) - FirebaseCoreExtension (10.19.0): - FirebaseCore (~> 10.0) - FirebaseCoreInternal (10.19.0): - "GoogleUtilities/NSData+zlib (~> 7.8)" - FirebaseFirestore (10.19.0): - FirebaseFirestore/AutodetectLeveldb (= 10.19.0) - FirebaseFirestore/AutodetectLeveldb (10.19.0): - FirebaseFirestore/Base - FirebaseFirestore/WithLeveldb - FirebaseFirestore/Base (10.19.0): - FirebaseAppCheckInterop (~> 10.19.0) - FirebaseCore (~> 10.19.0) - FirebaseCoreExtension (~> 10.19.0) - FirebaseFirestore/FirebaseFirestoreInternalWrapper - FirebaseSharedSwift (~> 10.19.0) - FirebaseFirestore/FirebaseFirestoreInternal (10.19.0) - FirebaseFirestore/FirebaseFirestoreInternalWrapper (10.19.0): - FirebaseFirestore/FirebaseFirestoreInternal - FirebaseFirestore/WithLeveldb (10.19.0): - FirebaseFirestore/Base - FirebaseSharedSwift (10.19.0) - Flutter (1.0.0) - GoogleUtilities/Environment (7.12.0): - PromisesObjC (< 3.0, >= 1.2) - GoogleUtilities/Logger (7.12.0): - GoogleUtilities/Environment - "GoogleUtilities/NSData+zlib (7.12.0)" - integration_test (0.0.1): - Flutter - nanopb (2.30909.1): - nanopb/decode (= 2.30909.1) - nanopb/encode (= 2.30909.1) - nanopb/decode (2.30909.1) - nanopb/encode (2.30909.1) - PromisesObjC (2.3.1) DEPENDENCIES: - cloud_firestore (from `.symlinks/plugins/cloud_firestore/ios`) - firebase_core (from `.symlinks/plugins/firebase_core/ios`) - FirebaseFirestore (from `https://github.com/invertase/firestore-ios-sdk-frameworks.git`, tag `replicate-spm-strategy`) - Flutter (from `Flutter`) - integration_test (from `.symlinks/plugins/integration_test/ios`) SPEC REPOS: trunk: - Firebase - FirebaseAppCheckInterop - FirebaseCore - FirebaseCoreExtension - FirebaseCoreInternal - FirebaseSharedSwift - GoogleUtilities - nanopb - PromisesObjC EXTERNAL SOURCES: cloud_firestore: :path: ".symlinks/plugins/cloud_firestore/ios" firebase_core: :path: ".symlinks/plugins/firebase_core/ios" FirebaseFirestore: :git: https://github.com/invertase/firestore-ios-sdk-frameworks.git :tag: replicate-spm-strategy Flutter: :path: Flutter integration_test: :path: ".symlinks/plugins/integration_test/ios" CHECKOUT OPTIONS: FirebaseFirestore: :git: https://github.com/invertase/firestore-ios-sdk-frameworks.git :tag: replicate-spm-strategy SPEC CHECKSUMS: cloud_firestore: 309f2d9d341bbf3f70d23bfbfede122738aa8570 Firebase: 63ce8ece0d43743dc28eacac0c6867a2d7fd5a9d firebase_core: 69d42e77bcb068f2e6371fab41509009d37a69ea FirebaseAppCheckInterop: 37884781f3e16a1ba47e7ec80a1e805f987788e3 FirebaseCore: dc5c7badf99d47613c52b2e3a57a64cd187f8554 FirebaseCoreExtension: c08d14c7b22e07994e876d837e6f58642f340087 FirebaseCoreInternal: b444828ea7cfd594fca83046b95db98a2be4f290 FirebaseFirestore: 8600ca81df160a6a02f567f6c050cf4a7b57b5e8 FirebaseSharedSwift: f34eeb7d3ea87a34497629b6ca41657beadef76a Flutter: f04841e97a9d0b0a8025694d0796dd46242b2854 GoogleUtilities: 0759d1a57ebb953965c2dfe0ba4c82e95ccc2e34 integration_test: 13825b8a9334a850581300559b8839134b124670 nanopb: d4d75c12cd1316f4a64e3c6963f879ecd4b5e0d5 PromisesObjC: c50d2056b5253dadbd6c2bea79b0674bd5a52fa4 PODFILE CHECKSUM: e74666071ac70ed969a3e74fb190fa4e0d089f45 COCOAPODS: 1.14.3 ```

Making `FirebaseSharedSwift` a part of the precompiled frameworks of base subspec

Branch: https://github.com/invertase/firestore-ios-sdk-frameworks/pull/87/files

This replicates the above except we include FirebaseSharedSwift as a pre-compiled framework with the base subspec:https://github.com/invertase/firestore-ios-sdk-frameworks/pull/87/files#diff-722406d90405bff17f7e918babfa09df6eb0e10808c09f55b7349f15d1996c99R53-R54

and remove the source dependency: https://github.com/invertase/firestore-ios-sdk-frameworks/pull/87/files#diff-722406d90405bff17f7e918babfa09df6eb0e10808c09f55b7349f15d1996c99R70

Now, when we build with just firestore, it succeeds in building. But when we include firebase performance (which depends on FirebaseSharedSwift) the build fails.

Xcode build logs when using firebase performance and firestore dependencies:

Podfile.lock

Could you help us move forward with this? Is there something we can do to match the strategy you employ with SPM that is missing? From what I understand, the FirebaseSharedSwift should be a dependency from source but that fails when we just have firestore as a dependency on the project. I hope the above makes sense, let me know if you need any further clarification.

paulb777 commented 7 months ago

Another approach at solving this might be to do a detailed comparison with the working Swift Package Manager Firestore binary distribution. (See the binaries at https://github.com/firebase/firebase-ios-sdk/blob/main/Package.swift#L1483 as well as grpc and other dependencies in the same file).

ncooke3 commented 7 months ago

Thanks @russellwheatley for the investigation, and my apologies that those paths did not lead to a solution.

Below is Firestore's dependency graph:

graph TD;
    FirebaseFirestore-->FirebaseSharedSwift;
    FirebaseFirestore-->FirebaseFirestoreInternal;
    FirebaseFirestoreInternal-->gRPC;
    FirebaseFirestoreInternal-->abseil;

For CocoaPods users, all of the above nodes are built from source.

As Paul mentioned above, the firebase-ios-sdk has a solution for SwiftPM users where the right-side children are distributed as binaries. In the SwiftPM solution, FirebaseFirestore and FiresbaseSharedSwift remain distributed as source (they are very small in size comparatively).

To mirror this setup, at a high level, I think this repo's FirebaseFirebase.podspec would need to depend on on the firebase-ios-sdk FirebaseFirebase.podspec, but swap out the FirebaseFirebaseInternal pod for an xcframework.

Let me know what you think of the above approach.

russellwheatley commented 7 months ago

@ncooke3 - thanks for the tips! It's proving to be a difficult problem to solve.

As Paul mentioned above, the firebase-ios-sdk has a solution for SwiftPM users where the right-side children are distributed as binaries. In the SwiftPM solution, FirebaseFirestore and FiresbaseSharedSwift remain distributed as source (they are very small in size comparatively).

I tried to follow what you've outlined and the graph you've shared. Here is the branch: https://github.com/invertase/firestore-ios-sdk-frameworks/compare/main...replicate-spm-strategy-18

Steps taken to match the above:

Made firebase-ios-sdk the source: main...replicate-spm-strategy-18#diff-722406d904
Depended on the relevant Pods for FirebaseFirestore: main...replicate-spm-strategy-18#diff-722406d904
Included FirebaseFirestore as source files (i.e. base.source_files = 'Firestore/Swift/Source/**/*.{h,m,swift}') : main...replicate-spm-strategy-18#diff-722406d904
depended on wrapped FirebaseFirestoreInternal as the binary framework (also tried without the wrapping): main...replicate-spm-strategy-18#diff-722406d904

The result is different now. We get this error:

Lexical or Preprocessor Issue (Xcode): 'FirebaseFirestore/FirebaseFirestore.h' file not found
/Users/russellwheatley/projects/flutterfire/packages/cloud_firestore/cloud_firestore/ios/Classes/Private/FLTFirebaseFirestoreExtension.h:3:8

It seems this file is missing: https://github.com/firebase/firebase-ios-sdk/blob/main/Firestore/Swift/FirebaseFirestore.h

I've tried to include it here: main...replicate-spm-strategy-18#diff-722406d904

In one of the many previous iterations, I also had something like this in the base subspec:

base.public_header_files = 'Firestore/Swift/**/*.h'

which also failed to include the header. Not sure if you have any thoughts on what could be stopping it from being included?

You'll have to excuse my ignorance, but I wasn't quite sure what you meant by:

I think this repo's FirebaseFirebase.podspec would need to depend on on the firebase-ios-sdk FirebaseFirebase.podspec

@paulb777

Another approach at solving this might be to do a detailed comparison with the working Swift Package Manager Firestore binary distribution. (See the binaries at firebase/firebase-ios-sdk@main/Package.swift#L1483 as well as grpc and other dependencies in the same file).

I downloaded the binary from that URL and it was the same result.

As a side question, considering you've made FirebaseFirestoreInternal binary a default part of the SPM distribution, I was curious if you were looking to do the same for cocoapods?

russellwheatley commented 7 months ago

An interesting aside, when I navigate to this file in Xcode /Users/russellwheatley/projects/flutterfire/packages/cloud_firestore/cloud_firestore/ios/Classes/Private/FLTFirebaseFirestoreExtension.h:3:8, the error disappears as if it has been resolved.

paulb777 commented 7 months ago

On the 'FirebaseFirestore/FirebaseFirestore.h' file not found issue can you do @import FirebaseFirestore; instead?

We've considered the switching the default CocoaPods distribution to binary, but are hesitant for at least three reasons: 1. There is a lot more CocoaPods legacy than SPM, that we would risk breaking. 2. We lose flexibility. For example, with the SPM binary distribution, there's no way to support both Xcode 14 and Vision OS early adopters. 3. The invertase solution has addressed most of the demand, probably since Xcode caching usually addresses the build time issue for Swift developers.

russellwheatley commented 7 months ago

On the 'FirebaseFirestore/FirebaseFirestore.h' file not found issue can you do @import FirebaseFirestore; instead?

No change I'm afraid.

We've considered the switching the default CocoaPods distribution to binary, but are hesitant for at least three reasons: 1. There is a lot more CocoaPods legacy than SPM, that we would risk breaking. 2. We lose flexibility. For example, with the SPM binary distribution, there's no way to support both Xcode 14 and Vision OS early adopters. 3. The invertase solution has addressed most of the demand, probably since Xcode caching usually addresses the build time issue for Swift developers.

Thanks for the clarification 👍

invertase / firestore-ios-sdk-frameworks