🐛 [cloud_firestore] issue with isolate communication.

[henoktadesse]

Bug report

Describe the bug Firestore stops receiving stream when starting another isolate

Steps to reproduce

Steps to reproduce the behavior:

1. open Firestore stream page(from the minimal project linked below) -> stream working(receiving data)
2. Click Play or start audio service/from another isolate
3. open Firestore stream page again -> stream not working/stuck at waiting/receiving no data

Expected behavior

Uninterrupted Firestore stream even background isolate is running.

---

Additional context

• detailed explanation about the bug

• minimal project

---

Flutter doctor

Run flutter doctor and paste the output below:

Click To Expand

``` [✓] Flutter (Channel beta, 1.22.0-12.1.pre, on Mac OS X 10.15.6 19G2021, locale en-ET) [✓] Android toolchain - develop for Android devices (Android SDK version 30.0.2) [✓] Xcode - develop for iOS and macOS (Xcode 12.0) [✓] Chrome - develop for the web [✓] Android Studio (version 4.0) [✓] VS Code (version 1.49.2) [✓] Connected device (3 available) ```

---

Flutter dependencies

Run flutter pub deps -- --style=compact and paste the output below:

Click To Expand

``` Dart SDK 2.10.0-110.3.beta Flutter SDK 1.22.0-12.1.pre audiotest 1.0.0+1 dependencies: - audio_service 0.15.0 [audio_session rxdart flutter_isolate flutter_cache_manager js flutter flutter_web_plugins] - cloud_firestore 0.14.0+2 [flutter meta quiver firebase_core firebase_core_platform_interface cloud_firestore_platform_interface cloud_firestore_web] - cupertino_icons 1.0.0 - firebase_core 0.5.0 [firebase_core_platform_interface flutter quiver meta firebase_core_web] - flutter 0.0.0 [characters collection meta typed_data vector_math sky_engine] - just_audio 0.4.5 [audio_session rxdart path path_provider async uuid flutter flutter_web_plugins] dev dependencies: - flutter_test 0.0.0 [flutter test_api path fake_async clock stack_trace vector_math async boolean_selector characters charcode collection matcher meta source_span stream_channel string_scanner term_glyph typed_data] transitive dependencies: - async 2.5.0-nullsafety [collection] - audio_session 0.0.7 [flutter flutter_web_plugins rxdart] - boolean_selector 2.1.0-nullsafety [source_span string_scanner] - characters 1.1.0-nullsafety.2 - charcode 1.2.0-nullsafety - clock 1.1.0-nullsafety - cloud_firestore_platform_interface 2.0.1 [flutter meta collection firebase_core plugin_platform_interface] - cloud_firestore_web 0.2.0+1 [flutter flutter_web_plugins firebase http_parser meta firebase_core cloud_firestore_platform_interface js] - collection 1.15.0-nullsafety.2 - convert 2.1.1 [charcode typed_data] - crypto 2.1.5 [collection convert typed_data] - fake_async 1.1.0-nullsafety [clock collection] - ffi 0.1.3 - file 5.2.1 [intl meta path] - firebase 7.3.0 [http http_parser js] - firebase_core_platform_interface 2.0.0 [flutter meta plugin_platform_interface quiver] - firebase_core_web 0.2.0 [firebase firebase_core_platform_interface flutter flutter_web_plugins meta js] - flutter_cache_manager 1.4.2 [flutter path_provider uuid http path sqflite pedantic clock file rxdart] - flutter_isolate 1.0.0+14 [flutter uuid] - flutter_web_plugins 0.0.0 [flutter characters collection meta typed_data vector_math] - http 0.12.2 [http_parser path pedantic] - http_parser 3.1.4 [charcode collection source_span string_scanner typed_data] - intl 0.16.1 [path] - js 0.6.2 - matcher 0.12.10-nullsafety [stack_trace] - meta 1.3.0-nullsafety.2 - path 1.8.0-nullsafety - path_provider 1.6.18 [flutter path_provider_platform_interface path_provider_macos path_provider_linux path_provider_windows] - path_provider_linux 0.0.1+2 [path xdg_directories path_provider_platform_interface flutter] - path_provider_macos 0.0.4+4 [flutter] - path_provider_platform_interface 1.0.3 [flutter meta platform plugin_platform_interface] - path_provider_windows 0.0.4+1 [path_provider_platform_interface meta path flutter ffi win32] - pedantic 1.9.2 [meta] - platform 2.2.1 - plugin_platform_interface 1.0.2 [meta] - process 3.0.13 [file intl meta path platform] - quiver 2.1.3 [matcher meta] - rxdart 0.24.1 - sky_engine 0.0.99 - source_span 1.8.0-nullsafety [charcode collection path term_glyph] - sqflite 1.3.1+1 [flutter sqflite_common path] - sqflite_common 1.0.2+1 [synchronized path meta] - stack_trace 1.10.0-nullsafety [path] - stream_channel 2.1.0-nullsafety [async] - string_scanner 1.1.0-nullsafety [charcode source_span] - synchronized 2.2.0+2 - term_glyph 1.2.0-nullsafety - test_api 0.2.19-nullsafety [async boolean_selector collection meta path source_span stack_trace stream_channel string_scanner term_glyph matcher] - typed_data 1.3.0-nullsafety.2 [collection] - uuid 2.2.2 [crypto convert] - vector_math 2.1.0-nullsafety.2 - win32 1.7.3 [ffi] - xdg_directories 0.1.2 [meta path process] ```

---

[Ehesp]

Thanks for the link & repro. Does indeed look like a bug so we'll get that looked into.

[TahaTesser]

Hi @henoktadesse Your code sample isn't running, please provide a runnable code sample

Launching lib/main.dart on RMX2001 in debug mode...
Note: /Users/tahatesser/.pub-cache/hosted/pub.dartlang.org/flutter_isolate-1.0.0+14/android/src/main/java/com/rmawatson/flutterisolate/FlutterIsolatePlugin.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: /Users/tahatesser/.pub-cache/hosted/pub.dartlang.org/flutter_isolate-1.0.0+14/android/src/main/java/com/rmawatson/flutterisolate/FlutterIsolatePlugin.java uses unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: /Users/tahatesser/.pub-cache/hosted/pub.dartlang.org/audio_service-0.15.0/android/src/main/java/com/ryanheise/audioservice/AudioServicePlugin.java uses unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
Note: /Users/tahatesser/.pub-cache/hosted/pub.dartlang.org/just_audio-0.4.5/android/src/main/java/com/ryanheise/just_audio/AudioPlayer.java uses unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':app:checkDebugDuplicateClasses'.
> 1 exception was raised by workers:
  java.lang.RuntimeException: Duplicate class com.google.common.util.concurrent.ListenableFuture found in modules guava-26.0-android.jar (com.google.guava:guava:26.0-android) and listenablefuture-1.0.jar (com.google.guava:listenablefuture:1.0)

  Go to the documentation to learn how to <a href="d.android.com/r/tools/classpath-sync-errors">Fix dependency resolution errors</a>.

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 35s
Exception: Gradle task assembleDebug failed with exit code 1
Exited (sigterm)

Thank you

[henoktadesse]

@TahaTesser thank you for your response. I created another project - audiotest it is working on the device I listed above.

[TahaTesser]

Hi @henoktadesse I am seeing the same issue but adding guava and multidex, solve that problem

dependencies {
    def multidex_version = "2.0.1"
    implementation "androidx.multidex:multidex:$multidex_version"
    implementation "org.jetbrains.kotlin:kotlin-stdlib-jdk7:$kotlin_version"
    implementation 'com.google.guava:guava:27.0.1-android'
}
apply plugin: 'com.google.gms.google-services'

However, clicking Audioplayer button does nothing for me

Please try your code project or simplify code sample

[henoktadesse]

That may have happened because I only configured the plugins on iOS since the issue occurred on iOS platform only. The Android side of the code is working perfectly.

[ryanheise]

@henoktadesse , you will be able to reproduce the same issue in a smaller example by using flutter_isolate. This is the smallest package that lets you create a second FlutterEngine which is necessary to reproduce the bug.

Try this in your example:

    // in some button...
      onPressed: () {
        FlutterIsolate.spawn(_isolateEntrypoint, "foo");
      }

// A "top level" function (i.e. not inside a class)
_isolateEntrypoint(String foo) {
    // This alone should be enough to reproduce the bug.
    WidgetsFlutterBinding.ensureInitialized();
    // Note: Now this FlutterEngine will be able to communicate with cloud_firestore
    // but the main FlutterEngine won't. In practice, depending on the app, an app may
    // want to communicate with cloud_firestore from either engine (or  both). In background apps,
    // more likely from here rather than the main isolate, but again that depends on the app.
}

As soon as you press the button, I would expect cloud_firestore to forget how to communicate with the main FlutterEngine for the reasons I pointed to in the detailed bug explanation.

@Ehesp as for how to fix it, apps that execute Dart code in the background will have more than one FlutterEngine. This includes apps like music players that need to play music in the background, or apps that can be woken up from the background to execute Dart code, e.g. in response to alarms or geolocation events, etc. cloud_firestore will fail in all of these apps because it cannot handle being instantiated more than once due to the singleton.

There are two solutions:

1. Most plugins can address this issue by getting rid of the singleton and just allowing the plugin to be instantiated multiple times "normally", each instance maintaining its own state (channel, _listeners, _transactions). It should just work. This is fine if the plugin only needs to deliver events to the Dart code that subscribed to those events.

2. Some plugins may want to broadcast the same events globally to all engines. In this case, the plugin needs to maintain a list of instances in a global variable and each time an event needs to be broadcast, it needs to iterate over each plugin instance and call invokeMethod using the method channel in that plugin instance. For an example of this approach, see here.

[henoktadesse]

@Salakar @TahaTesser @Ehesp Any progress on this ?

[Ehesp]

Once messaging is out we'll look into it

[henoktadesse]

Thank you, We will be waiting!

[LucianoFerreira]

Any feedback, I have this problem too.

TKS

[Chess-er]

Facing same problem, @Ehesp thanks for your response, we will appreciate it if you gave us an estimation of when this could probably be fixed to adjust accordingly, seems like it's gonna take a while. Thanks :)

[Ehesp]

https://github.com/FirebaseExtended/flutterfire/pull/4209 This PR addresses a particular issue with isolate communication, however I'm not sure if fixes this specific issue.

[Chess-er]

@Ehesp I tried it, the way it fails somehow changed but didn't fix the problem.

[peacerevolution]

I have the problem too. I needed to switch to AWS but would love to come back to Firestore once the problem is gone.

[Chess-er]

@peacerevolution you are lucky enough to switch, unlike our case that everything is built on firestore and we are just waiting until it's solved, time unknown when

[sc4v3ng3r]

I did a quick fix to this available here.

What this fix does is basically avoiding the ios only plugin singleton instance to be attached to the new created flutter engine if the plugin is already attached to one. Also it's avoiding the plugin cleanup when a secondary flutter engine is killed.

Bottom line if you're planning call the firestore plugin functions only from the main app isolate it will work for you like a charm. But if you want to call firestore functions from the main and/or secondary isolates... hum... the plugin method calls and events will only respond to the app main isolate.

Feel free to fork, update and do your own hacking.

You can try play with it adding the following into your app pubspec.yaml file:

 cloud_firestore:
    git:
      url: https://github.com/sc4v3ng3r/flutterfire.git
      ref: voicewebapp_fix
      path: packages/cloud_firestore/cloud_firestore

Hope it helps.

[henoktadesse]

@sc4v3ng3r it helped so much, thanks.

[rmill777]

I did a quick fix to this available here.

What this fix does is basically avoiding the ios only plugin singleton instance to be attached to the new created flutter engine if the plugin is already attached to one. Also it's avoiding the plugin cleanup when a secondary flutter engine is killed.

Bottom line if you're planning call the firestore plugin functions only from the main app isolate it will work for you like a charm. But if you want to call firestore functions from the main and/or secondary isolates... hum... the plugin method calls and events will only respond to the app main isolate.

Feel free to fork, update and do your own hacking.

You can try play with it adding the following into your app pubspec.yaml file:

 cloud_firestore:
    git:
      url: https://github.com/sc4v3ng3r/flutterfire.git
      ref: voicewebapp_fix
      path: packages/cloud_firestore/cloud_firestore

Hope it helps.

Did not solve the issue in my case.

[ened]

Hi @Chess-er, @rmill777, @sc4v3ng3r could you please give the PR another try and report detail issues here or on the PR itself?

In our projects, we need the main isolate AND the background isolate(s) to have full access to the firestore; which is also how we tested this PR.

Add this to your pubspec.yaml and please submit logs or a minimal project if you find issues:

dependency_overrides:
  cloud_firestore:
    git:
      url: https://github.com/ened/flutterfire.git
      ref: fix/4108-cloud-firestore
      path: packages/cloud_firestore/cloud_firestore
  cloud_firestore_platform_interface:
    git:
      url: https://github.com/ened/flutterfire.git
      ref: fix/4108-cloud-firestore
      path: packages/cloud_firestore/cloud_firestore_platform_interface

[Chess-er]

@ened Thanks very much I used your PR and its working without any problems.

[pschuegr]

Confirmed as working on my side as well - thanks @ened!