🐛 Firestore Snapshot Listeners Attaching Before ID Token Refresh, Resulting in Stale Data

[ramsayamarin]

Bug report

I'm experiencing a significant issue with my Flutter application that uses Firebase for data handling. This problem is impacting the user experience and causing delays in our marketing initiatives.

Every couple of days, I've observed that all existing Firestore snapshot listeners in the app become "stuck" and continue to display outdated data when the app is resumed after being in the background for an extended period (5-10 hours). It appears these snapshot listeners reattach before the Firebase Auth ID token gets a chance to refresh. Consequently, they can't fetch the updated data and remain stuck with stale data. However, new snapshot listeners that are created after the app resumes function correctly, and write operations also work as expected.

Although the Auth ID token refreshes when the app resumes, the timing misalignment with the reattachment of the snapshot listeners hampers them from reflecting the most current data. Despite users remaining logged in, the pre-existing snapshot listeners encounter a 'Permission Denied' error due to not capturing the refreshed token.

The inconsistent nature of this issue and the time it takes to manifest make it challenging to create a demonstrative example, primarily because it appears to involve an internal bug in Firestore's workings which I don't understand how to best reproduce. In addition, force-refreshing the ID token frequently to accelerate the occurrence of this bug may risk hitting quota limits.

This problem surfaces every other day, regardless of an active internet connection, raising serious usability concerns. The only workaround currently available to users is to restart the app, a solution that is far from intuitive.

For a detailed discussion with a Firebase engineer who identified the cause regarding this issue, please refer to the linked StackOverflow post: [https://stackoverflow.com/questions/76436462/firebase-logged-in-user-with-expired-auth-id-token?noredirect=1]. A video illustrating the issue is posted here [https://youtube.com/shorts/Fq67pKY0yDo?feature=share].

Given its frequency and potential impact on other Firebase users, this issue urgently requires the Firebase team's attention.

I'm using the latest version of the Firebase pub packages.

Flutter doctor

Run flutter doctor and paste the output below:

Click To Expand

``` [✓] Flutter (Channel stable, 3.10.4, on Microsoft Windows [Version 10.0.22621.1778], locale en-US) [✓] Windows Version (Installed version of Windows is version 10 or higher) [✓] Android toolchain - develop for Android devices (Android SDK version 33.0.1) [✓] Chrome - develop for the web [✓] Visual Studio - develop for Windows (Visual Studio Community 2022 17.6.0) [✓] Android Studio (version 2022.2) [✓] IntelliJ IDEA Ultimate Edition (version 2023.1) [✓] Connected device (3 available) [✓] Network resources • No issues found! ```

---

[darshankawar]

SO link : https://stackoverflow.com/questions/76436462/firebase-logged-in-user-with-expired-auth-id-token?noredirect=1

Video link : https://youtube.com/shorts/Fq67pKY0yDo?feature=share

The inconsistent nature of this issue and the time it takes to manifest make it challenging to create a demonstrative example, primarily because it appears to involve an internal bug in Firestore's workings which I don't understand how to best reproduce. In addition, force-refreshing the ID token frequently to accelerate the occurrence of this bug may risk hitting quota limits.

This problem surfaces every other day, regardless of an active internet connection, raising serious usability concerns. The only workaround currently available to users is to restart the app, a solution that is far from intuitive.

For a detailed discussion with a Firebase engineer who identified the cause regarding this issue, please refer to the linked

Based on above details provided, keeping it open and labeling for team's attention.

/cc @russellwheatley

Also tagging @puf since he was involved in the SO discussion for the issue reported.

[russellwheatley]

Hey @RamsayGit, I was under the impression that the Firestore event listeners use the token set at the time of initialisation (maybe this has changed) (incorrect statement, see here). If that is the case, it would be prudent to create a StreamSubscription and cancel() (& reinitialise).

Just to be clear, the event listeners on the native side are not cleaned up when the app goes into background mode. They remain there. This is because some apps might require the snapshot event listener to stay active. If you don't require them to be active, you can close them out (as mentioned above) and reinitialise upon the app coming back into the foreground which would stop you from encountering permission errors after 5-10 hours of the app in the background.

[ramsayamarin]

@russellwheatley Your proposed solution is indeed the patch that I've applied to my app while waiting for a resolution to this issue. As a measure, I close long-lived listeners on each app resume and then recreate them. However, a point of concern remains; I'm uncertain whether this workaround will incur read costs, or if it will simply fetch them from cache and bill only for new results.

While this temporary solution appears to have been effective thus far (I have not encountered the error for the past two days since applying the patch), I'm continuing to monitor the situation closely. Despite this, it doesn't appear to align with the intended behavior as I understand it. Based on the documentation and various insights from Frank's answers on StackOverflow, it seems this should be a seamless process that is internally managed by Firebase, rather than requiring manual maintenance of listeners in relation to the ID token.

Additionally, there are still some aspects that I'm unclear about. Specifically, I'm unsure whether this listener reinitialization should occur immediately upon app resumption or if I need to manually monitor for a token refresh that might even occur mid-application, and then reinitialize based on these events. The clarification on these points would be greatly appreciated.

Looking forward to any additional insights you can provide on this matter.

class AdminHangoutRepository extends ChangeNotifier
    with WidgetsBindingObserver {
  Hangout? get hangout => _hangout;

  SnapshotMetadata? get metadata => _metadata;

  StreamSubscription<QuerySnapshot<Map<String, dynamic>>>? _hangoutSubscription;
  Hangout? _hangout;
  SnapshotMetadata? _metadata;

  AdminHangoutRepository() {
    WidgetsBinding.instance.addObserver(this);
    _attachListener();
  }

  @override
  void dispose() {
    WidgetsBinding.instance.removeObserver(this);
    _hangoutSubscription?.cancel();
    super.dispose();
  }

  @override
  void didChangeAppLifecycleState(AppLifecycleState state) {
    if (state == AppLifecycleState.resumed) {
      _attachListener();
    }
  }

  void _attachListener() {
    _hangoutSubscription?.cancel();
    _hangoutSubscription = FirebaseFirestore.instance
        .collection(FirePath.hangouts)
        .where....
        .snapshots(includeMetadataChanges: true)
        .transform(ignorePermissionDeniedTransformer())
        .listen((snapshot) {
          ...
        }, onError: (error) {
          FirebaseCrashlytics.instance.recordError(
            error,
            StackTrace.current,
            reason: 'Error in AdminHangoutRepository',
          );
        });
  }
}

[russellwheatley]

Hey @RamsayGit, I was under the impression that the Firestore event listeners use the token set at the time of initialisation (maybe this has changed). If that is the case, it would be prudent to create a StreamSubscription and cancel() (& reinitialise).

@RamsayGit - This statement is incorrect, I now understand that the fresh auth token ought to be picked up by the Firebase SDK and reattach the event listeners under the hood. My sense is, this is a bug on the firebase-android-sdk. I'll open an issue on that repo and see if we can get some resolution.

[russellwheatley]

@RamsayGit - do you still get permission-denied exceptions when the app is in the background? My hunch is that it would still occur when the app is in background, but it would be ideal if you could confirm if this is true. It would hone in a little more on the bug if we removed the variable of app state.

[russellwheatley]

@RamsayGit - I've created this issue here: https://github.com/firebase/firebase-android-sdk/issues/5101

[ramsayamarin]

@russellwheatley I've addressed the issue and no longer observe it, as I now recreate all snapshot listeners on app resume and refresh the token. This issue occurred in the foreground within one or two days after launching the app in a production environment, something I observed multiple times over a week, though not in a debug environment.

I'm uncertain about the exceptions it might emit. I recall personally observing an instance of the permission denied error, but I cannot confirm if the listener always raises an exception. Instead, it may stop receiving updates and go stale. I also can't confirm if the issue also occurred on iPhone, as my personal device is Android.

This issue is very significant because Firebase apps could become disabled within a couple of days, given that modern devices can indefinitely maintain apps in the background. I believe Firebase engineers could replicate this issue by creating a simple app with a long lived snapshot listener and data that increments periodically. The listener will stop updating within a couple of days. The issue might be expedited by refreshing the token more frequently, but as I'm not familiar with Firebase internals, I can't make a definitive statement.

In summary, while I can't confirm whether this issue happens in the background or if it's exclusive to Android, I can affirm that it occurs in Android's foreground within two days.

Thank you for your attention to this bug and creating the issue thread.