SharezoneApp / sharezone-app

Sharezone is a collaborative school organization app for iOS, Android, macOS and web with +500,000 downloads. Built with Flutter & Firebase.
https://sharezone.net
European Union Public License 1.2
254 stars 46 forks source link

Optimize Firestore Reads & Writes #1594

Open nilsreichardt opened 1 month ago

nilsreichardt commented 1 month ago

Description

This ticket is a collection of places where we can optimize reads & write with the goal of reducing our Firestore bills.

Next Lesson Calculator: Use cached Firestore document

https://github.com/SharezoneApp/sharezone-app/blob/f904a1181115bbfe4d8b0cc324c2464669e4baf7/app/lib/util/next_lesson_calculator/next_lesson_calculator.dart#L43

We can add a Source.cache parameter. This doesn't count as a read and is faster. The user doc should always be in the cache. Maybe this feature isn't available for the web.

Remove old joinedUsers course structure in our backend

Migrate all users to assignedUserArrays

nilsreichardt commented 1 month ago

For Next Lesson Calculator:

user_api.dart:

Future<AppUser> get({
  FirestoreGetSource source = FirestoreGetSource.serverAndCache,
}) {
  return references.users
      .doc(uID)
      .get(
        GetOptions(
          source: switch (source) {
            FirestoreGetSource.serverAndCache => Source.serverAndCache,
            FirestoreGetSource.server => Source.server,
            FirestoreGetSource.cache => Source.cache,
          },
        ),
      )
      .then((snapshot) => AppUser.fromData(snapshot.data(), id: snapshot.id));
}

cloud_firestore_helper.dart

/// An enumeration of firestore source types.
enum FirestoreGetSource {
  /// Causes Firestore to try to retrieve an up-to-date (server-retrieved)
  /// snapshot, but fall back to returning cached data if the server can't be
  /// reached.
  serverAndCache,

  /// Causes Firestore to avoid the cache, generating an error if the server cannot be reached. Note
  /// that the cache will still be updated if the server request succeeds. Also
  /// note that
  /// latency-compensation still takes effect, so any pending write operations will be visible in the
  /// returned data (merged into the server-provided data).
  server,

  /// Causes Firestore to immediately return a value from the cache, ignoring
  /// the server completely (implying that the returned value may be stale with
  /// respect to the value on the server). If there is no data in the cache to
  /// satisfy the `get` call, [DocumentReference.get] will throw a
  /// [FirebaseException] and [Query.get] will return an empty
  /// [QuerySnapshotPlatform] with no documents.
  cache,
}
nilsreichardt commented 1 month ago

https://medium.com/@sijohn.mathew/tracing-firestore-queries-unlock-insights-with-google-cloud-audit-logs-and-log-analytics-ca508f409303