search

firebase / firebase-ios-sdk

Firebase SDK for Apple App Development
https://firebase.google.com
Apache License 2.0
5.59k stars 1.46k forks source link

Using .orderBy in a Firestore query omits documents #13210

Open WillBishop opened 3 months ago

WillBishop commented 3 months ago

Description

I have a recipe manager which relies heavily on the cache to show users their recipes.

I have a case I can reproduce locally (though I'm unsure how to reproduce it in a sample project) where doing a .getDocuments() on a collection returns 46 items, but doing a collection.order(by: "name", descending: false). getDocuments() returns only 40 items.

Every single document contains a name field.

Reproducing the issue

I'm not sure how to provide a reproducible example, but I can describe my setup.

On launch, clients query the cached firestore for the first 15 documents ordered by name. If they reach the bottom of the list I query the next 15 and so on, until there are no documents left.

I also attach a listener to the collection to monitor for remote changes, and items are inserted in the list as needed.

If I query my cache with no filters or sorts applied I see 46 documents, if I sort by name and attach no other modifiers to my query I get only 40 entries.

You can see these expectations in the logs by filtering for "User Code"

Firebase SDK Version

10.28.1

Xcode Version

15.3

Installation Method

Swift Package Manager

Firebase Product(s)

Firestore

Targeted Platforms

iOS

Relevant Log Output

2024-07-02 18:39:35.130047+0930 Pestle[3064:660942] 10.28.1 - [FirebaseCore][I-COR000003] The default Firebase app has not yet been configured. Add `FirebaseApp.configure()` to your application initialization. This can be done in in the App Delegate's application(_:didFinishLaunchingWithOptions:)` (or the `@main` struct's initializer in SwiftUI). Read more: https://goo.gl/ctyzm8.
2024-07-02 18:39:35.131059+0930 Pestle[3064:660942] 10.28.1 - [FirebaseCore][I-COR000001] Configuring the default app.
2024-07-02 18:39:35.132657+0930 Pestle[3064:660550] <Warning> [AppCheckCore][I-GAC004001] App Check debug token: '6F04CB17-8ECD-46AE-909D-41EC4C66C0F6'.
2024-07-02 18:39:35.134451+0930 Pestle[3064:660942] 10.28.1 - [FirebaseMessaging][I-FCM002000] FIRMessaging library version 10.28.1
2024-07-02 18:39:35.134477+0930 Pestle[3064:660942] 10.28.1 - [GULReachability][I-REA902003] Monitoring the network status
2024-07-02 18:39:35.134491+0930 Pestle[3064:660942] 10.28.1 - [FirebaseInstallations][I-FIS002000] -[FIRInstallationsIDController createGetInstallationItemPromise], appName: __FIRAPP_DEFAULT
2024-07-02 18:39:35.134504+0930 Pestle[3064:660942] 10.28.1 - [FirebaseAppCheck][I-FAA005001] Firebase App Check debug token: '6F04CB17-8ECD-46AE-909D-41EC4C66C0F6'.
2024-07-02 18:39:35.134516+0930 Pestle[3064:660942] 10.28.1 - [FirebaseCore][I-COR000033] Data Collection flag is not set.
2024-07-02 18:39:35.134528+0930 Pestle[3064:660942] 10.28.1 - [GoogleUtilities/AppDelegateSwizzler][I-SWZ001011] App Delegate Proxy is disabled.
2024-07-02 18:39:35.137095+0930 Pestle[3064:660550] [Firebase/Crashlytics] Version 10.28.1
2024-07-02 18:39:35.140576+0930 Pestle[3064:660934] 10.28.1 - [FirebaseSessions][I-SES000000] Version 10.28.1. Expecting subscriptions from: [Crashlytics]
2024-07-02 18:39:35.140689+0930 Pestle[3064:660934] 10.28.1 - [FirebaseCrashlytics][I-CLS000000] [Crashlytics] Running on iPhone15,2, 18.0.0 (22A5297f)
2024-07-02 18:39:35.140755+0930 Pestle[3064:660934] 10.28.1 - [FirebaseCrashlytics][I-CLS000000] Registering Sessions SDK subscription for session data
2024-07-02 18:39:35.140816+0930 Pestle[3064:660934] 10.28.1 - [FirebaseSessions][I-SES000000] Registering Sessions SDK subscriber with name: Crashlytics, data collection enabled: true
2024-07-02 18:39:35.140843+0930 Pestle[3064:660934] 10.28.1 - [FirebaseCrashlytics][I-CLS000000] Session ID changed: c973ffa29d6c42e4bf120dc5b89f3e40
2024-07-02 18:39:35.141625+0930 Pestle[3064:660934] 10.28.1 - [FirebaseCrashlytics][I-CLS000000] [Crashlytics:Settings] Settings expired because build instance changed
2024-07-02 18:39:35.142957+0930 Pestle[3064:660942] 10.28.1 - [FirebaseCrashlytics][I-CLS000000] Root: /var/mobile/Containers/Data/Application/B12BC8BD-4835-418D-8521-9A77C3C89768/Library/Caches/com.crashlytics.data/com.willbishop.Simmer
[TelemetryDeck: INFO] Loading Telemetry cache from: file:///var/mobile/Containers/Data/Application/B12BC8BD-4835-418D-8521-9A77C3C89768/Library/Caches/telemetrysignalcache
2024-07-02 18:39:35.217802+0930 Pestle[3064:660550] [User Defaults] Couldn't read values in CFPrefsPlistSource<0x303849c20> (Domain: group.willbishop.Pestle, User: kCFPreferencesAnyUser, ByHost: Yes, Container: (null), Contents Need Refresh: Yes): Using kCFPreferencesAnyUser with a container is only allowed for System Containers, detaching from cfprefsd
2024-07-02 18:39:35.220777+0930 Pestle[3064:660934] 10.28.1 - [FirebaseCrashlytics][I-CLS000000] [Crashlytics:Crash:Reports:Event] Firebase Analytics SDK not detected. Crash-free statistics and breadcrumbs will not be reported
2024-07-02 18:39:35.220810+0930 Pestle[3064:660934] 10.28.1 - [FirebaseCrashlytics][I-CLS000000] Automatic data collection is enabled.
2024-07-02 18:39:35.222150+0930 Pestle[3064:660934] 10.28.1 - [FirebaseCrashlytics][I-CLS000000] Unsent reports will be uploaded at startup
2024-07-02 18:39:35.321988+0930 Pestle[3064:660941] 10.28.1 - [FirebaseFirestore][I-FST000001] Initializing. Current user: MYUSERID
2024-07-02 18:39:35.322075+0930 Pestle[3064:660941] 10.28.1 - [FirebaseFirestore][I-FST000001] Using /var/mobile/Containers/Data/Application/B12BC8BD-4835-418D-8521-9A77C3C89768/Library/Application Support/firestore/__FIRAPP_DEFAULT/cookingapp-57562/main for LevelDB storage
Applied Snapshot
2024-07-02 18:39:35.358430+0930 Pestle[3064:660955] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction Start LevelDB: 0 changes (0 bytes):>
2024-07-02 18:39:35.358456+0930 Pestle[3064:660955] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction Start MutationQueue: 0 changes (0 bytes):>
2024-07-02 18:39:35.358466+0930 Pestle[3064:660955] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction Start IndexManager: 0 changes (0 bytes):>
2024-07-02 18:39:35.358476+0930 Pestle[3064:660955] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction migrate overlays: 0 changes (0 bytes):>
2024-07-02 18:39:35.358484+0930 Pestle[3064:660955] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction NextMutationBatchAfterBatchID: 0 changes (0 bytes):>
2024-07-02 18:39:35.358495+0930 Pestle[3064:660955] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction Allocate target: 0 changes (0 bytes):>
2024-07-02 18:39:35.359561+0930 Pestle[3064:660955] 10.28.1 - [FirebaseFirestore][I-FST000001] Using full collection scan to execute query: Query(canonical_id=users/MYUSERID|f:|ob:__name__asc)
2024-07-02 18:39:35.359618+0930 Pestle[3064:660955] 10.28.1 - [FirebaseFirestore][I-FST000001] SDK will not create cache indexes for query: Query(canonical_id=users/MYUSERID|f:|ob:__name__asc), since it only creates cache indexes for collection contains more than or equal to 100 documents.
2024-07-02 18:39:35.359656+0930 Pestle[3064:660955] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction ExecuteQuery: 0 changes (0 bytes):>
2024-07-02 18:39:35.359672+0930 Pestle[3064:660955] 10.28.1 - [FirebaseFirestore][I-FST000001] WatchStream (h\^\rm\^A
2024-07-02 18:39:35.359683+0930 Pestle[3064:660955] 10.28.1 - [FirebaseAuth][I-AUT000002] Token auto-refresh enabled.
2024-07-02 18:39:35.359695+0930 Pestle[3064:660955] 10.28.1 - [FirebaseAuth][I-AUT000004] Token auto-refresh scheduled in 40:11 for the new token.
2024-07-02 18:39:35.359706+0930 Pestle[3064:660955] 10.28.1 - [FirebaseFirestore][I-FST000001] Using full collection scan to execute query: Query(canonical_id=users/MYUSERID/privateRecipes|f:|ob:__name__asc|lt:f)
2024-07-02 18:39:35.359718+0930 Pestle[3064:660955] 10.28.1 - [FirebaseAuth][I-AUT000017] Has valid access token. Estimated expiration date: 2024-07-02 09:54:46 +0000, current date: 2024-07-02 09:09:35 +0000
2024-07-02 18:39:35.641671+0930 Pestle[3064:660955] [loading] Unable to create bundle at URL (file:///System/Library/CoreServices/SystemVersion.bundle): does not exist or not a directory (0)
2024-07-02 18:39:35.641701+0930 Pestle[3064:660955] [loading] Unable to create bundle at URL (file:///System/Library/CoreServices/SystemVersion.bundle): does not exist or not a directory (0)
2024-07-02 18:39:35.641717+0930 Pestle[3064:660955] [loading] Unable to create bundle at URL (file:///System/Library/CoreServices/SystemVersion.bundle): does not exist or not a directory (0)
2024-07-02 18:39:35.652411+0930 Pestle[3064:660936] 10.28.1 - [GULReachability][I-REA902004] Network status has changed. Code:3, status:Connected
2024-07-02 18:39:35.655051+0930 Pestle[3064:660932] [] nw_path_necp_check_for_updates Failed to copy updated result (22)
User Code: Executing getDocuments. Expecting 46
2024-07-02 18:39:35.831797+0930 Pestle[3064:660932] 10.28.1 - [FirebaseFirestore][I-FST000001] GetToken aborted due to token change.
2024-07-02 18:39:35.831877+0930 Pestle[3064:660932] 10.28.1 - [FirebaseAuth][I-AUT000017] Has valid access token. Estimated expiration date: 2024-07-02 09:54:46 +0000, current date: 2024-07-02 09:09:35 +0000
2024-07-02 18:39:35.831924+0930 Pestle[3064:660932] 10.28.1 - [FirebaseMessaging][I-FCM023012] Provisioning profile has specifically provisioned devices, most likely a Dev profile.
2024-07-02 18:39:35.832099+0930 Pestle[3064:660932] 10.28.1 - [FirebaseMessaging][I-FCM023013] APNS Environment in profile: development
2024-07-02 18:39:35.832532+0930 Pestle[3064:660932] 10.28.1 - [FirebaseInstallations][I-FIS002001] -[FIRInstallationsIDController installationWithValidAuthTokenForcingRefresh:0], appName: __FIRAPP_DEFAULT
2024-07-02 18:39:35.832685+0930 Pestle[3064:660932] 10.28.1 - [FirebaseFirestore][I-FST000001] Query: Query(canonical_id=users/MYUSERID/privateRecipes|f:|ob:__name__asc|lt:f), scans 773 local documents and returns 46 documents as results.
2024-07-02 18:39:35.832739+0930 Pestle[3064:660932] 10.28.1 - [FirebaseFirestore][I-FST000001] The SDK decides to create cache indexes for query: Query(canonical_id=users/MYUSERID/privateRecipes|f:|ob:__name__asc|lt:f), as using cache indexes may help improve performance.
2024-07-02 18:39:35.832785+0930 Pestle[3064:660932] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction ExecuteQuery: 0 changes (0 bytes):>
2024-07-02 18:39:35.833048+0930 Pestle[3064:660932] 10.28.1 - [FirebaseFirestore][I-FST000001] Using index privateRecipes to execute target users/MYUSERID/privateRecipes|f:dateAdded>=time(1719738575,328838825)|ob:dateAddedasc__name__asc
2024-07-02 18:39:35.833184+0930 Pestle[3064:660932] 10.28.1 - [FirebaseCrashlytics][I-CLS000000] Settings downloaded successfully
2024-07-02 18:39:35.844348+0930 Pestle[3064:660971] 10.28.1 - [FirebaseSessions][I-SES000000] Data Collection is enabled for at least one Subscriber
2024-07-02 18:39:35.844388+0930 Pestle[3064:660971] 10.28.1 - [FirebaseSessions][I-SES000000] [Settings] Cache is not expired, no fetch will be made.
2024-07-02 18:39:35.844437+0930 Pestle[3064:660971] 10.28.1 - [FirebaseInstallations][I-FIS002001] -[FIRInstallationsIDController installationWithValidAuthTokenForcingRefresh:0], appName: __FIRAPP_DEFAULT
2024-07-02 18:39:35.844488+0930 Pestle[3064:660971] 10.28.1 - [FirebaseInstallations][I-FIS002000] -[FIRInstallationsIDController createGetInstallationItemPromise], appName: __FIRAPP_DEFAULT
2024-07-02 18:39:35.909993+0930 Pestle[3064:660933] 10.28.1 - [FirebaseSessions][I-SES000000] Successfully logged Session Start event to GoogleDataTransport
2024-07-02 18:39:35.910890+0930 Pestle[3064:660933] 10.28.1 - [FirebaseSessions][I-SES000000] Successfully logged Session Start event
2024-07-02 18:39:35.914246+0930 Pestle[3064:660933] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction ExecuteQuery: 0 changes (0 bytes):>
2024-07-02 18:39:35.914380+0930 Pestle[3064:660933] 10.28.1 - [FirebaseFirestore][I-FST000001] Using full collection scan to execute query: Query(canonical_id=users/MYUSERID/privateRecipes|f:|ob:__name__asc|lt:f)
2024-07-02 18:39:36.008612+0930 Pestle[3064:660932] 10.28.1 - [FirebaseFirestore][I-FST000001] Query: Query(canonical_id=users/MYUSERID/privateRecipes|f:|ob:__name__asc|lt:f), scans 773 local documents and returns 46 documents as results.
2024-07-02 18:39:36.012231+0930 Pestle[3064:660932] 10.28.1 - [FirebaseFirestore][I-FST000001] The SDK decides to create cache indexes for query: Query(canonical_id=users/MYUSERID/privateRecipes|f:|ob:__name__asc|lt:f), as using cache indexes may help improve performance.
2024-07-02 18:39:36.012267+0930 Pestle[3064:660932] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction ExecuteQuery: 0 changes (0 bytes):>
2024-07-02 18:39:36.014727+0930 Pestle[3064:660932] 10.28.1 - [FirebaseAuth][I-AUT000017] Has valid access token. Estimated expiration date: 2024-07-02 09:54:46 +0000, current date: 2024-07-02 09:09:36 +0000
2024-07-02 18:39:36.015886+0930 Pestle[3064:660932] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction NextMutationBatchAfterBatchID: 0 changes (0 bytes):>
2024-07-02 18:39:36.016108+0930 Pestle[3064:660932] 10.28.1 - [FirebaseFirestore][I-FST000001] Credential Changed. Current user: MYUSERID
2024-07-02 18:39:36.016276+0930 Pestle[3064:660932] 10.28.1 - [FirebaseAuth][I-AUT000017] Has valid access token. Estimated expiration date: 2024-07-02 09:54:46 +0000, current date: 2024-07-02 09:09:36 +0000
2024-07-02 18:39:36.016314+0930 Pestle[3064:660932] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction NextMutationBatchAfterBatchID: 0 changes (0 bytes):>
2024-07-02 18:39:36.016341+0930 Pestle[3064:660932] 10.28.1 - [FirebaseFirestore][I-FST000001] Using index privateRecipes to execute target users/MYUSERID/privateRecipes|f:|ob:nameasc__name__asc
2024-07-02 18:39:36.371359+0930 Pestle[3064:660932] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction ExecuteQuery: 0 changes (0 bytes):>
2024-07-02 18:39:36.371933+0930 Pestle[3064:660932] 10.28.1 - [FirebaseFirestore][I-FST000001] Using index privateRecipes to execute target users/MYUSERID/privateRecipes|f:|ob:nameasc__name__asc
User Code: Recevied Optional(40) documents
2024-07-02 18:39:40.305127+0930 Pestle[3064:660941] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction ExecuteQuery: 0 changes (0 bytes):>
2024-07-02 18:39:40.305371+0930 Pestle[3064:660941] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction Allocate target: 3 changes (186 bytes):
  - Put [query_target: canonical_id=betaAnnouncements|f:endRelevant>time(1719911375,691611051)|ob:endRelevantasc__name__asc target_id=13980] (0 bytes)
  - Put [target: target_id=13980] (162 bytes)
  - Put [target_global:] (24 bytes)>
2024-07-02 18:39:40.306759+0930 Pestle[3064:660941] 10.28.1 - [FirebaseFirestore][I-FST000001] Using full collection scan to execute query: Query(canonical_id=betaAnnouncements|f:endRelevant>time(1719911375,691611051)|ob:endRelevantasc__name__asc)
2024-07-02 18:39:40.307517+0930 Pestle[3064:660941] 10.28.1 - [FirebaseFirestore][I-FST000001] SDK will not create cache indexes for query: Query(canonical_id=betaAnnouncements|f:endRelevant>time(1719911375,691611051)|ob:endRelevantasc__name__asc), since it only creates cache indexes for collection contains more than or equal to 100 documents.
2024-07-02 18:39:40.375912+0930 Pestle[3064:660941] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction ExecuteQuery: 0 changes (0 bytes):>
2024-07-02 18:39:40.375999+0930 Pestle[3064:660941] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction Locally write mutations: 7 changes (70324 bytes):
  - Put [collection_parent: collection_id=users] (0 bytes)
  - Put [document_mutation: user_id=MYUSERID path=users/MYUSERID batch_id=1] (0 bytes)
  - Put [document_overlays: user_id=MYUSERID path=users/MYUSERID batch_id=1] (42059 bytes)
  - Put [document_overlays_collection_group_index: user_id=MYUSERID collection_group=users batch_id=1 path=users/MYUSERID] (0 bytes)
  - Put [document_overlays_collection_index: user_id=MYUSERID path=users batch_id=1 document_id=MYUSERID] (0 bytes)
  - Put [document_overlays_largest_batch_id_index: user_id=MYUSERID batch_id=1 path=users/MYUSERID] (0 bytes)
  - Put [mutation: user_id=MYUSERID batch_id=1] (28265 bytes)>
2024-07-02 18:39:40.524922+0930 Pestle[3064:660941] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction NotifyLocalViewChanges: 0 changes (0 bytes):>
2024-07-02 18:39:40.599815+0930 Pestle[3064:660941] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction NextMutationBatchAfterBatchID: 0 changes (0 bytes):>
2024-07-02 18:39:40.758985+0930 Pestle[3064:660941] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction NextMutationBatchAfterBatchID: 0 changes (0 bytes):>
2024-07-02 18:39:40.759689+0930 Pestle[3064:660941] 10.28.1 - [FirebaseFirestore][I-FST000001] Creating Firestore stub.
2024-07-02 18:39:40.759776+0930 Pestle[3064:660941] 10.28.1 - [FirebaseAuth][I-AUT000017] Has valid access token. Estimated expiration date: 2024-07-02 09:54:46 +0000, current date: 2024-07-02 09:09:39 +0000
2024-07-02 18:39:40.839330+0930 Pestle[3064:660937] 10.28.1 - [FirebaseFirestore][I-FST000001] LocalStore Ignoring outdated update for betaAnnouncements/mailinglist. Current version: Timestamp(seconds=1706179480, nanoseconds=788218000)  Remote version: Timestamp(seconds=1706179480, nanoseconds=788218000)
2024-07-02 18:39:40.839426+0930 Pestle[3064:660937] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction Apply remote event: 5 changes (216 bytes):
  - Put [document_target: path=betaAnnouncements/mailinglist target_id=0] (3 bytes)
  - Put [document_target: path=betaAnnouncements/mailinglist target_id=13980] (0 bytes)
  - Put [target: target_id=13980] (189 bytes)
  - Put [target_document: target_id=13980 path=betaAnnouncements/mailinglist] (0 bytes)
  - Put [target_global:] (24 bytes)>
2024-07-02 18:39:40.839502+0930 Pestle[3064:660937] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction NotifyLocalViewChanges: 0 changes (0 bytes):>
2024-07-02 18:39:40.839887+0930 Pestle[3064:660937] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction Apply remote event: 1 changes (24 bytes):
  - Put [target_global:] (24 bytes)>
2024-07-02 18:39:40.839927+0930 Pestle[3064:660937] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction NotifyLocalViewChanges: 0 changes (0 bytes):>
2024-07-02 18:39:41.033686+0930 Pestle[3064:660550] Metal API Validation Enabled
2024-07-02 18:39:41.147800+0930 Pestle[3064:660550] [Unknown process name] CGImageProviderGetContentHeadroom: Bad headroom value 0.000000 for SDR, returning 1.0
2024-07-02 18:39:41.153555+0930 Pestle[3064:660550] [Unknown process name] CGImageProviderGetContentHeadroom: Bad headroom value 0.000000 for SDR, returning 1.0
2024-07-02 18:39:41.160571+0930 Pestle[3064:660550] [Unknown process name] CGImageProviderGetContentHeadroom: Bad headroom value 0.000000 for SDR, returning 1.0
2024-07-02 18:39:41.187012+0930 Pestle[3064:660972] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction Release target: 2 changes (227 bytes):
  - Put [target: target_id=13980] (203 bytes)
  - Put [target_global:] (24 bytes)>
2024-07-02 18:39:41.189991+0930 Pestle[3064:660550] [Unknown process name] CGImageProviderGetContentHeadroom: Bad headroom value 0.000000 for SDR, returning 1.0
2024-07-02 18:39:41.192967+0930 Pestle[3064:660550] [Unknown process name] CGImageProviderGetContentHeadroom: Bad headroom value 0.000000 for SDR, returning 1.0
2024-07-02 18:39:41.196554+0930 Pestle[3064:660550] [Unknown process name] CGImageProviderGetContentHeadroom: Bad headroom value 0.000000 for SDR, returning 1.0
User Code: Executing getDocuments. Expecting 46
2024-07-02 18:39:41.223270+0930 Pestle[3064:660937] 10.28.1 - [FirebaseFirestore][I-FST000001] Using index privateRecipes to execute target users/MYUSERID/privateRecipes|f:|ob:dateAddeddesc__name__desc
2024-07-02 18:39:41.306808+0930 Pestle[3064:660973] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction ExecuteQuery: 0 changes (0 bytes):>
User Code: Recevied Optional(40) documents
2024-07-02 18:39:41.606172+0930 Pestle[3064:660972] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction Set stream token: 1 changes (11 bytes):
  - Put [mutation_queue: user_id=MYUSERID] (11 bytes)>
2024-07-02 18:39:41.606184+0930 Pestle[3064:660972] 10.28.1 - [FirebaseFirestore][I-FST000001] (null)

If using Swift Package Manager, the project's Package.resolved

{ "originHash" : "59824019fe0baafd60e68f8074c49456c84efe83e448e2affee2e4f43135c13b", "pins" : [ { "identity" : "abseil-cpp-binary", "kind" : "remoteSourceControl", "location" : "https://github.com/google/abseil-cpp-binary.git", "state" : { "revision" : "748c7837511d0e6a507737353af268484e1745e2", "version" : "1.2024011601.1" } }, { "identity" : "app-check", "kind" : "remoteSourceControl", "location" : "https://github.com/google/app-check.git", "state" : { "revision" : "3b62f154d00019ae29a71e9738800bb6f18b236d", "version" : "10.19.2" } }, { "identity" : "firebase-ios-sdk", "kind" : "remoteSourceControl", "location" : "https://github.com/firebase/firebase-ios-sdk", "state" : { "revision" : "e57841b296d04370ea23580f908881b0ccab17b9", "version" : "10.28.1" } }, { "identity" : "googleappmeasurement", "kind" : "remoteSourceControl", "location" : "https://github.com/google/GoogleAppMeasurement.git", "state" : { "revision" : "fe727587518729046fc1465625b9afd80b5ab361", "version" : "10.28.0" } }, { "identity" : "googledatatransport", "kind" : "remoteSourceControl", "location" : "https://github.com/google/GoogleDataTransport.git", "state" : { "revision" : "a637d318ae7ae246b02d7305121275bc75ed5565", "version" : "9.4.0" } }, { "identity" : "googleutilities", "kind" : "remoteSourceControl", "location" : "https://github.com/google/GoogleUtilities.git", "state" : { "revision" : "57a1d307f42df690fdef2637f3e5b776da02aad6", "version" : "7.13.3" } }, { "identity" : "grpc-binary", "kind" : "remoteSourceControl", "location" : "https://github.com/google/grpc-binary.git", "state" : { "revision" : "e9fad491d0673bdda7063a0341fb6b47a30c5359", "version" : "1.62.2" } }, { "identity" : "gtm-session-fetcher", "kind" : "remoteSourceControl", "location" : "https://github.com/google/gtm-session-fetcher.git", "state" : { "revision" : "a2ab612cb980066ee56d90d60d8462992c07f24b", "version" : "3.5.0" } }, { "identity" : "interop-ios-for-google-sdks", "kind" : "remoteSourceControl", "location" : "https://github.com/google/interop-ios-for-google-sdks.git", "state" : { "revision" : "2d12673670417654f08f5f90fdd62926dc3a2648", "version" : "100.0.0" } }, { "identity" : "leveldb", "kind" : "remoteSourceControl", "location" : "https://github.com/firebase/leveldb.git", "state" : { "revision" : "a0bc79961d7be727d258d33d5a6b2f1023270ba1", "version" : "1.22.5" } }, { "identity" : "nanopb", "kind" : "remoteSourceControl", "location" : "https://github.com/firebase/nanopb.git", "state" : { "revision" : "b7e1104502eca3a213b46303391ca4d3bc8ddec1", "version" : "2.30910.0" } }, { "identity" : "promises", "kind" : "remoteSourceControl", "location" : "https://github.com/google/promises.git", "state" : { "revision" : "540318ecedd63d883069ae7f1ed811a2df00b6ac", "version" : "2.4.0" } }, { "identity" : "swift-log", "kind" : "remoteSourceControl", "location" : "https://github.com/apple/swift-log", "state" : { "revision" : "9cb486020ebf03bfa5b5df985387a14a98744537", "version" : "1.6.1" } }, { "identity" : "swift-protobuf", "kind" : "remoteSourceControl", "location" : "https://github.com/apple/swift-protobuf.git", "state" : { "revision" : "9f0c76544701845ad98716f3f6a774a892152bcb", "version" : "1.26.0" } }, ], "version" : 3 }

If using CocoaPods, the project's Podfile.lock

No response

WillBishop commented 3 months ago

For some more context, I track the number of documents by counting the docs in the cache with no filters or orders.

Ordered

recipeCollection
    .order(by: "name")
    .getDocuments(source: .cache) { snapshot, error in
        // 40 documents in the snapshot
    }

Unordered

recipeCollection
    .getDocuments(source: .cache) { snapshot, error in
        // 46 documents in the snapshot
    }
WillBishop commented 3 months ago

I quietened my code a bit to make sure I only execute a singly query, here are those logs:

2024-07-02 18:54:16.285121+0930 Pestle[3169:675912] 10.28.1 - [FirebaseCore][I-COR000003] The default Firebase app has not yet been configured. Add `FirebaseApp.configure()` to your application initialization. This can be done in in the App Delegate's application(_:didFinishLaunchingWithOptions:)` (or the `@main` struct's initializer in SwiftUI). Read more: https://goo.gl/ctyzm8.
2024-07-02 18:54:16.285155+0930 Pestle[3169:675912] 10.28.1 - [FirebaseCore][I-COR000001] Configuring the default app.
2024-07-02 18:54:16.351539+0930 Pestle[3169:675467] <Warning> [AppCheckCore][I-GAC004001] App Check debug token: 'DEBUGTOKEN'.
2024-07-02 18:54:16.365594+0930 Pestle[3169:675467] [Firebase/Crashlytics] Version 10.28.1
2024-07-02 18:54:16.367630+0930 Pestle[3169:675927] 10.28.1 - [FirebaseAppCheck][I-FAA005001] Firebase App Check debug token: 'DEBUGTOKEN'.
2024-07-02 18:54:16.367658+0930 Pestle[3169:675927] 10.28.1 - [FirebaseCore][I-COR000033] Data Collection flag is not set.
2024-07-02 18:54:16.367671+0930 Pestle[3169:675927] 10.28.1 - [GoogleUtilities/AppDelegateSwizzler][I-SWZ001011] App Delegate Proxy is disabled.
2024-07-02 18:54:16.369618+0930 Pestle[3169:675927] 10.28.1 - [FirebaseMessaging][I-FCM002000] FIRMessaging library version 10.28.1
2024-07-02 18:54:16.369864+0930 Pestle[3169:675927] 10.28.1 - [GULReachability][I-REA902003] Monitoring the network status
2024-07-02 18:54:16.369887+0930 Pestle[3169:675927] 10.28.1 - [FirebaseInstallations][I-FIS002000] -[FIRInstallationsIDController createGetInstallationItemPromise], appName: __FIRAPP_DEFAULT
2024-07-02 18:54:16.369904+0930 Pestle[3169:675927] 10.28.1 - [FirebaseSessions][I-SES000000] Version 10.28.1. Expecting subscriptions from: [Crashlytics]
2024-07-02 18:54:16.369920+0930 Pestle[3169:675927] 10.28.1 - [FirebaseCrashlytics][I-CLS000000] [Crashlytics] Running on iPhone15,2, 18.0.0 (22A5297f)
2024-07-02 18:54:16.369936+0930 Pestle[3169:675927] 10.28.1 - [FirebaseCrashlytics][I-CLS000000] Registering Sessions SDK subscription for session data
2024-07-02 18:54:16.370394+0930 Pestle[3169:675927] 10.28.1 - [FirebaseSessions][I-SES000000] Registering Sessions SDK subscriber with name: Crashlytics, data collection enabled: true
2024-07-02 18:54:16.370417+0930 Pestle[3169:675927] 10.28.1 - [FirebaseCrashlytics][I-CLS000000] Session ID changed: 301cada883404b96b31f90b27d55f022
2024-07-02 18:54:16.370433+0930 Pestle[3169:675927] 10.28.1 - [FirebaseCrashlytics][I-CLS000000] [Crashlytics:Settings] Settings expired because build instance changed
2024-07-02 18:54:16.370447+0930 Pestle[3169:675927] 10.28.1 - [FirebaseCrashlytics][I-CLS000000] Root: /var/mobile/Containers/Data/Application/A0035EDF-7ED5-41D8-8B3C-A59E944E5CA6/Library/Caches/com.crashlytics.data/com.willbishop.Simmer
2024-07-02 18:54:16.528536+0930 Pestle[3169:675926] 10.28.1 - [FirebaseCrashlytics][I-CLS000000] [Crashlytics:Crash:Reports:Event] Firebase Analytics SDK not detected. Crash-free statistics and breadcrumbs will not be reported
2024-07-02 18:54:16.528577+0930 Pestle[3169:675926] 10.28.1 - [FirebaseCrashlytics][I-CLS000000] Automatic data collection is enabled.
2024-07-02 18:54:16.528592+0930 Pestle[3169:675926] 10.28.1 - [FirebaseCrashlytics][I-CLS000000] Unsent reports will be uploaded at startup
2024-07-02 18:54:16.528614+0930 Pestle[3169:675926] 10.28.1 - [FirebaseFirestore][I-FST000001] Initializing. Current user: MYUSERID
2024-07-02 18:54:16.528628+0930 Pestle[3169:675926] 10.28.1 - [FirebaseFirestore][I-FST000001] Using /var/mobile/Containers/Data/Application/A0035EDF-7ED5-41D8-8B3C-A59E944E5CA6/Library/Application Support/firestore/__FIRAPP_DEFAULT/cookingapp-57562/main for LevelDB storage
2024-07-02 18:54:16.580176+0930 Pestle[3169:675923] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction Start LevelDB: 0 changes (0 bytes):>
2024-07-02 18:54:16.582832+0930 Pestle[3169:675921] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction Start MutationQueue: 0 changes (0 bytes):>
2024-07-02 18:54:16.583234+0930 Pestle[3169:675921] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction Start IndexManager: 0 changes (0 bytes):>
2024-07-02 18:54:16.583514+0930 Pestle[3169:675921] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction migrate overlays: 0 changes (0 bytes):>
2024-07-02 18:54:16.583684+0930 Pestle[3169:675914] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction NextMutationBatchAfterBatchID: 0 changes (0 bytes):>
2024-07-02 18:54:16.585100+0930 Pestle[3169:675914] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction Allocate target: 0 changes (0 bytes):>
2024-07-02 18:54:16.585589+0930 Pestle[3169:675914] 10.28.1 - [FirebaseFirestore][I-FST000001] Using full collection scan to execute query: Query(canonical_id=users/MYUSERID|f:|ob:__name__asc)
Applied Snapshot
2024-07-02 18:54:16.587029+0930 Pestle[3169:675914] 10.28.1 - [FirebaseFirestore][I-FST000001] SDK will not create cache indexes for query: Query(canonical_id=users/MYUSERID|f:|ob:__name__asc), since it only creates cache indexes for collection contains more than or equal to 100 documents.
2024-07-02 18:54:16.587051+0930 Pestle[3169:675914] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction ExecuteQuery: 0 changes (0 bytes):>
2024-07-02 18:54:16.587293+0930 Pestle[3169:675926] 10.28.1 - [FirebaseFirestore][I-FST000001] (null)
2024-07-02 18:54:16.587326+0930 Pestle[3169:675926] 10.28.1 - [FirebaseAuth][I-AUT000002] Token auto-refresh enabled.
2024-07-02 18:54:16.587398+0930 Pestle[3169:675926] 10.28.1 - [FirebaseAuth][I-AUT000004] Token auto-refresh scheduled in 25:30 for the new token.
2024-07-02 18:54:16.587428+0930 Pestle[3169:675926] 10.28.1 - [FirebaseAuth][I-AUT000017] Has valid access token. Estimated expiration date: 2024-07-02 09:54:46 +0000, current date: 2024-07-02 09:24:16 +0000
2024-07-02 18:54:16.587496+0930 Pestle[3169:675926] 10.28.1 - [FirebaseFirestore][I-FST000001] Using full collection scan to execute query: Query(canonical_id=users/MYUSERID/privateRecipes|f:|ob:__name__asc|lt:f)
2024-07-02 18:54:16.892708+0930 Pestle[3169:675912] 10.28.1 - [GULReachability][I-REA902004] Network status has changed. Code:3, status:Connected
2024-07-02 18:54:16.937782+0930 Pestle[3169:675919] 10.28.1 - [FirebaseFirestore][I-FST000001] GetToken aborted due to token change.
2024-07-02 18:54:16.941951+0930 Pestle[3169:675919] 10.28.1 - [FirebaseAuth][I-AUT000017] Has valid access token. Estimated expiration date: 2024-07-02 09:54:46 +0000, current date: 2024-07-02 09:24:16 +0000
2024-07-02 18:54:16.942000+0930 Pestle[3169:675919] 10.28.1 - [FirebaseMessaging][I-FCM023012] Provisioning profile has specifically provisioned devices, most likely a Dev profile.
2024-07-02 18:54:16.943450+0930 Pestle[3169:675919] 10.28.1 - [FirebaseMessaging][I-FCM023013] APNS Environment in profile: development
2024-07-02 18:54:16.959529+0930 Pestle[3169:675919] 10.28.1 - [FirebaseInstallations][I-FIS002001] -[FIRInstallationsIDController installationWithValidAuthTokenForcingRefresh:0], appName: __FIRAPP_DEFAULT
2024-07-02 18:54:16.959658+0930 Pestle[3169:675919] 10.28.1 - [FirebaseSessions][I-SES000000] Data Collection is enabled for at least one Subscriber
2024-07-02 18:54:16.963401+0930 Pestle[3169:675921] 10.28.1 - [FirebaseSessions][I-SES000000] [Settings] Cache is not expired, no fetch will be made.
2024-07-02 18:54:16.963774+0930 Pestle[3169:675921] 10.28.1 - [FirebaseInstallations][I-FIS002001] -[FIRInstallationsIDController installationWithValidAuthTokenForcingRefresh:0], appName: __FIRAPP_DEFAULT
2024-07-02 18:54:16.995733+0930 Pestle[3169:675921] 10.28.1 - [FirebaseInstallations][I-FIS002000] -[FIRInstallationsIDController createGetInstallationItemPromise], appName: __FIRAPP_DEFAULT
2024-07-02 18:54:16.995805+0930 Pestle[3169:675921] 10.28.1 - [FirebaseFirestore][I-FST000001] Query: Query(canonical_id=users/MYUSERID/privateRecipes|f:|ob:__name__asc|lt:f), scans 773 local documents and returns 46 documents as results.
2024-07-02 18:54:16.995829+0930 Pestle[3169:675921] 10.28.1 - [FirebaseFirestore][I-FST000001] The SDK decides to create cache indexes for query: Query(canonical_id=users/MYUSERID/privateRecipes|f:|ob:__name__asc|lt:f), as using cache indexes may help improve performance.
2024-07-02 18:54:16.995980+0930 Pestle[3169:675921] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction ExecuteQuery: 0 changes (0 bytes):>
2024-07-02 18:54:17.009213+0930 Pestle[3169:675967] 10.28.1 - [FirebaseFirestore][I-FST000001] Using index privateRecipes to execute target users/MYUSERID/privateRecipes|f:dateAdded>=time(1719739456,569427013)|ob:dateAddedasc__name__asc
2024-07-02 18:54:17.009597+0930 Pestle[3169:675967] 10.28.1 - [FirebaseSessions][I-SES000000] Successfully logged Session Start event to GoogleDataTransport
2024-07-02 18:54:17.009874+0930 Pestle[3169:675967] 10.28.1 - [FirebaseSessions][I-SES000000] Successfully logged Session Start event
2024-07-02 18:54:17.101235+0930 Pestle[3169:675919] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction ExecuteQuery: 0 changes (0 bytes):>
2024-07-02 18:54:17.102116+0930 Pestle[3169:675919] 10.28.1 - [FirebaseFirestore][I-FST000001] Using full collection scan to execute query: Query(canonical_id=users/MYUSERID/privateRecipes|f:|ob:__name__asc|lt:f)
2024-07-02 18:54:18.224269+0930 Pestle[3169:675967] 10.28.1 - [FirebaseCrashlytics][I-CLS000000] Settings downloaded successfully
2024-07-02 18:54:21.411952+0930 Pestle[3169:675913] 10.28.1 - [FirebaseFirestore][I-FST000001] Query: Query(canonical_id=users/MYUSERID/privateRecipes|f:|ob:__name__asc|lt:f), scans 773 local documents and returns 46 documents as results.
2024-07-02 18:54:21.414109+0930 Pestle[3169:675913] 10.28.1 - [FirebaseFirestore][I-FST000001] The SDK decides to create cache indexes for query: Query(canonical_id=users/MYUSERID/privateRecipes|f:|ob:__name__asc|lt:f), as using cache indexes may help improve performance.
2024-07-02 18:54:21.414659+0930 Pestle[3169:675913] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction ExecuteQuery: 0 changes (0 bytes):>
2024-07-02 18:54:21.414973+0930 Pestle[3169:675913] 10.28.1 - [FirebaseFirestore][I-FST000001] (null)
2024-07-02 18:54:21.416005+0930 Pestle[3169:675913] 10.28.1 - [FirebaseFirestore][I-FST000001] (null)
2024-07-02 18:54:21.416580+0930 Pestle[3169:675913] 10.28.1 - [FirebaseFirestore][I-FST000001] (null)
2024-07-02 18:54:21.416990+0930 Pestle[3169:675913] 10.28.1 - [FirebaseFirestore][I-FST000001] (null)
2024-07-02 18:54:21.417255+0930 Pestle[3169:675913] 10.28.1 - [FirebaseAuth][I-AUT000017] Has valid access token. Estimated expiration date: 2024-07-02 09:54:46 +0000, current date: 2024-07-02 09:24:19 +0000
2024-07-02 18:54:21.417346+0930 Pestle[3169:675913] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction NextMutationBatchAfterBatchID: 0 changes (0 bytes):>
2024-07-02 18:54:21.418158+0930 Pestle[3169:675913] 10.28.1 - [FirebaseFirestore][I-FST000001] Credential Changed. Current user: MYUSERID
2024-07-02 18:54:21.418571+0930 Pestle[3169:675913] 10.28.1 - [FirebaseFirestore][I-FST000001] (null)
2024-07-02 18:54:21.418639+0930 Pestle[3169:675913] 10.28.1 - [FirebaseFirestore][I-FST000001] (null)
2024-07-02 18:54:21.419327+0930 Pestle[3169:675913] 10.28.1 - [FirebaseFirestore][I-FST000001] (null)
2024-07-02 18:54:21.583952+0930 Pestle[3169:675913] 10.28.1 - [FirebaseFirestore][I-FST000001] (null)
2024-07-02 18:54:21.584695+0930 Pestle[3169:675913] 10.28.1 - [FirebaseAuth][I-AUT000017] Has valid access token. Estimated expiration date: 2024-07-02 09:54:46 +0000, current date: 2024-07-02 09:24:19 +0000
2024-07-02 18:54:21.584836+0930 Pestle[3169:675913] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction NextMutationBatchAfterBatchID: 0 changes (0 bytes):>
2024-07-02 18:54:22.247576+0930 Pestle[3169:675972] 10.28.1 - [FirebaseFirestore][I-FST000001] Using index privateRecipes to execute target users/MYUSERID/privateRecipes|f:|ob:nameasc__name__asc
2024-07-02 18:54:22.361015+0930 Pestle[3169:675921] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction ExecuteQuery: 0 changes (0 bytes):>
2024-07-02 18:54:22.361564+0930 Pestle[3169:675921] 10.28.1 - [FirebaseFirestore][I-FST000001] Using index privateRecipes to execute target users/MYUSERID/privateRecipes|f:|ob:nameasc__name__asc
User Code: Recevied Optional(40) documents
2024-07-02 18:54:22.545203+0930 Pestle[3169:675919] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction ExecuteQuery: 0 changes (0 bytes):>
2024-07-02 18:54:22.545227+0930 Pestle[3169:675919] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction Allocate target: 3 changes (186 bytes):
  - Put [query_target: canonical_id=betaAnnouncements|f:endRelevant>time(1719912256,937561988)|ob:endRelevantasc__name__asc target_id=13996] (0 bytes)
  - Put [target: target_id=13996] (162 bytes)
  - Put [target_global:] (24 bytes)>
2024-07-02 18:54:22.545240+0930 Pestle[3169:675919] 10.28.1 - [FirebaseFirestore][I-FST000001] Using full collection scan to execute query: Query(canonical_id=betaAnnouncements|f:endRelevant>time(1719912256,937561988)|ob:endRelevantasc__name__asc)
2024-07-02 18:54:22.545251+0930 Pestle[3169:675919] 10.28.1 - [FirebaseFirestore][I-FST000001] SDK will not create cache indexes for query: Query(canonical_id=betaAnnouncements|f:endRelevant>time(1719912256,937561988)|ob:endRelevantasc__name__asc), since it only creates cache indexes for collection contains more than or equal to 100 documents.
2024-07-02 18:54:22.545260+0930 Pestle[3169:675919] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction ExecuteQuery: 0 changes (0 bytes):>
2024-07-02 18:54:22.548402+0930 Pestle[3169:675920] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction Locally write mutations: 7 changes (70324 bytes):
  - Put [collection_parent: collection_id=users] (0 bytes)
  - Put [document_mutation: user_id=MYUSERID path=users/MYUSERID batch_id=1] (0 bytes)
  - Put [document_overlays: user_id=MYUSERID path=users/MYUSERID batch_id=1] (42059 bytes)
  - Put [document_overlays_collection_group_index: user_id=MYUSERID collection_group=users batch_id=1 path=users/MYUSERID] (0 bytes)
  - Put [document_overlays_collection_index: user_id=MYUSERID path=users batch_id=1 document_id=MYUSERID] (0 bytes)
  - Put [document_overlays_largest_batch_id_index: user_id=MYUSERID batch_id=1 path=users/MYUSERID] (0 bytes)
  - Put [mutation: user_id=MYUSERID batch_id=1] (28265 bytes)>
2024-07-02 18:54:22.549066+0930 Pestle[3169:675920] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction NotifyLocalViewChanges: 0 changes (0 bytes):>
2024-07-02 18:54:22.549700+0930 Pestle[3169:675919] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction NextMutationBatchAfterBatchID: 0 changes (0 bytes):>
2024-07-02 18:54:22.549746+0930 Pestle[3169:675919] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction NextMutationBatchAfterBatchID: 0 changes (0 bytes):>
2024-07-02 18:54:22.549780+0930 Pestle[3169:675919] 10.28.1 - [FirebaseFirestore][I-FST000001] (null)
2024-07-02 18:54:22.549814+0930 Pestle[3169:675919] 10.28.1 - [FirebaseFirestore][I-FST000001] Creating Firestore stub.
2024-07-02 18:54:22.549908+0930 Pestle[3169:675919] 10.28.1 - [FirebaseAuth][I-AUT000017] Has valid access token. Estimated expiration date: 2024-07-02 09:54:46 +0000, current date: 2024-07-02 09:24:22 +0000
2024-07-02 18:54:22.566817+0930 Pestle[3169:675919] 10.28.1 - [FirebaseFirestore][I-FST000001] (null)
2024-07-02 18:54:22.566952+0930 Pestle[3169:675919] 10.28.1 - [FirebaseFirestore][I-FST000001] (null)
2024-07-02 18:54:22.865056+0930 Pestle[3169:675927] 10.28.1 - [FirebaseFirestore][I-FST000001] (null)
2024-07-02 18:54:23.420373+0930 Pestle[3169:675972] 10.28.1 - [FirebaseFirestore][I-FST000001] (null)
2024-07-02 18:54:23.525642+0930 Pestle[3169:675972] 10.28.1 - [FirebaseFirestore][I-FST000001] (null)
2024-07-02 18:54:23.525884+0930 Pestle[3169:675972] 10.28.1 - [FirebaseFirestore][I-FST000001] (null)
2024-07-02 18:54:23.525964+0930 Pestle[3169:675972] 10.28.1 - [FirebaseFirestore][I-FST000001] (null)
2024-07-02 18:54:23.526146+0930 Pestle[3169:675972] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction Set stream token: 1 changes (11 bytes):
  - Put [mutation_queue: user_id=MYUSERID] (11 bytes)>
2024-07-02 18:54:23.526227+0930 Pestle[3169:675972] 10.28.1 - [FirebaseFirestore][I-FST000001] (null)
2024-07-02 18:54:23.622348+0930 Pestle[3169:675919] 10.28.1 - [FirebaseFirestore][I-FST000001] (null)
2024-07-02 18:54:23.622406+0930 Pestle[3169:675919] 10.28.1 - [FirebaseFirestore][I-FST000001] (null)
2024-07-02 18:54:23.622429+0930 Pestle[3169:675919] 10.28.1 - [FirebaseFirestore][I-FST000001] (null)
2024-07-02 18:54:23.622450+0930 Pestle[3169:675919] 10.28.1 - [FirebaseFirestore][I-FST000001] (null)
2024-07-02 18:54:23.622474+0930 Pestle[3169:675919] 10.28.1 - [FirebaseFirestore][I-FST000001] (null)
2024-07-02 18:54:23.622493+0930 Pestle[3169:675919] 10.28.1 - [FirebaseFirestore][I-FST000001] (null)
2024-07-02 18:54:23.622513+0930 Pestle[3169:675919] 10.28.1 - [FirebaseFirestore][I-FST000001] (null)
2024-07-02 18:54:23.622532+0930 Pestle[3169:675919] 10.28.1 - [FirebaseFirestore][I-FST000001] (null)
2024-07-02 18:54:23.622608+0930 Pestle[3169:675919] 10.28.1 - [FirebaseFirestore][I-FST000001] (null)
2024-07-02 18:54:23.622640+0930 Pestle[3169:675919] 10.28.1 - [FirebaseFirestore][I-FST000001] (null)
2024-07-02 18:54:23.622689+0930 Pestle[3169:675919] 10.28.1 - [FirebaseFirestore][I-FST000001] LocalStore Ignoring outdated update for betaAnnouncements/mailinglist. Current version: Timestamp(seconds=1706179480, nanoseconds=788218000)  Remote version: Timestamp(seconds=1706179480, nanoseconds=788218000)
2024-07-02 18:54:23.622722+0930 Pestle[3169:675919] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction Apply remote event: 5 changes (214 bytes):
  - Put [document_target: path=betaAnnouncements/mailinglist target_id=0] (3 bytes)
  - Put [document_target: path=betaAnnouncements/mailinglist target_id=13996] (0 bytes)
  - Put [target: target_id=13996] (188 bytes)
  - Put [target_document: target_id=13996 path=betaAnnouncements/mailinglist] (0 bytes)
  - Put [target_global:] (23 bytes)>
2024-07-02 18:54:23.622895+0930 Pestle[3169:675919] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction NotifyLocalViewChanges: 0 changes (0 bytes):>
2024-07-02 18:54:23.628326+0930 Pestle[3169:675921] 10.28.1 - [FirebaseFirestore][I-FST000001] (null)
2024-07-02 18:54:23.628370+0930 Pestle[3169:675921] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction Release target: 2 changes (224 bytes):
  - Put [target: target_id=13996] (201 bytes)
  - Put [target_global:] (23 bytes)>
2024-07-02 18:54:23.920254+0930 Pestle[3169:675921] 10.28.1 - [FirebaseFirestore][I-FST000001] (null)
2024-07-02 18:54:23.920382+0930 Pestle[3169:675921] 10.28.1 - [FirebaseFirestore][I-FST000001] (null)
2024-07-02 18:54:24.013282+0930 Pestle[3169:675919] 10.28.1 - [FirebaseFirestore][I-FST000001] (null)
2024-07-02 18:54:24.013460+0930 Pestle[3169:675919] 10.28.1 - [FirebaseFirestore][I-FST000001] (null)
2024-07-02 18:54:24.042494+0930 Pestle[3169:675919] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction Acknowledge batch: 10 changes (61641 bytes):
  - Delete [document_mutation: user_id=MYUSERID path=users/MYUSERID batch_id=1]
  - Delete [document_overlays: user_id=MYUSERID path=users/MYUSERID batch_id=1]
  - Delete [document_overlays_collection_group_index: user_id=MYUSERID collection_group=users batch_id=1 path=users/MYUSERID]
  - Delete [document_overlays_collection_index: user_id=MYUSERID path=users batch_id=1 document_id=MYUSERID]
  - Delete [document_overlays_largest_batch_id_index: user_id=MYUSERID batch_id=1 path=users/MYUSERID]
  - Delete [mutation: user_id=MYUSERID batch_id=1]
  - Put [document_target: path=users/MYUSERID target_id=0] (3 bytes)
  - Put [mutation_queue: user_id=MYUSERID] (13 bytes)
  - Put [remote_document: path=users/MYUSERID] (61625 bytes)
  - Put [remote_document_read_time: path=users snapshot_version=Timestamp(seconds=1719912263, nanoseconds=903104000) document_id=MYUSERID] (0 bytes)>
2024-07-02 18:54:24.044487+0930 Pestle[3169:675919] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction NotifyLocalViewChanges: 0 changes (0 bytes):>
2024-07-02 18:54:24.044963+0930 Pestle[3169:675919] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction NextMutationBatchAfterBatchID: 0 changes (0 bytes):>
2024-07-02 18:54:24.100841+0930 Pestle[3169:675921] 10.28.1 - [FirebaseFirestore][I-FST000001] (null)
2024-07-02 18:54:24.110291+0930 Pestle[3169:675921] 10.28.1 - [FirebaseFirestore][I-FST000001] (null)
2024-07-02 18:54:24.110412+0930 Pestle[3169:675921] 10.28.1 - [FirebaseFirestore][I-FST000001] (null)
2024-07-02 18:54:24.110439+0930 Pestle[3169:675921] 10.28.1 - [FirebaseFirestore][I-FST000001] (null)
2024-07-02 18:54:24.124135+0930 Pestle[3169:675921] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction Apply remote event: 4 changes (61787 bytes):
  - Put [remote_document: path=users/MYUSERID] (61623 bytes)
  - Put [remote_document_read_time: path=users snapshot_version=Timestamp(seconds=1719912263, nanoseconds=903104000) document_id=MYUSERID] (0 bytes)
  - Put [target: target_id=40] (140 bytes)
  - Put [target_global:] (24 bytes)>
2024-07-02 18:54:24.126057+0930 Pestle[3169:675921] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction NotifyLocalViewChanges: 0 changes (0 bytes):>
WillBishop commented 3 months ago

Using .orderBy("dateModified") (which is a FIRTimestamp), returns 40 documents also, even though it's never been used as an orderBy parameter, so I don't think an index for it would exist yet?

tom-andersen commented 3 months ago

Thank you for the logs. This will be difficult to fix without a reproduction.

Some things you might want to try are:

  1. Disable client side indexing.
  2. Clear the cache.

If either of these fix the problem, that will be helpful information as well.

tom-andersen commented 3 months ago

@WillBishop

In addition to above, it is not enough to disableIndexAutoCreation, you will need to deleteAllIndexes.

Again, we would love a reproduction. But feedback on above changes will also be helpful.

WillBishop commented 3 months ago

Disabling client side indexing and deleting the indexes does result in the documents being visible again. Once indexing is renabled, documents are again lost.

When documents are lost, they consistently do not return in future queries. But when indexes are deleted and rebuilt, a different set of documents are lost.

I'd love to provide a reproduction, but given i'm not even sure how the bug is produced I'm sure I'd be able to do that.

An interesting thing I found is I copied all the documents to a new collection and read from that. In that case it was able to read all 47 documents, but as soon as I switched to the original collection it returned only 42 documents.

Using the exact same query if I change it to .getDocuments(source: .server), I still get 42 documents only.

wu-hui commented 3 months ago

Hi @WillBishop

This is useful information. Can you share some of the document content from both the set of missing documents, and the set of documents that actually show up?

WillBishop commented 2 months ago

Privately I can yes, what's the best way to share them with you?

wu-hui commented 2 months ago

Privately I can yes, what's the best way to share them with you?

You can create a private repo and invite me as a collaborator.

WillBishop commented 2 months ago

Done! You'll find it here: https://github.com/WillBishop/MissingDocs

wu-hui commented 2 months ago

Thanks, will take a look and try to reproduce soon.

WillBishop commented 1 month ago

Hey, any update on this?

wu-hui commented 1 month ago

Hi, sorry for the delay. We are overwhelmed by the projects going on within firestore, and i could not find anytime to work on this. That being said, I will try to spend some time to work on this in the next couple weeks.

wu-hui commented 1 month ago

I spent some time trying to reproduce this and failed. I did find something interesting though:

This line: 2024-07-02 18:39:36.008612+0930 Pestle[3064:660932] 10.28.1 - [FirebaseFirestore][I-FST000001] Query: Query(canonical_id=users/MYUSERID/privateRecipes|f:|ob:__name__asc|lt:f), scans 773 local documents and returns 46 documents as results.

Says it is doing a collection scan with no filter/orderBy (other than name which is document id), and there are a total of 773 documents in the SDK cache. Yet, only 46 were returned as result. How can this be? Isn't 773 the expected count?

This triggers the auto indexing process, and leads to your second query with orderBy("name") run with a newly created index, which is missing documents as you see.

So I think there might be at least two issues here, and they both seem to have something to do with the contents in the SDK cache being in a state they are not supposed to be.

In the short term, you can proceed with disabling auto indexing, which should make it possible for you to move forward.

At this point, I'd like to request a minimum reproduction app from you if possible. You can strip out all other logic and only leave this recipe collection to be populated and then queried. We'd be really grateful if you could take the effort to make our product better.

You can update the private repo with your reproducing App. In terms of data, you can create a new free firebase project, and share the googleservices.info file in the private repo as well.

Thanks!