Firestore Persistence Loading Speed

[scottfr]

Describe your environment

• Operating System version: OS X
• Browser version: 79
• Firebase SDK version: 7.6.1
• Firebase Product: firestore

Describe the problem

I've been testing enabling Firestore persistence and loading persistence enabled is actually noticeably slower than loading the remote data without persistence.

Looking at the profiler, it appears that the persistence data loading is broken up into small pieces with delays in between them. Here is a small snippet of a flame chart during load:

Each of those "flames" is a Firestore process, but there are large gaps between them and the app is spending a lot of time waiting. This screenshot is only about 2ms of time, but the same pattern goes on for seconds during load.

Is there some way we can speed this up? Either on our end or in the Firestore library?

[var-const]

Apologies for the slow reply. Could you please provide some additional information to help us better understand the issue?

• what kind of a query are you running?
• how many documents are you loading? What's the rough average size of the documents?

Anything that would help us reproduce the issue you're seeing would be great. If you can post some code snippets, it would be very helpful.

[google-oss-bot]

Hey @scottfr. We need more information to resolve this issue but there hasn't been an update in 7 days. I'm marking the issue as stale and if there are no new updates in the next 3 days I will close it automatically.

If you have more information that will help us get to the bottom of this, just add a comment!

[scottfr]

My loading case is fairly complex, and I haven't managed to create a simple repro that duplicates what I am seeing.

The sawtooth loading pattern is straightforward to reproduce though. To do so:

1) Initialize the data:

firebase.initializeApp({
  projectId: '### PROJECT ID ####'
});

var db = firebase.firestore();
db.enablePersistence({
  synchronizeTabs: true
});

for (let i =0; i< 400; i++) {
db.collection("users").add({
  first: "Ada",
  last: "Lovelace",
  born: 1815 * Math.random(),
  permissions: {'u123': Math.random() > .5 ? 'owner': 'none'}
})

2) Then load the data and check the performance (note you should load twice to make sure you get the cached data):

firebase.initializeApp({
  projectId: '### PROJECT ID ####'
});

var db = firebase.firestore();
db.enablePersistence({
  synchronizeTabs: true
});

async function load() {
  console.log((await db.collection('users').get()).docs);
}

load();

Performance zoomed out:

Zoomed in:

It appears to be primarily stepping between iterations of "t.onsuccess @ simple_db.ts:704".

In this simpler example, it seems like the remote and local persistence takes roughly the same time in total.

[filipesilva]

Heya, I've also noticed that having persistence on seems to slow down writes.

I've setup a repro that tries to isolate that as much as possible. It uses the firebase emulators and clears both the browser indexedDB and the emulator database on each run.

git clone https://github.com/filipesilva/firestore-persistence-perf
cd firestore-persistence-perf
yarn
yarn emulators

Go to http://localhost:5000/ and you can see the average time per write and total time in the console. You can edit public/index.html to change things.

Without persistence I see these numbers:

write average 9103.647349992534
total time 18868.66000003647

With persistence I see these numbers:

write average 12344.25214999821
total time 24767.469999962486

So around 1.5x. Not as big of a difference as I thought for the initial writes.

I don't know how much emulator usage is a factor though.

[wu-hui]

Hi scottfr,

Thanks for doing profiling. Unfortunately, I don't have any advice to give as what application does do not affect this behavior. I am looking into ways to improve this, but nothing concrete yet.

Hi filipesilva,

If I am not mistaken, you are bench-marking the write performance. It is actually not too surprising that persistence writes are slower than without persistence, because it needs to hit the actual local storage while without persistence, everything is in memory. If write performance is critical to you, and you don't need persistence (eg, if it's OK user writes will be missing if they close the tab too soon, or if it's OK your app does not work with multi-tab), you could just go without persistence.

Thanks.

[jorroll]

We've also ran into this issue in our internal company messaging application. It got to be such a problem that we've disabled Firestore persistence. If it's helpful, here's a video showcasing the performance impact we saw.

• https://www.loom.com/share/81a7023707de4c90bdbdb8079ad27f0b

Note, I have no plans to create a full reproduction at this time.

[wu-hui]

@jorroll

It'd be helpful if you can give some descriptions of how many documents in the collection you try to query, when you do the operation in the demo video. That would help us understand the usage pattern. If your use case fits, you could consider turning on client (cache) index: https://firebase.google.com/docs/reference/js/firestore_.md#setindexconfiguration

Note this is a preview feature, and might change going forward. But it'd be nice to know if indexing helps.

IndexedDB with Chrome is generally slower, you could also try this from Safari, which should give you a better performance (not necessarily acceptable).

[google-oss-bot]

Hey @scottfr. We need more information to resolve this issue but there hasn't been an update in 5 weekdays. I'm marking the issue as stale and if there are no new updates in the next 5 days I will close it automatically.

If you have more information that will help us get to the bottom of this, just add a comment!

[google-oss-bot]

Since there haven't been any recent updates here, I am going to close this issue.

@scottfr if you're still experiencing this problem and want to continue the discussion just leave a comment here and we are happy to re-open this.