firebase / firebase-admin-node

Firebase Admin Node.js SDK
https://firebase.google.com/docs/admin/setup
Apache License 2.0
1.6k stars 359 forks source link

Firebase query orderByKey().limitToFirst(n).startAfter() returns n-1 items iso n #2350

Open timalenustmf opened 8 months ago

timalenustmf commented 8 months ago

[READ] Step 1: Are you in the right place?

yes

[REQUIRED] Step 2: Describe your environment

[REQUIRED] Step 3: Describe the problem

Firebase query of type: admin.database.Database.ref('collection').orderByKey().limitToFirst(100).startAfter(lastKey) only returns 99 items iso 100 even if more keys are in the firebase realtime database.

On version 11.4.1 of firebase-admin this query behaves as expected and 100 items are returned.

Steps to reproduce:

What happened? How can we make the problem occur? This could be a description, log/console output, etc.

Relevant Code:

let query;
let lastDocument = null;
let lastBatchSize: number = 100;

while (lastBatchSize === 100) {
    query = database.ref('collection').orderByKey()
      .limitToFirst(100);

    if (lastDocument !== null) {
      query = query.startAfter(lastDocument.key);
    }

    const snapshot = await query.get();
    const keys = Object.keys(snapshot.val() || {});
    const docs = keys.map((key) => snapshot.child(key));

    if (docs.length === 0) break;
    console.log("docs length " + docs.length); //prints 99 on 11.11.0, prints 100 on 11.4.1

    if (docs.length) {
      lastDocument = docs[docs.length - 1];
    }
    lastBatchSize = docs.length;
  }
google-oss-bot commented 8 months ago

I couldn't figure out how to label this issue, so I've labeled it for a human to triage. Hang tight.

willemjanvankranenburg commented 3 weeks ago

We experience the same problem, but not on every firebase environment.

Every environment we have has the exact same configuration and same indexes, but if we run a script with the same query on one environment it returns exactly the limit, but on other environments it returns limit-1.

We use firebase-admin 11.8.0 and nodejs 16