search

googleapis / nodejs-firestore

Node.js client for Google Cloud Firestore: a NoSQL document database built for automatic scaling, high performance, and ease of application development.
https://cloud.google.com/firestore/
Apache License 2.0
636 stars 147 forks source link

Error getting documents: Error: 9 FAILED_PRECONDITION when using collectionGroup #1866

Open dumitruPuggle opened 11 months ago

dumitruPuggle commented 11 months ago

Environment details

Steps to reproduce

  1. Use any collectionGroup and call get function 
    db.collectionGroup("textTemplates")
.where("id", "==", templateId)
.get()
  2. You'll receive an error: 
    Google API requested!
- URL: "https://oauth2.googleapis.com/token"
- Be careful, this may be a production service.
>  Error getting documents:  Error: 9 FAILED_PRECONDITION: 
>      at callErrorFromStatus (/Users/dumitrucucu/Desktop/VRSEN/admagic-ai/back/functions/node_modules/@grpc/grpc-js/build/src/call.js:31:19)
>      at Object.onReceiveStatus (/Users/dumitrucucu/Desktop/VRSEN/admagic-ai/back/functions/node_modules/@grpc/grpc-js/build/src/client.js:357:73)
>      at Object.onReceiveStatus (/Users/dumitrucucu/Desktop/VRSEN/admagic-ai/back/functions/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:323:181)
>      at /Users/dumitrucucu/Desktop/VRSEN/admagic-ai/back/functions/node_modules/@grpc/grpc-js/build/src/resolving-call.js:94:78
>      at process.processTicksAndRejections (node:internal/process/task_queues:77:11)
>  for call at
>      at ServiceClientImpl.makeServerStreamRequest (/Users/dumitrucucu/Desktop/VRSEN/admagic-ai/back/functions/node_modules/@grpc/grpc-js/build/src/client.js:340:32)
>      at ServiceClientImpl.<anonymous> (/Users/dumitrucucu/Desktop/VRSEN/admagic-ai/back/functions/node_modules/@grpc/grpc-js/build/src/make-client.js:105:19)
>      at /Users/dumitrucucu/Desktop/VRSEN/admagic-ai/back/functions/node_modules/@google-cloud/firestore/build/src/v1/firestore_client.js:227:29
>      at /Users/dumitrucucu/Desktop/VRSEN/admagic-ai/back/functions/node_modules/@google-cloud/firestore/node_modules/google-gax/build/src/streamingCalls/streamingApiCaller.js:38:28
>      at /Users/dumitrucucu/Desktop/VRSEN/admagic-ai/back/functions/node_modules/@google-cloud/firestore/node_modules/google-gax/build/src/normalCalls/timeout.js:44:16
>      at Object.request (/Users/dumitrucucu/Desktop/VRSEN/admagic-ai/back/functions/node_modules/@google-cloud/firestore/node_modules/google-gax/build/src/streamingCalls/streaming.js:130:40)
>      at makeRequest (/Users/dumitrucucu/Desktop/VRSEN/admagic-ai/back/functions/node_modules/retry-request/index.js:141:28)
>      at retryRequest (/Users/dumitrucucu/Desktop/VRSEN/admagic-ai/back/functions/node_modules/retry-request/index.js:109:5)
>      at StreamProxy.setStream (/Users/dumitrucucu/Desktop/VRSEN/admagic-ai/back/functions/node_modules/@google-cloud/firestore/node_modules/google-gax/build/src/streamingCalls/streaming.js:121:37)
>      at StreamingApiCaller.call (/Users/dumitrucucu/Desktop/VRSEN/admagic-ai/back/functions/node_modules/@google-cloud/firestore/node_modules/google-gax/build/src/streamingCalls/streamingApiCaller.js:54:16)
>  Caused by: Error
>      at Query._get (/Users/dumitrucucu/Desktop/VRSEN/admagic-ai/back/functions/node_modules/@google-cloud/firestore/build/src/reference.js:1717:23)
>      at Query.get (/Users/dumitrucucu/Desktop/VRSEN/admagic-ai/back/functions/node_modules/@google-cloud/firestore/build/src/reference.js:1705:21)
>      at startTextSequencesHandler (/Users/dumitrucucu/Desktop/VRSEN/admagic-ai/back/functions/lib/functions/callable/start-text-sequences.js:25:10)
>      at fixedLen (/Users/dumitrucucu/Desktop/VRSEN/admagic-ai/back/functions/node_modules/firebase-functions/lib/v2/providers/https.js:118:31)
>      at /Users/dumitrucucu/Desktop/VRSEN/admagic-ai/back/functions/node_modules/firebase-functions/lib/common/providers/https.js:467:32
>      at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
>      at async runFunction (/Users/dumitrucucu/.nvm/versions/node/v18.15.0/lib/node_modules/firebase-tools/lib/emulator/functionsEmulatorRuntime.js:506:9)
>      at async runHTTPS (/Users/dumitrucucu/.nvm/versions/node/v18.15.0/lib/node_modules/firebase-tools/lib/emulator/functionsEmulatorRuntime.js:531:5)
>      at async /Users/dumitrucucu/.nvm/versions/node/v18.15.0/lib/node_modules/firebase-tools/lib/emulator/functionsEmulatorRuntime.js:693:21 {
>    code: 9,
>    details: '',
>    metadata: Metadata { internalRepr: Map(0) {}, options: {} }
>  }

I created the necessary indexes on Firestore Console:

Screenshot 2023-07-25 at 17 42 50

Making sure to follow these steps will guarantee the quickest resolution possible.

Thanks!

tom-andersen commented 11 months ago

I suspect you are using OAuth 2.0.

See: https://firebase.google.com/docs/admin/setup

Note: OAuth 2.0 refresh tokens are not supported for connecting to Cloud Firestore.

dumitruPuggle commented 11 months ago

I found the issue. All I had to do is to use orderBy, and after that I received an index link for firestore console.

diginikkari commented 11 months ago

I think that the main issue here is that the FAILED_PRECONDITION error is not including create query index -link.

leonardopn commented 10 months ago

My firebase-admin: ^11.10.1

Same problem running this code snippet in a deployed code:

static async getAllByIdProduct(id: string) {
    const query = await firestore
        .collectionGroup("modelos")
        .where("idsDeProduto", "array-contains", id)
        .get();

    return query.docs.map(doc => ({
        data: doc.data() as RoteiroModelDTO,
        ref: doc.ref as DocumentReference<RoteiroModelDTO>,
    }));
}

Error in GCP:

image

It really seems that the link to create the indexes should come, but it's just showing a generic error

dconeybe commented 10 months ago

Thank you for this additional information. @tom-andersen will follow up early next week.

tom-andersen commented 10 months ago

Update:

@leonardopn I verified the server logs, and the error message describing the problem along with link to create index was provided. But somewhere from the time that response is sent to cloud function and the output being logged, the error message is being stripped out. I see no reason for why the SDK should do this, so I wonder if there is something in the logging setup that causes this.

If you run the query locally on your development machine, I imagine the error with link should appear.

I will see if I can reproduce error with cloud function.

dumitruPuggle commented 9 months ago

Hi guys

Was this issue resolved? I'm asking because I'm sill getting this error in production.

Screenshot 2023-09-19 at 23 02 31
tom-andersen commented 9 months ago

Hi @dumitruPuggle

We still don't know why the text is stripped from the error. However, the resolution remains the same. Create the required composite index. For now you will need to run the code locally (or not within cloud function), and then there will be a helpful link as part of error.

ivanmartinvalle commented 8 months ago

Running into this also. As an alternative to running the query locally, I was able to get the index creation link by using the GCP Console with the Query Builder. But even if the link wasn't generated via that method, it does allow you to quickly test if a particular query would be supported, and then manually go through the process of adding the index in the GCP Console.

dumitruPuggle commented 8 months ago

Hey guys,

I found an alternative solution to fix this problem. Here are the steps:

Screenshot 2023-11-05 at 00 27 30

  1. Go to Firestore

  2. Click on Query builder tab

  3. Replicate the same query through UI:

    Screenshot 2023-11-05 at 00 28 10

That's it. You get the index link in console.

alexstanbury commented 6 months ago

This is absolutely ridiculous, you can't warn us when developing and you 'don't know why the text is stripped from the error', so we can't even set up a proper alert in case a missing index gets through to production. Come on Google, do better.

jelling commented 2 months ago

I was referred here by GCP support - which we pay for - so it's extra frustrating to see that a repeatable issue affecting basic functionality is known but receiving no attention. GCP support also referred me to this likely related issue that has been open since 2022.

It is impossible for me to believe that with all the smart people I know at Google that no one can figure out why an error message is getting truncated. So when can we expect a resolution for this?

I know no one the support team is going to lose sleep if we leave Firebase, but we have had so many similar Firebase support experiences - long open issues with basic functionality - that it's increasingly hard to believe that GCP sees Firebase as a priority.

kossnocorp commented 2 months ago

I believe that the issue should receive higher priority. It seems to have slipped through the cracks, and the team isn't really aware of the problem.

Not being able to receive the index link makes the collection group's DX hellish. Let's alone say we have to deploy to production and hope to catch the errors before users do, but we also have to guess which query is failing.

@MarkDuckworth forgive me for pinging you. I can see your commits in the main branch, so hopefully, you can access the actual issue priority.

tom-andersen commented 2 months ago

Hi @kossnocorp,

I am trying to move this forward, since this has somewhat fallen between teams. So here are some updates:

  1. Logging is not the cause. The text message is stripped before exception is emitted as part of Firebase Function.
  2. I have produced a reproduction and shared it with Firebase Functions. They will investigate whether this happens as part of Firebase Function or further back in the system under a Cloud Function. Someone from FIrebase Functions will investigate further.

When we know more, I will update this issue.

jelling commented 2 months ago

I’m fairly certain this is not related to Firebase functions as I am having the problem while using Next.JS to query Firestore.

On Tue, Apr 30, 2024 at 12:30 PM Tom Andersen @.***> wrote:

Hi @kossnocorp https://github.com/kossnocorp,

I am trying to move this forward, since this has somewhat fallen between teams. So here are some updates:

  1. Logging is not the cause. The text message is stripped before exception is emitted as part of Firebase Function.
  2. I have produced a reproduction and shared it with Firebase Functions. They will investigate whether this happens as part of Firebase Function or further back in the system under a Cloud Function. Someone from FIrebase Functions will investigate further.

When we know more, I will update this issue.

— Reply to this email directly, view it on GitHub https://github.com/googleapis/nodejs-firestore/issues/1866#issuecomment-2086146218, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAOMZ7SGJKLDDBK5SJBMESDY77IMXAVCNFSM6AAAAAA2XF5LCSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAOBWGE2DMMRRHA . You are receiving this because you commented.Message ID: @.***>

tom-andersen commented 2 months ago

This problem sometimes occurs in NodeJS 18 and 20, but not 16 from our observation.

For Cloud Functions and Firebase Functions, reducing the engine to NodeJS version 16 removes the problem.

sirusbaladi commented 1 month ago

orderBy

this fixed it for me.