Error while - Querying the index -Vector Search with Firestore

[devopsevd]

[REQUIRED] Step 2: Describe your configuration

• Extension name: _ googlecloud/firestore-vector-search@0.0.3
• Extension version: _ 0.0.3
• Configuration values:
  • _ collection path: testembed
  • _ Default query limit(Optional): 1
  • _ Distance measure COSINE
  • _ Input field name: input
  • _ Output field name: embedding
  • _ Status field name: status
  • _ Cloud Functions location: us-central1

[REQUIRED] Step 3: Describe the problem

Receiving an Error while querying the index.

error: '{"error":{"message":"The function must be called while authenticated.","status":"UNAUTHENTICATED"}}' executionId: x96sgykheo49

Steps to reproduce:

As per the instructions provided in the How to use this extension section on the extension

Querying the index *Important: Before you will be able to query the collection, firestore will have to build a vector index. This extension will trigger the basic index (no prefilters) upon installation or reconfiguration.

Once the index is created, you may query it either through a callable cloud function deployed by the extension:

gcloud functions --project examples-c0581 call ext-firestore-vector-search-queryCallable --data '{"data": {"query":"test"}}' Or by adding a document to the collection _firestore-vector-search/index/queries with a query field and an (optional) limit field.

Expected result

Expected a query result for the searched query - "test"

Actual result

Received error using both the approaches 1) gcloud functions --project examples-c0581 call ext-firestore-vector-search-queryCallable --data '{"data": {"query":"test"}}' error: '{"error":{"message":"The function must be called while authenticated.","status":"UNAUTHENTICATED"}}' executionId: x96sgykheo49 2) creating a query field under the collection _firestore-vector-search/index/queries query "test"(string) status(map) textQuery(map) createTime 11 May 2024 at 21:37:58 UTC-4 (timestamp) startTime 11 May 2024 at 21:38:03 UTC-4 (timestamp) state "ERROR" (string) updateTime 11 May 2024 at 21:38:07 UTC-4

[devopsevd]

I had to manually create an index using the following command, and the approach with creating a query field under the collection _firestore-vector-search/index/queries started working. gcloud alpha firestore indexes composite create --project=examples-c0581 --collection-group=testembed --query-scope=COLLECTION --field-config=vector-config='{"dimension":"768","flat": "{}"}',field-path=embedding

However the first approach with queryCallable still throws the error. "The function must be called while authenticated."

[karloti]

I found some explanation of the problem! https://cloud.google.com/functions/docs/securing/managing-access-iam#allowing_unauthenticated_function_invocation

[cabljac]

Hm so there seem to be two issues here, the index isn't being created initially, and then the docs are definitely missing the authentication instructions (the function does require some kind of auth)

Looking into this today

[cabljac]

OK it's actually quite annoying. We need to authenticate through firebase and get an id token if we want to call the function through the CLI.

I should update the docs to reflect this, and provide a different snippet.

I haven't been able to reproduce the other issue so far. Do you have any relevant logs of the backfillTrigger function deployed? - i have reproduced this now.

[cabljac]

Were you using Vertex AI for embeddings by any chance? I think I've found the index problem