search

googleapis / google-cloud-go

Google Cloud Client Libraries for Go.
https://cloud.google.com/go/docs/reference
Apache License 2.0
3.72k stars 1.28k forks source link

firestore: Add support for vector search in golang cloud firestore client #10219

Closed alexu84 closed 2 months ago

alexu84 commented 4 months ago

Is your feature request related to a problem? Please describe. Currently, the Golang Cloud Firestore client does not support vector stores or vector searches natively. This feature is essential for various applications, such as recommendation systems and similarity searches. Adding this support would streamline the process and remove the need for additional infrastructure or services to handle vector operations.

Describe the solution you'd like I would like the Golang Cloud Firestore client to provide native support for vector stores and vector searches. This would include:

  1. API Methods to store and retrieve vectors.
  2. A query method similar to QueryDocuments that allows searching for documents by their vector similarity.
  3. Efficient storage and retrieval mechanisms for vectors to ensure good performance.

Example usage:

// Storing a vector
err := firestoreClient.StoreVector(ctx, documentID, myVector)

// Searching by vector similarity
results, err := firestoreClient.SearchByVector(ctx, queryVector, topN)

Describe alternatives you've considered One alternative is to use third-party libraries or services that offer vector storage and search capabilities, such as FAISS or Elasticsearch. However, this requires additional maintenance and integration efforts, which could be avoided with native support in the Cloud Firestore client.

Additional context Vector search capability is becoming increasingly important in various fields, such as machine learning, AI, and data analysis. Providing this feature natively in the Golang Cloud Firestore client would significantly enhance its applicability and ease of use for developers working on such projects.

wjkoh commented 3 months ago

+1. https://firebase.google.com/docs/firestore/vector-search seems very promising to me!

wjkoh commented 3 months ago

FindNearest has been added to firestore/apiv1/firestorepb/query.pb.go in the main branch. Can we release an update to cloud.google.com/go/firestore, which was last published in March 2024? This would allow users to directly call the gRPC function and use Firestore vector search in Go.

bhshkh commented 2 months ago

Vector search support is being added thru https://github.com/googleapis/google-cloud-go/pull/10548

bhshkh commented 2 months ago

Released

https://pkg.go.dev/cloud.google.com/go/firestore@v1.16.0#Query.FindNearest