pinecone-io / pinecone-python-client

The Pinecone Python client
https://www.pinecone.io/docs
Apache License 2.0
309 stars 80 forks source link

Implement query_namespaces over grpc #416

Closed jhamon closed 1 week ago

jhamon commented 1 week ago

Problem

I want to maintain parity across REST / GRPC implementations. This PR adds a query_namespaces implementation for the GRPC index client.

Solution

Use a ThreadPoolExecutor to execute queries in parallel, then aggregate the results QueryResultsAggregator.

Usage

import random
from pinecone.grpc import PineconeGRPC

pc = PineconeGRPC(api_key="key")
index = pc.Index(host="jen1024-dojoi3u.svc.apw5-4e34-81fa.pinecone.io", pool_threads=25)

query_vec = [random.random() for i in range(1024)]
combined_results = index.query_namespaces(
    vector=query_vec,
    namespaces=["ns1", "ns2", "ns3", "ns4"],
    include_values=False,
    include_metadata=True,
    filter={"genre": {"$eq": "drama"}},
    top_k=50
)

for vec in combined_results.matches:
    print(vec.get('id'), vec.get('score'))
print(combined_results.usage)

Type of Change

Test Plan

Describe specific steps for validating this change.