milvus-io / milvus-sdk-node

The Official Mivus node.js sdk(client)
https://milvus.io
Apache License 2.0
126 stars 39 forks source link

Support for group_by_field in Node SDK Search Function, not working #367

Open mariagq777 opened 4 days ago

mariagq777 commented 4 days ago

Hi Milvus SDK Team,

I’m experiencing an issue with the milvus-sdk-node when using the group_by_field parameter in a search query. Here’s a summary of the issue and the steps I’ve followed:

Expected Behavior In Python, the search query works perfectly using group_by_field to group results by a specific field. Here's the Python code that functions as expected:

res = client.search(
  collection_name="JURISPRUDENCIA",
  data=[query_vector],
  search_params={"metric_type": "COSINE", "params": {"level": 1}},
  limit=30,
  output_fields=["_id", "mongoId", "text", "year", "matter", "unixTime", "province"],
  consistency_level="Strong",
  group_by_field="mongoId"
)

In this Python query, setting group_by_field="mongoId" groups the results by the mongoId field without any issues.

Issue in Node.js SDK Attempting to perform the equivalent operation in the Node.js SDK results in an error message. Here’s the Node.js code I’m using:

const params = {
  collection_name: "JURISPRUDENCIA",
  output_fields: [
    "_id",
    "mongoId",
    "text",
    "year",
    "matter",
    "unixTime",
    "province",
  ],
  limit: 30,
  data: [
    {
      anns_field: "vector",
      data: embedding,
      params: {
        level: 1,
      },
    },
  ],
  filter: "",
  consistency_level: "Bounded",
  group_by_field: "mongoId",
};

const result = await milvusClient.search(params);
console.log(result);

Error Message When running the above code, I receive the following error message:

{
  status: {
    extra_info: {},
    error_code: 'UnexpectedError',
    reason: 'not support search_group_by operation in the hybrid search',
    code: 65535,
    retriable: false,
    detail: 'not support search_group_by operation in the hybrid search'
  },
  results: []
}

Investigation & Findings The error disappears if I remove group_by_field="mongoId". This suggests that the group_by_field parameter is not supported in the same way within the Node.js SDK.

I’ve tested this using text embedding vectors, including text-embedding-3-small and text-embedding-3-large, but the issue persists across embedding sizes.

Steps to Reproduce Use the Node.js SDK to create a search query with the group_by_field parameter set. Run the query and observe the error message. Remove the group_by_field parameter to verify that the query executes without errors.

Milvus-node-sdk version: "@zilliz/milvus2-sdk-node": "^2.4.9"

Milvus version: milvus-standalone, milvus:v2.4.12,

Thank you for your help and for your great work with Milvus!

Pablo

shanghaikid commented 6 hours ago

I will check this soon, it maybe a bug.

shanghaikid commented 6 hours ago

If you put an search object in the data, the sdk wil do an hybrid search.

I think you should use this format to search.

const params = {
  collection_name: "JURISPRUDENCIA",
  output_fields: [
    "_id",
    "mongoId",
    "text",
    "year",
    "matter",
    "unixTime",
    "province",
  ],
  limit: 30,
  data: embedding,
  params: {
    level: 1,
  },
  filter: "",
  consistency_level: "Bounded",
  group_by_field: "mongoId",
};
shanghaikid commented 6 hours ago

I will improve compatibility further later.