[Bug]: [standalone] search raises error on GPU_IVF_FLAT indexed field with limit>256

[zhanyuan-gacrnd]

Is there an existing issue for this?

• [X] I have searched the existing issues

Environment

- Milvus version: milvusdb/milvus:v2.3.0-gpu
- Deployment mode: standalone 
- MQ type(rocksmq, pulsar or kafka):    
- SDK version(e.g. pymilvus v2.0.0rc2):pymilvus v2.3.0
- OS(Ubuntu or CentOS): Ubuntu
- CPU/Memory: 
- GPU: NVIDIA A100-SXM4-80GB
- Others:

Current Behavior

When I conduct a vector search on GPU_IVF_FLAT indexed field, and set limit > 256, following exception is raised

raise MilvusException(response.status.error_code, response.status.reason) pymilvus.exceptions.MilvusException: <MilvusException: (code=1, message=attempt #0: fail to Search, QueryNode ID=11, reason=worker(11) query failed: UnexpectedError: Assert "failed to search: out of range in json: param k out of range [ 1,256 ]" at /go/src/github.com/milvus-io/milvus/internal/core/src/index/VectorMemIndex.cpp:319: channel=by-dev-rootcoord-dml_7_444090662168255959v0: fail to search/query shard leader: attempt #1: no available shard delegator found: service unavailable: fail to search on all shard leaders)>

Expected Behavior

When I conduct a vector search on GPU_IVF_FLAT indexed field, and set limit > 256, search result should return

Steps To Reproduce

No response

Milvus Log

No response

Anything else?

No response

[yanliang567]

/assign @liliu-z is it a limit for now?

/unassign

[liliu-z]

Hi @zhanyuan-gacrnd Due to RAFT's limit, we don't support GPU_IVF_FLAT's k needs to be smaller than 256 and GPU_IVF_PQ's k needs to be smaller than 8192. https://milvus.io/docs/index.md#GPU_IVF_FLAT We are working on removing this limit and hopefully can fix this in the following versions of 2.3.x