milvus-io / milvus

A cloud-native vector database, storage for next generation AI applications
https://milvus.io
Apache License 2.0
29.61k stars 2.84k forks source link

[Bug]: Sometines count(*) with expr array_contains_any gets unexpected result #36363

Open ThreadDao opened 1 week ago

ThreadDao commented 1 week ago

Is there an existing issue for this?

Environment

- Milvus version:2.4-20240919-e66beb7a-amd64
- Deployment mode(standalone or cluster):cluster
- MQ type(rocksmq, pulsar or kafka):    
- SDK version(e.g. pymilvus v2.0.0rc2):
- OS(Ubuntu or CentOS): 
- CPU/Memory: 
- GPU: 
- Others:

Current Behavior

  1. Create a collection with float32 array field
  2. insert array data like this with start :=0, nb:= 3000, capacity := 100
    case entity.FieldTypeFloat:
        floatValues := make([][]float32, 0, nb)
        for i := start; i < start+nb; i++ {
            floatArray := make([]float32, 0, capacity)
            for j := 0; j < capacity; j++ {
                floatArray = append(floatArray, float32(i+j))
            }
            floatValues = append(floatValues, floatArray)
  3. query count(*) Strong consistency_level with expr array_contains_any (floatArray, [0, 100, 10000]) and expect to get count: 101 but gets 57 collection_name: rjUJ
    2024/09/19 17:07:09 query_test.go:862: array_contains_any (floatArray, [0, 100, 10000])
    2024/09/19 17:07:09 milvus_client.go:14: (ApiRequest): func [Query], args: [context.Background.WithDeadline(2024-09-19 17:08:31.254002988 +0800 CST m=+120.002914854 [1m22.202364572s]) rjUJ [] array_contains_any (floatArray, [0, 100, 10000]) [count(*)] []]
    2024/09/19 17:07:12 milvus_client.go:21: (ApiResponse): func [Query], results: [[0xc01f8321e0]]
    2024/09/19 17:07:12 query_test.go:865: type:Int64 field_name:"count(*)" scalars:<long_data:<data:57 > > 
    query_test.go:867: 
            Error Trace:    /home/zong/zong/go-projects/milvus-sdk-go/test/testcases/query_test.go:867
            Error:          Not equal: 
                            expected: 101
                            actual  : 57
            Test:           TestQueryArrayFieldExpr

Expected Behavior

go sdk Test cases: TestQueryArrayFieldExpr https://github.com/milvus-io/milvus-sdk-go/blob/v2.4.x/test/testcases/query_test.go#L771

run case:

go test -v search_test.go main_test.go -test.run TestQueryArrayFieldExpr --addr="xxx:19530"

Steps To Reproduce

No response

Milvus Log

pods:

output-dynamic-1-milvus-datanode-7fdfd947b-8rtjj                  1/1     Running            0                169m    10.104.4.254    4am-node11   <none>           <none>
output-dynamic-1-milvus-indexnode-6ffb5bf4dd-qbbg4                1/1     Running            0                174m    10.104.18.11    4am-node25   <none>           <none>
output-dynamic-1-milvus-mixcoord-555477c6cd-df6vr                 1/1     Running            0                172m    10.104.19.26    4am-node28   <none>           <none>
output-dynamic-1-milvus-proxy-86f865df6b-7bfdp                    1/1     Running            0                167m    10.104.17.58    4am-node23   <none>           <none>
output-dynamic-1-milvus-querynode-0-7855fcf455-bb88t              1/1     Running            0                170m    10.104.25.114   4am-node30   <none>           <none>
output-dynamic-1-milvus-querynode-0-7855fcf455-pchxc              1/1     Running            0                171m    10.104.21.114   4am-node24   <none>           <none>

Anything else?

No response

yanliang567 commented 6 days ago

/assign @xiaocai2333 sounds like a array_contains_any issue. /unassign

xiaocai2333 commented 3 days ago

maybe it was handoffing. I will check it.

xiaocai2333 commented 1 day ago

I can't reproduce it, and can not find the commit in master branch.

xiaocai2333 commented 1 day ago

maybe you used a wrong image.

xiaocai2333 commented 1 day ago

/assign @ThreadDao please update new case when it reproduce.