search

milvus-io / milvus

A cloud-native vector database, storage for next generation AI applications
https://milvus.io
Apache License 2.0
30.36k stars 2.91k forks source link

[Bug]: [benchmark] query primary key field which build autoindex raises error `Reverse_Lookup should not be handled by inverted index` #35946

Closed wangting0128 closed 2 months ago

wangting0128 commented 2 months ago

Is there an existing issue for this?

Environment

- Milvus version:master-20240902-325f1987-amd64
- Deployment mode(standalone or cluster):cluster
- MQ type(rocksmq, cluster or kafka): cluster   
- SDK version(e.g. pymilvus v2.0.0rc2):2.5.0rc36
- OS(Ubuntu or CentOS): 
- CPU/Memory: 
- GPU: 
- Others:

Current Behavior

server:

NAME                                                              READY   STATUS      RESTARTS        AGE     IP              NODE         NOMINATED NODE   READINESS GATES
bitmap-pk-int64-etcd-0                                            1/1     Running     0               8h      10.104.32.100   4am-node39   <none>           <none>
bitmap-pk-int64-etcd-1                                            1/1     Running     0               8h      10.104.16.82    4am-node21   <none>           <none>
bitmap-pk-int64-etcd-2                                            1/1     Running     0               8h      10.104.21.58    4am-node24   <none>           <none>
bitmap-pk-int64-milvus-datanode-78d7fc5f59-w6dpm                  1/1     Running     2 (8h ago)      8h      10.104.5.234    4am-node12   <none>           <none>
bitmap-pk-int64-milvus-indexnode-64ff5d8c94-4xlzq                 1/1     Running     2 (8h ago)      8h      10.104.4.232    4am-node11   <none>           <none>
bitmap-pk-int64-milvus-indexnode-64ff5d8c94-j88xb                 1/1     Running     2 (8h ago)      8h      10.104.13.49    4am-node16   <none>           <none>
bitmap-pk-int64-milvus-indexnode-64ff5d8c94-m4fn6                 1/1     Running     1 (8h ago)      8h      10.104.1.94     4am-node10   <none>           <none>
bitmap-pk-int64-milvus-indexnode-64ff5d8c94-qbb4k                 1/1     Running     2 (8h ago)      8h      10.104.9.229    4am-node14   <none>           <none>
bitmap-pk-int64-milvus-mixcoord-7cb6d646fc-zfgj5                  1/1     Running     1 (8h ago)      8h      10.104.6.38     4am-node13   <none>           <none>
bitmap-pk-int64-milvus-proxy-7845d75bb7-8jpmh                     1/1     Running     1 (8h ago)      8h      10.104.6.40     4am-node13   <none>           <none>
bitmap-pk-int64-milvus-querynode-578599654d-gwjxs                 1/1     Running     2 (8h ago)      8h      10.104.17.141   4am-node23   <none>           <none>
bitmap-pk-int64-milvus-querynode-578599654d-h95z8                 1/1     Running     2 (8h ago)      8h      10.104.6.39     4am-node13   <none>           <none>
bitmap-pk-int64-milvus-querynode-578599654d-wmd7c                 1/1     Running     2 (8h ago)      8h      10.104.20.5     4am-node22   <none>           <none>
bitmap-pk-int64-minio-0                                           1/1     Running     0               8h      10.104.32.98    4am-node39   <none>           <none>
bitmap-pk-int64-minio-1                                           1/1     Running     0               8h      10.104.16.80    4am-node21   <none>           <none>
bitmap-pk-int64-minio-2                                           1/1     Running     0               8h      10.104.21.54    4am-node24   <none>           <none>
bitmap-pk-int64-minio-3                                           1/1     Running     0               8h      10.104.25.239   4am-node30   <none>           <none>
bitmap-pk-int64-pulsar-bookie-0                                   1/1     Running     0               8h      10.104.32.99    4am-node39   <none>           <none>
bitmap-pk-int64-pulsar-bookie-1                                   1/1     Running     0               8h      10.104.16.81    4am-node21   <none>           <none>
bitmap-pk-int64-pulsar-bookie-2                                   1/1     Running     0               8h      10.104.21.59    4am-node24   <none>           <none>
bitmap-pk-int64-pulsar-bookie-init-xsnmw                          0/1     Completed   0               8h      10.104.17.142   4am-node23   <none>           <none>
bitmap-pk-int64-pulsar-broker-0                                   1/1     Running     0               8h      10.104.13.52    4am-node16   <none>           <none>
bitmap-pk-int64-pulsar-proxy-0                                    1/1     Running     0               8h      10.104.1.95     4am-node10   <none>           <none>
bitmap-pk-int64-pulsar-pulsar-init-46pwc                          0/1     Completed   0               8h      10.104.9.228    4am-node14   <none>           <none>
bitmap-pk-int64-pulsar-recovery-0                                 1/1     Running     0               8h      10.104.13.51    4am-node16   <none>           <none>
bitmap-pk-int64-pulsar-zookeeper-0                                1/1     Running     0               8h      10.104.32.97    4am-node39   <none>           <none>
bitmap-pk-int64-pulsar-zookeeper-1                                1/1     Running     0               8h      10.104.16.84    4am-node21   <none>           <none>
bitmap-pk-int64-pulsar-zookeeper-2                                1/1     Running     0               8h      10.104.25.241   4am-node30   <none>           <none>

client log:

[2024-09-03 12:21:30,499 - DEBUG - fouram]: (api_request)  : [Collection.query] args: ['(id % 10) == 1', None, None, None], kwargs: {'limit': 10}, [requestId: 0cd28506-69ef-11ef-afc0-36592b0b7d05] (api_request.py:77)
[2024-09-03 12:21:31,106 - ERROR - fouram]: RPC error: [query], <MilvusException: (code=65535, message=fail to Query on QueryNode 6: worker(6) query failed: Operator::GetOutput failed for [Operator:FilterBits, plan node id: 0] :  => Reverse_Lookup should not be handled by inverted index at /go/src/github.com/milvus-io/milvus/internal/core/src/index/InvertedIndexTantivy.h:141

Expected Behavior

No response

Steps To Reproduce

concurrent test and calculation of RT and QPS

        :purpose:  `primary key: INT64`
            1. building `BITMAP` index on all supported 12 scalar fields, hybrid index on INT64 primary key field
            2. the other 22 scalar fields build `INVERTED`, `Trie`, `STL_SORT` indexes
            3. 4 fields of different vector types
            4. search for different expressions on BITMAP index fields

        :test steps:
            1. create collection with fields:
                'binary_vector': 128dim
                'float16_vector': 128dim
                'bfloat16_vector': 128dim
                'sparse_float_vector': sparse_range=[1, 100] <- the range of non-zero values of a sparse vector
                'id': primary key type is INT64

                all scalar fields: varchar max_length=10, array max_capacity=9
            2. build indexes:
                BIN_IVF_FLAT: 'binary_vector'
                IVF_SQ8: 'float16_vector'
                HNSW: 'bfloat16_vector'
                SPARSE_WAND: 'sparse_float_vector'

                default scalar index: 'id'
                BITMAP: '*_1' all supported field names
                INVERTED: 'array_float_1', 'array_double_1', 'float_2', 'double_2', 'bool_2', 'array_int8_2',
                          'array_int16_2', 'array_int32_2', 'array_int64_2', 'array_varchar_2', 'array_bool_2',
                          'array_float_2', 'array_double_2'
                Trie: 'varchar_2'
                STL_SORT: 'float_1', 'double_1', 'int8_2', 'int16_2', 'int32_2', 'int64_2'
            3. insert 5 million data
            4. flush collection
            5. build indexes again using the same params
            6. load collection
            7. concurrent request:
                - search
                - query
                - hybrid_search

Milvus Log

No response

Anything else?

client config:

{
     "dataset_params": {
          "metric_type": "JACCARD",
          "vector_field_name": "binary_vector",
          "dim": 128,
          "sparse_range": [
               1,
               100
          ],
          "max_length": 10,
          "scalars_index": {
               "id": {},
               "int8_1": {
                    "index_type": "BITMAP"
               },
               "int16_1": {
                    "index_type": "BITMAP"
               },
               "int32_1": {
                    "index_type": "BITMAP"
               },
               "int64_1": {
                    "index_type": "BITMAP"
               },
               "varchar_1": {
                    "index_type": "BITMAP"
               },
               "bool_1": {
                    "index_type": "BITMAP"
               },
               "array_int8_1": {
                    "index_type": "BITMAP"
               },
               "array_int16_1": {
                    "index_type": "BITMAP"
               },
               "array_int32_1": {
                    "index_type": "BITMAP"
               },
               "array_int64_1": {
                    "index_type": "BITMAP"
               },
               "array_varchar_1": {
                    "index_type": "BITMAP"
               },
               "array_bool_1": {
                    "index_type": "BITMAP"
               },
               "array_float_1": {
                    "index_type": "INVERTED"
               },
               "array_double_1": {
                    "index_type": "INVERTED"
               },
               "float_2": {
                    "index_type": "INVERTED"
               },
               "double_2": {
                    "index_type": "INVERTED"
               },
               "bool_2": {
                    "index_type": "INVERTED"
               },
               "array_int8_2": {
                    "index_type": "INVERTED"
               },
               "array_int16_2": {
                    "index_type": "INVERTED"
               },
               "array_int32_2": {
                    "index_type": "INVERTED"
               },
               "array_int64_2": {
                    "index_type": "INVERTED"
               },
               "array_varchar_2": {
                    "index_type": "INVERTED"
               },
               "array_bool_2": {
                    "index_type": "INVERTED"
               },
               "array_float_2": {
                    "index_type": "INVERTED"
               },
               "array_double_2": {
                    "index_type": "INVERTED"
               },
               "varchar_2": {
                    "index_type": "Trie"
               },
               "float_1": {
                    "index_type": "STL_SORT"
               },
               "double_1": {
                    "index_type": "STL_SORT"
               },
               "int8_2": {
                    "index_type": "STL_SORT"
               },
               "int16_2": {
                    "index_type": "STL_SORT"
               },
               "int32_2": {
                    "index_type": "STL_SORT"
               },
               "int64_2": {
                    "index_type": "STL_SORT"
               }
          },
          "vectors_index": {
               "float16_vector": {
                    "index_type": "IVF_SQ8",
                    "index_param": {
                         "nlist": 1024
                    },
                    "metric_type": "L2"
               },
               "bfloat16_vector": {
                    "index_type": "HNSW",
                    "index_param": {
                         "M": 8,
                         "efConstruction": 200
                    },
                    "metric_type": "L2"
               },
               "sparse_float_vector": {
                    "index_type": "SPARSE_WAND",
                    "index_param": {
                         "drop_ratio_build": 0.2
                    },
                    "metric_type": "IP"
               }
          },
          "scalars_params": {
               "array_int8_1": {
                    "params": {
                         "max_capacity": 9
                    }
               },
               "array_int16_1": {
                    "params": {
                         "max_capacity": 9
                    }
               },
               "array_int32_1": {
                    "params": {
                         "max_capacity": 9
                    }
               },
               "array_int64_1": {
                    "params": {
                         "max_capacity": 9
                    }
               },
               "array_double_1": {
                    "params": {
                         "max_capacity": 9
                    }
               },
               "array_float_1": {
                    "params": {
                         "max_capacity": 9
                    }
               },
               "array_varchar_1": {
                    "params": {
                         "max_capacity": 9
                    }
               },
               "array_bool_1": {
                    "params": {
                         "max_capacity": 9
                    }
               },
               "array_int8_2": {
                    "params": {
                         "max_capacity": 9
                    }
               },
               "array_int16_2": {
                    "params": {
                         "max_capacity": 9
                    }
               },
               "array_int32_2": {
                    "params": {
                         "max_capacity": 9
                    }
               },
               "array_int64_2": {
                    "params": {
                         "max_capacity": 9
                    }
               },
               "array_double_2": {
                    "params": {
                         "max_capacity": 9
                    }
               },
               "array_float_2": {
                    "params": {
                         "max_capacity": 9
                    }
               },
               "array_varchar_2": {
                    "params": {
                         "max_capacity": 9
                    }
               },
               "array_bool_2": {
                    "params": {
                         "max_capacity": 9
                    }
               }
          },
          "dataset_name": "local",
          "dataset_size": 5000000,
          "ni_per": 5000
     },
     "collection_params": {
          "other_fields": [
               "float16_vector",
               "bfloat16_vector",
               "sparse_float_vector",
               "int8_1",
               "int16_1",
               "int32_1",
               "int64_1",
               "double_1",
               "float_1",
               "varchar_1",
               "bool_1",
               "json_1",
               "array_int8_1",
               "array_int16_1",
               "array_int32_1",
               "array_int64_1",
               "array_double_1",
               "array_float_1",
               "array_varchar_1",
               "array_bool_1",
               "int8_2",
               "int16_2",
               "int32_2",
               "int64_2",
               "double_2",
               "float_2",
               "varchar_2",
               "bool_2",
               "json_2",
               "array_int8_2",
               "array_int16_2",
               "array_int32_2",
               "array_int64_2",
               "array_double_2",
               "array_float_2",
               "array_varchar_2",
               "array_bool_2"
          ],
          "shards_num": 2,
          "varchar_id": false
     },
     "resource_groups_params": {
          "reset": false
     },
     "database_user_params": {
          "reset_rbac": false,
          "reset_db": false
     },
     "index_params": {
          "index_type": "BIN_IVF_FLAT",
          "index_param": {
               "nlist": 2048
          }
     },
     "concurrent_params": {
          "concurrent_number": 1,
          "during_time": 1800,
          "interval": 20,
          "spawn_rate": null
     },
     "concurrent_tasks": [
          {
               "type": "search",
               "weight": 0,
               "params": {
                    "nq": 1000,
                    "top_k": 10,
                    "search_param": {
                         "nprobe": 16
                    },
                    "expr": "id >= 100",
                    "guarantee_timestamp": null,
                    "partition_names": null,
                    "output_fields": [
                         "*"
                    ],
                    "ignore_growing": false,
                    "group_by_field": null,
                    "timeout": null,
                    "random_data": true,
                    "check_task": "check_search_output",
                    "check_items": {
                         "output_fields": [
                              "float16_vector",
                              "bfloat16_vector",
                              "sparse_float_vector",
                              "int8_1",
                              "int16_1",
                              "int32_1",
                              "int64_1",
                              "double_1",
                              "float_1",
                              "varchar_1",
                              "bool_1",
                              "json_1",
                              "array_int8_1",
                              "array_int16_1",
                              "array_int32_1",
                              "array_int64_1",
                              "array_double_1",
                              "array_float_1",
                              "array_varchar_1",
                              "array_bool_1",
                              "int8_2",
                              "int16_2",
                              "int32_2",
                              "int64_2",
                              "double_2",
                              "float_2",
                              "varchar_2",
                              "bool_2",
                              "json_2",
                              "array_int8_2",
                              "array_int16_2",
                              "array_int32_2",
                              "array_int64_2",
                              "array_double_2",
                              "array_float_2",
                              "array_varchar_2",
                              "array_bool_2",
                              "id",
                              "binary_vector"
                         ],
                         "nq": 1000
                    }
               }
          },
          {
               "type": "query",
               "weight": 1,
               "params": {
                    "ids": null,
                    "expr": "(id % 10) == 1",
                    "output_fields": null,
                    "offset": null,
                    "limit": 10,
                    "ignore_growing": false,
                    "partition_names": null,
                    "timeout": null,
                    "consistency_level": null,
                    "random_data": false,
                    "random_count": 0,
                    "random_range": [
                         0,
                         1
                    ],
                    "field_name": "id",
                    "field_type": "int64",
                    "check_task": "check_response",
                    "check_items": null
               }
          },
          {
               "type": "hybrid_search",
               "weight": 0,
               "params": {
                    "nq": 10,
                    "top_k": 10,
                    "reqs": [
                         {
                              "search_param": {
                                   "nprobe": 128
                              },
                              "anns_field": "binary_vector",
                              "expr": "(int64_1 % 10) == 1",
                              "top_k": 100
                         },
                         {
                              "search_param": {
                                   "nprobe": 64
                              },
                              "anns_field": "float16_vector",
                              "expr": "ARRAY_LENGTH(array_int16_1) >= 5 && array_contains_any(array_bool_1, [true])",
                              "top_k": 10
                         },
                         {
                              "search_param": {
                                   "ef": 32
                              },
                              "anns_field": "bfloat16_vector",
                              "expr": "(int32_1 % 100) <= 50",
                              "top_k": 30
                         },
                         {
                              "search_param": {
                                   "drop_ratio_search": 0.1
                              },
                              "anns_field": "sparse_float_vector",
                              "expr": "(varchar_1 like \"1%\") && (bool_1 == True)"
                         }
                    ],
                    "rerank": {
                         "RRFRanker": []
                    },
                    "output_fields": null,
                    "ignore_growing": false,
                    "guarantee_timestamp": null,
                    "partition_names": null,
                    "timeout": null,
                    "random_data": true,
                    "check_task": "check_search_output",
                    "check_items": null
               }
          }
     ]
}
wangting0128 commented 2 months ago

verification passed

image: master-20240906-9871841f