[Bug]: [benchmark][standalone]After Milvus load data, search reports an error "collection not loaded into memory when search"

[jingkl]

Is there an existing issue for this?

• [X] I have searched the existing issues

Environment

- Milvus version:master-20220824-90de312
- Deployment mode(standalone or cluster):standalone
- SDK version(e.g. pymilvus v2.0.0rc2):2.2.0.dev19
- OS(Ubuntu or CentOS): 
- CPU/Memory: 
- GPU: 
- Others:

Current Behavior

server-instance fouram-cron-1661356800-6 server-configmap server-single-16c64m client-configmap client-search-sift50m-ivf-flat-2048

fouram-cron-1661356800-6-etcd-0                                  1/1     Running     0             2m52s   10.104.4.9     4am-node11   <none>           <none>
fouram-cron-1661356800-6-milvus-standalone-7ccb945f66-8l2mb      1/1     Running     0             2m52s   10.104.4.6     4am-node11   <none>           <none>
fouram-cron-1661356800-6-minio-69c95c578b-zzcqm                  1/1     Running     0             2m51s   10.104.6.222   4am-node13   <none>           <none>

[2022-08-24 16:46:59,238] [    INFO] - 50000000 (milvus_benchmark.runners.search:292)
[2022-08-24 16:46:59,238] [    INFO] - Start load collection (milvus_benchmark.runners.search:293)
[2022-08-24 17:07:01,007] [   DEBUG] - Milvus load_collection run in 1201.7693s (milvus_benchmark.client:56)
[2022-08-24 17:07:01,008] [   DEBUG] - {'load_time': 1201.7697} (milvus_benchmark.runners.search:296)
[2022-08-24 17:07:01,008] [    INFO] - Start run case (milvus_benchmark.main:88)
[2022-08-24 17:07:01,008] [   DEBUG] - Start run query, run 1 of 30 (milvus_benchmark.runners.search:306)
[2022-08-24 17:07:01,008] [    INFO] - {'nq': 1, 'topk': 1, 'search_param': {'nprobe': 8}, 'filter': [], 'guarantee_timestamp': None} (milvus_benchmark.runners.search:307)
[2022-08-24 17:07:01,013] [   ERROR] - RPC error: [search], <MilvusException: (code=1, message=collection:sift_50m_128_l2 or partition:[] not loaded into memory when search)>, <Time:{'RPC start': '2022-08-24 17:07:01.008425', 'RPC error': '2022-08-24 17:07:01.013487'}> (pymilvus.decorators:95)
[2022-08-24 17:07:01,015] [   ERROR] - Traceback (most recent call last):
  File "main.py", line 95, in run_suite
    result = runner.run_case(case_metric, **case)
  File "/src/milvus_benchmark/runners/search.py", line 309, in run_case
    _query_res = self.milvus.query(case_param["vector_query"], filter_query=case_param["filter_query"],
  File "/src/milvus_benchmark/client.py", line 52, in wrapper
    result = func(*args, **kwargs)
  File "/src/milvus_benchmark/client.py", line 338, in query
    result = self._milvus.search(tmp_collection_name, **params)
  File "/usr/local/lib/python3.8/dist-packages/pymilvus/client/stub.py", line 844, in search
    return handler.search(collection_name, data, anns_field, param, limit, expression,
  File "/usr/local/lib/python3.8/dist-packages/pymilvus/decorators.py", line 96, in handler
    raise e
  File "/usr/local/lib/python3.8/dist-packages/pymilvus/decorators.py", line 92, in handler
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/pymilvus/decorators.py", line 74, in handler
    raise e
  File "/usr/local/lib/python3.8/dist-packages/pymilvus/decorators.py", line 48, in handler
    return func(self, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/pymilvus/client/grpc_handler.py", line 452, in search
    return self._execute_search_requests(requests, timeout, **_kwargs)
  File "/usr/local/lib/python3.8/dist-packages/pymilvus/client/grpc_handler.py", line 416, in _execute_search_requests
    raise pre_err
  File "/usr/local/lib/python3.8/dist-packages/pymilvus/client/grpc_handler.py", line 407, in _execute_search_requests
    raise MilvusException(response.status.error_code, response.status.reason)
pymilvus.exceptions.MilvusException: <MilvusException: (code=1, message=collection:sift_50m_128_l2 or partition:[] not loaded into memory when search)>
 (milvus_benchmark.main:98)

Expected Behavior

No response

Steps To Reproduce

1. create a collection
2.build index and insert data
3.build index again,load
4.search -> raise errors

Milvus Log

No response

Anything else?

client-search-sift50m-ivf-flat-2048

  config.yaml: 
    insert_search_performance:
      collections:
        -
          collection_name: sift_50m_128_l2
          ni_per: 50000
          build_index: true
          index_type: ivf_flat 
          index_param:
            nlist: 2048
          run_count: 30
          top_ks: [1, 10, 100, 1000]
          nqs: [1, 10, 100, 500, 1000, 1200]
          search_params:
            -
              nprobe: 8
            -
              nprobe: 32

[yanliang567]

/assign @xige-16 /unassign

[xige-16]

may be has been fixed by after refacting of querycoord, please verify. @jingkl /assign @jingkl

[jingkl]

This issue is the same as issue #18508, close it first and continue to focus on it in #18508