task run failed when i excut the testing

[ZhangTomLiang]

error: code=0, message=metric type not found or not supported, supported: [L2 IP]

WARNING: pre run case error: <MilvusException: (code=0, message=metric type not found or not supported, supported: [L2 IP])> (task_runner.py:92) (546) 2024-06-19 07:49:05,797 | WARNING: [1/1] case {'label': <CaseLabel.Performance: 2>, 'dataset': {'data': {'name': 'Cohere', 'size': 1000000, 'dim': 768, 'metric_type': <MetricType.COSINE: 'COSINE'>}}, 'db': 'Milvus'} failed to run, reason=<MilvusException: (code=0, message=metric type not found or not supported, supported: [L2 IP])> (interface.py:187) (546) Traceback (most recent call last): File "/opt/code/vectordb_bench/interface.py", line 168, in _async_task_v2 case_res.metrics = runner.run(drop_old) ^^^^^^^^^^^^^^^^^^^^ File "/opt/code/vectordb_bench/backend/task_runner.py", line 96, in run self._pre_run(drop_old) File "/opt/code/vectordb_bench/backend/task_runner.py", line 93, in _pre_run raise e from None File "/opt/code/vectordb_bench/backend/task_runner.py", line 86, in _pre_run self.init_db(drop_old) File "/opt/code/vectordb_bench/backend/task_runner.py", line 77, in init_db self.db = db_cls( ^^^^^^^ File "/opt/code/vectordb_bench/backend/clients/milvus/milvus.py", line 64, in init col.create_index( File "/usr/local/lib/python3.11/site-packages/pymilvus/orm/collection.py", line 1371, in create_index return conn.create_index(self._name, field_name, index_params, timeout=timeout, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/pymilvus/decorators.py", line 147, in handler raise e from e File "/usr/local/lib/python3.11/site-packages/pymilvus/decorators.py", line 143, in handler return func(*args, *kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/pymilvus/decorators.py", line 182, in handler return func(self, args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/pymilvus/decorators.py", line 122, in handler raise e from e File "/usr/local/lib/python3.11/site-packages/pymilvus/decorators.py", line 87, in handler return func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/pymilvus/client/grpc_handler.py", line 987, in create_index check_status(status) File "/usr/local/lib/python3.11/site-packages/pymilvus/client/utils.py", line 63, in check_status raise MilvusException(status.code, status.reason, status.error_code)

step: git clone https://github.com/zilliztech/VectorDBBench.git cd VectorDBBench pip install -e ".[test]" init_bench

[XuanYang-cn]

@ZhangTomLiang The execption is rasied by Milvus. Seems like you're using an earlier verision of Milvus that doesn't support COSINE metric type.

Milvus supports COSINE metric type on 2.3.0 and later.

You could use an earlier version of vdbbench that would normolize cohere data and work with Milvus<2.3.0

[XuanYang-cn]

/assign @ZhangTomLiang

[ZhangTomLiang]

thank you XuanYang for the reply. but i tried the following commands, still have the same problem. Can we provide us the latest mirror image of milvus？it would be great.

commands: pip install vectordb-bench==0.0.2 init_bench

error: File "/usr/local/lib/python3.11/site-packages/pymilvus/client/utils.py", line 63, in check_status raise MilvusException(status.code, status.reason, status.error_code) pymilvus.exceptions.MilvusException: <MilvusException: (code=0, message=metric type not found or not supported, supported: [L2 IP])> 2024-06-20 03:43:18,205 | INFO |Task summary: run_id=783c0, task_label=2024062003 (models.py:289) 2024-06-20 03:43:18,205 | INFO |DB | db_label case label | load_dur qps latency(p99) recall max_load_count | label (models.py:289) 2024-06-20 03:43:18,206 | INFO |------ | -------- -------------------- ---------- | ----------- ---------- --------------- ------------- -------------- | ----- (models.py:289) 2024-06-20 03:43:18,206 | INFO |Milvus | Performance1536D500K 2024062003 | 0.0 0.0 0.0 0.0 0 | x (models.py:289) 2024-06-20 03:43:18,206 | WARNING: Replacing existing result with the same file_name: /opt/code/vectordb_bench/results/Milvus/result_20240620_2024062003_milvus.json (models.py:143) (1910) 2024-06-20 03:43:18,206 | INFO: write results to disk /opt/code/vectordb_bench/results/Milvus/result_20240620_2024062003_milvus.json (models.py:147) (1910) 2024-06-20 03:43:18,207 | INFO: Succes to finish task: label=2024062003, run_id=783c0641179f4e94a5f22671241b7294 (interface.py:207) (1910)

[XuanYang-cn]

@ZhangTomLiang Please use vectordb-bench==0.0.1 for Milvus that doesn't support COSINE.

Or you could also using the latest Milvus 2.4.5 that supports the COSINE metrics.