Hello. I was trying to run a test run for Milvus Standalone with the Search Performance Test (100M Dataset, 768 Dim). However, I got the following output with an error of specified key does not exist:
2024-01-29 19:15:09,634 | INFO |Task summary: run_id=92ba5, task_label=2024012919_dockertes (models.py:285)
2024-01-29 19:15:09,634 | INFO |DB | db_label case label | load_dur qps latency(p99) recall max_load_count | label (models.py:285)
2024-01-29 19:15:09,634 | INFO |------ | -------- ----------------- -------------------- | ----------- ------------ --------------- ------------- -------------- | ----- (models.py:285)
2024-01-29 19:15:09,634 | INFO |Milvus | Performance768D1M 2024012919_dockertes | 562.5584 2261.2764 0.0044 0.9821 0 | :) (models.py:285)
2024-01-29 19:15:09,634 | INFO: write results to disk /home/labuser/.local/lib/python3.11/site-packages/vectordb_bench/results/Milvus/result_20240129_2024012919_dockertes_milvus.json (models.py:143) (3442542)
2024-01-29 19:15:09,635 | INFO: Succes to finish task: label=2024012919_dockertes, run_id=92ba58d5fda64c8f88100a54d0f18c0a (interface.py:207) (3442542)
2024-01-29 19:19:01,624 | INFO: generated uuid for the tasks: 81e208a5a31844e8be6f45b9ff165e66 (interface.py:69) (3442368)
2024-01-29 19:19:01,624 | INFO | DB | CaseType Dataset Filter | task_label (task_runner.py:288)
2024-01-29 19:19:01,624 | INFO | ----------- | ------------ -------------------- ------- | ------- (task_runner.py:288)
2024-01-29 19:19:01,624 | INFO | Milvus | Performance LAION-LARGE-100M None | 2024012919_100Mdockertest (task_runner.py:288)
2024-01-29 19:19:01,624 | INFO: task submitted: id=81e208a5a31844e8be6f45b9ff165e66, 2024012919_100Mdockertest, case number: 1 (interface.py:235) (3442368)
2024-01-29 19:19:02,223 | INFO: [1/1] start case: {'label': <CaseLabel.Performance: 2>, 'dataset': {'data': {'name': 'LAION', 'size': 100000000, 'dim': 768, 'metric_type': <MetricType.L2: 'L2'>}}, 'db': 'Milvus'}, drop_old=True (interface.py:167) (3463933)
2024-01-29 19:19:02,490 | INFO: Milvus client drop_old collection: VectorDBBenchCollection (milvus.py:45) (3463933)
2024-01-29 19:19:02,494 | INFO: Milvus create collection: VectorDBBenchCollection (milvus.py:55) (3463933)
2024-01-29 19:19:03,162 | INFO: local dataset root path not exist, creating it: /mnt/milvus_data/laion/laion_large_100m (data_source.py:126) (3463933)
2024-01-29 19:19:03,162 | INFO: Start to downloading files, total count: 104 (data_source.py:142) (3463933)
2%|███▊ | 2/104 [00:00<00:46, 2.19it/s]
2024-01-29 19:19:04,074 | WARNING: pre run case error: The specified key does not exist. (task_runner.py:92) (3463933)
2024-01-29 19:19:04,074 | WARNING: [1/1] case {'label': <CaseLabel.Performance: 2>, 'dataset': {'data': {'name': 'LAION', 'size': 100000000, 'dim': 768, 'metric_type': <MetricType.L2: 'L2'>}}, 'db': 'Milvus'} failed to run, reason=The specified key does not exist. (interface.py:187) (3463933)
Traceback (most recent call last):
File "/home/labuser/.local/lib/python3.11/site-packages/vectordb_bench/interface.py", line 168, in _async_task_v2
case_res.metrics = runner.run(drop_old)
^^^^^^^^^^^^^^^^^^^^
File "/home/labuser/.local/lib/python3.11/site-packages/vectordb_bench/backend/task_runner.py", line 96, in run
self._pre_run(drop_old)
File "/home/labuser/.local/lib/python3.11/site-packages/vectordb_bench/backend/task_runner.py", line 93, in _pre_run
raise e from None
File "/home/labuser/.local/lib/python3.11/site-packages/vectordb_bench/backend/task_runner.py", line 87, in _pre_run
self.ca.dataset.prepare(self.dataset_source)
File "/home/labuser/.local/lib/python3.11/site-packages/vectordb_bench/backend/dataset.py", line 202, in prepare
source.reader().read(
File "/home/labuser/.local/lib/python3.11/site-packages/vectordb_bench/backend/data_source.py", line 145, in read
self.fs.download(s3_file, local_ds_root.as_posix())
File "/home/labuser/.local/lib/python3.11/site-packages/fsspec/spec.py", line 1534, in download
return self.get(rpath, lpath, recursive=recursive, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/labuser/.local/lib/python3.11/site-packages/fsspec/asyn.py", line 118, in wrapper
return sync(self.loop, func, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/labuser/.local/lib/python3.11/site-packages/fsspec/asyn.py", line 103, in sync
raise return_result
File "/home/labuser/.local/lib/python3.11/site-packages/fsspec/asyn.py", line 56, in _runner
result[0] = await coro
^^^^^^^^^^
File "/home/labuser/.local/lib/python3.11/site-packages/fsspec/asyn.py", line 650, in _get
return await _run_coros_in_chunks(
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/labuser/.local/lib/python3.11/site-packages/fsspec/asyn.py", line 254, in _run_coros_in_chunks
await asyncio.gather(*chunk, return_exceptions=return_exceptions),
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/asyncio/tasks.py", line 452, in wait_for
return await fut
^^^^^^^^^
File "/home/labuser/.local/lib/python3.11/site-packages/s3fs/core.py", line 1224, in _get_file
body, content_length = await _open_file(range=0)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/labuser/.local/lib/python3.11/site-packages/s3fs/core.py", line 1215, in _open_file
resp = await self._call_s3(
^^^^^^^^^^^^^^^^^^^^
File "/home/labuser/.local/lib/python3.11/site-packages/s3fs/core.py", line 348, in _call_s3
return await _error_wrapper(
^^^^^^^^^^^^^^^^^^^^^
File "/home/labuser/.local/lib/python3.11/site-packages/s3fs/core.py", line 140, in _error_wrapper
raise err
FileNotFoundError: The specified key does not exist.
Currently starting up the Milvus Standalone through Docker Compose with the following docker-compose.yml:
Hello. I was trying to run a test run for Milvus Standalone with the Search Performance Test (100M Dataset, 768 Dim). However, I got the following output with an error of specified key does not exist:
Currently starting up the Milvus Standalone through Docker Compose with the following docker-compose.yml: