search

zilliztech / VectorDBBench

A Benchmark Tool for VectorDB
MIT License
558 stars 151 forks source link

Not able to connect to weavaiate on Kubernetes cluster from vectorDBBench #400

Open kilo1234 opened 2 weeks ago

kilo1234 commented 2 weeks ago

Hi, I am trying to run vectorDBBench on the weaviate Kubernetes cluster and not on the weaviate cloud. I am getting below error such that :

failed to run, reason=Weaviate did not start up in 5 seconds. Either the Weaviate URL http://10.103.208.135:80/v1 is wrong or Weaviate did not start up in the interval given in 'startup_period”

Can I know if the code in vectorDBBench is meant to connect only to weaviate cloud but not to local weaviate Kubernetes cluster?

xiaofan-luan commented 2 weeks ago

what is the error you it? I guess you probaly need a url issue or network issue or some authentication

xiaofan-luan commented 2 weeks ago

but we need errors

kilo1234 commented 2 weeks ago

Below is the error I get after running start init command in vectordbBench:

2024-11-05 12:51:19,771 | INFO: Starting run (task_runner.py:100) (1839131) 2024-11-05 12:51:27,986 | WARNING: pre run case error: Weaviate did not start up in 5 seconds. Either the Weaviate URL http://localhost:50051 is wrong or Weaviate did not start up in the interval given in 'startup_period'. (task_runner.py:96) (1839131) 2024-11-05 12:51:27,986 | WARNING: [1/1] case {'label': <CaseLabel.Performance: 2>, 'dataset': {'data': {'name': 'Cohere', 'size': 1000000, 'dim': 768, 'metric_type': <MetricType.COSINE: 'COSINE'>}}, 'db': 'WeaviateCloud'} failed to run, reason=Weaviate did not start up in 5 seconds. Either the Weaviate URL http://localhost:50051 is wrong or Weaviate did not start up in the interval given in 'startup_period'. (interface.py:184) (1839131) Traceback (most recent call last): File "/home/ubuntu/myvenv/lib/python3.11/site-packages/vectordb_bench/interface.py", line 165, in _async_task_v2 case_res.metrics = runner.run(drop_old) ^^^^^^^^^^^^^^^^^^^^ File "/home/ubuntu/myvenv/lib/python3.11/site-packages/vectordb_bench/backend/task_runner.py", line 102, in run self._pre_run(drop_old) File "/home/ubuntu/myvenv/lib/python3.11/site-packages/vectordb_bench/backend/task_runner.py", line 97, in _pre_run raise e from None File "/home/ubuntu/myvenv/lib/python3.11/site-packages/vectordb_bench/backend/task_runner.py", line 88, in _pre_run self.init_db(drop_old) File "/home/ubuntu/myvenv/lib/python3.11/site-packages/vectordb_bench/backend/task_runner.py", line 78, in init_db self.db = db_cls( ^^^^^^^ File "/home/ubuntu/myvenv/lib/python3.11/site-packages/vectordb_bench/backend/clients/weaviate_cloud/weaviate_cloud.py", line 36, in init client = Client(**db_config) ^^^^^^^^^^^^^^^^^^^ File "/home/ubuntu/myvenv/lib/python3.11/site-packages/weaviate/client.py", line 150, in init self._connection = Connection( ^^^^^^^^^^^ File "/home/ubuntu/myvenv/lib/python3.11/site-packages/weaviate/connect/connection.py", line 162, in init self.wait_for_weaviate(startup_period) File "/home/ubuntu/myvenv/lib/python3.11/site-packages/weaviate/connect/connection.py", line 642, in wait_for_weaviate raise WeaviateStartUpError( weaviate.exceptions.WeaviateStartUpError: Weaviate did not start up in 5 seconds. Either the Weaviate URL http://localhost:50051 is wrong or Weaviate did not start up in the interval given in 'startup_period'. 2024-11-05 12:51:27,988 | INFO |Task summary: run_id=0840e, task_label=2024110512 (models.py:355) 2024-11-05 12:51:27,988 | INFO |DB | db_label case label | load_dur qps latency(p99) recall max_load_count | label (models.py:355) 2024-11-05 12:51:27,988 | INFO |------------- | -------- ----------------- ---------- | ----------- ---------- --------------- ------------- -------------- | ----- (models.py:355) 2024-11-05 12:51:27,988 | INFO |WeaviateCloud | Performance768D1M 2024110512 | 0.0 0.0 0.0 0.0 0 | x (models.py:355) 2024-11-05 12:51:27,988 | WARNING: Replacing existing result with the same file_name: /home/ubuntu/myvenv/lib/python3.11/site-packages/vectordb_bench/results/WeaviateCloud/result_20241105_2024110512_weaviatecloud.json (models.py:209) (1839131) 2024-11-05 12:51:27,988 | INFO: write results to disk /home/ubuntu/myvenv/lib/python3.11/site-packages/vectordb_bench/results/WeaviateCloud/result_20241105_2024110512_weaviatecloud.json (models.py:213) (1839131) 2024-11-05 12:51:27,989 | INFO: Success to finish task: label=2024110512, run_id=0840eb025d2f4e45b8ca6309e953fe74 (interface.py:203) (1839131) 2024-11-05 12:59:11,790 | INFO: generated uuid for the tasks: 015a5f8e26fd48f2ae96029dcdad9836 (interface.py:66) (1829519) 2024-11-05 12:59:11,791 | INFO | DB | CaseType Dataset Filter | task_label (task_runner.py:338) 2024-11-05 12:59:11,791 | INFO | ----------- | ------------ -------------------- ------- | ------- (task_runner.py:338) 2024-11-05 12:59:11,791 | INFO | WeaviateCloud | Performance Cohere-MEDIUM-1M None | 2024110512 (task_runner.py:338) 2024-11-05 12:59:11,791 | INFO: task submitted: id=015a5f8e26fd48f2ae96029dcdad9836, 2024110512, case number: 1 (interface.py:231) (1829519) 2024-11-05 12:59:12.068 WARNING streamlit.runtime.scriptrunner_utils.script_run_context: Thread 'MainThread': missing ScriptRunContext! This warning can be ignored when running in bare mode.

alwayslove2013 commented 2 weeks ago

@kilo1234 VectorDBBench was initially focused on Weaviate Cloud, and it has not been tested for connectivity with a local Weaviate instance.

We warmly welcome contributions from the community to enhance it.

kilo1234 commented 1 week ago

Hi @@. @.> , My benchmark run stops abruptly in the middle while running on weaviate. I have raised the issue: https://github.com/zilliztech/VectorDBBench/issues/406. Please have a look.

Thanks, Pavan