search

eosphoros-ai / DB-GPT

AI Native Data App Development framework with AWEL(Agentic Workflow Expression Language) and Agents
http://docs.dbgpt.cn
MIT License
13.71k stars 1.84k forks source link

[BUG]: 数据库问答报错 Error:Index not found, please create an instance before querying #278

Closed UTimeStrange closed 11 months ago

UTimeStrange commented 1 year ago

bug如下

企业微信截图_16878357581834 
2023-06-27 11:18:02 | INFO | webserver | add_text. ip: . len: 7
2023-06-27 11:18:02 | INFO | webserver | User message send!c55f4de4-1495-11ee-b579-5254002e4088,SQL生成与诊断,,新增知识库对话,DB问答,customer_credit,
2023-06-27 11:18:02 | INFO | stdout | chat scene:**chat_with_db_qa**
2023-06-27 11:18:02,619 INFO sqlalchemy.engine.Engine use `customer_credit`
2023-06-27 11:18:02 | INFO | sqlalchemy.engine.Engine | use `customer_credit`
2023-06-27 11:18:02,619 INFO sqlalchemy.engine.Engine [cached since 2665s ago] {}
2023-06-27 11:18:02 | INFO | sqlalchemy.engine.Engine | [cached since 2665s ago] {}
2023-06-27 11:18:02,623 INFO sqlalchemy.engine.Engine SHOW FULL TABLES FROM `customer_credit`
2023-06-27 11:18:02 | INFO | sqlalchemy.engine.Engine | SHOW FULL TABLES FROM `customer_credit`
2023-06-27 11:18:02,623 INFO sqlalchemy.engine.Engine [raw sql] {}
2023-06-27 11:18:02 | INFO | sqlalchemy.engine.Engine | [raw sql] {}
2023-06-27 11:18:02,626 INFO sqlalchemy.engine.Engine SHOW CREATE TABLE `customer_credit`.`cas_err_log`
2023-06-27 11:18:02 | INFO | sqlalchemy.engine.Engine | SHOW CREATE TABLE `customer_credit`.`cas_err_log`
2023-06-27 11:18:02,626 INFO sqlalchemy.engine.Engine [raw sql] {}
2023-06-27 11:18:02 | INFO | sqlalchemy.engine.Engine | [raw sql] {}
2023-06-27 11:18:02,630 INFO sqlalchemy.engine.Engine SHOW CREATE TABLE `customer_credit`.`cc_test`
2023-06-27 11:18:02 | INFO | sqlalchemy.engine.Engine | SHOW CREATE TABLE `customer_credit`.`cc_test`
2023-06-27 11:18:02,630 INFO sqlalchemy.engine.Engine [raw sql] {}
2023-06-27 11:18:02 | INFO | sqlalchemy.engine.Engine | [raw sql] {}
2023-06-27 11:18:02,632 INFO sqlalchemy.engine.Engine SHOW CREATE TABLE `customer_credit`.`contractrisklabel_err_log`
2023-06-27 11:18:02 | INFO | sqlalchemy.engine.Engine | SHOW CREATE TABLE `customer_credit`.`contractrisklabel_err_log`
2023-06-27 11:18:02,633 INFO sqlalchemy.engine.Engine [raw sql] {}
2023-06-27 11:18:02 | INFO | sqlalchemy.engine.Engine | [raw sql] {}
2023-06-27 11:18:02,636 INFO sqlalchemy.engine.Engine SHOW CREATE TABLE `customer_credit`.`customer_credit_index_ams`
2023-06-27 11:18:02 | INFO | sqlalchemy.engine.Engine | SHOW CREATE TABLE `customer_credit`.`customer_credit_index_ams`
2023-06-27 11:18:02,636 INFO sqlalchemy.engine.Engine [raw sql] {}
2023-06-27 11:18:02 | INFO | sqlalchemy.engine.Engine | [raw sql] {}
2023-06-27 11:18:02,639 INFO sqlalchemy.engine.Engine SHOW CREATE TABLE `customer_credit`.`customer_credit_index_csig`
2023-06-27 11:18:02 | INFO | sqlalchemy.engine.Engine | SHOW CREATE TABLE `customer_credit`.`customer_credit_index_csig`
2023-06-27 11:18:02,640 INFO sqlalchemy.engine.Engine [raw sql] {}
2023-06-27 11:18:02 | INFO | sqlalchemy.engine.Engine | [raw sql] {}
2023-06-27 11:18:02,644 INFO sqlalchemy.engine.Engine SHOW CREATE TABLE `customer_credit`.`customer_credit_result_ams`
2023-06-27 11:18:02 | INFO | sqlalchemy.engine.Engine | SHOW CREATE TABLE `customer_credit`.`customer_credit_result_ams`
2023-06-27 11:18:02,644 INFO sqlalchemy.engine.Engine [raw sql] {}
2023-06-27 11:18:02 | INFO | sqlalchemy.engine.Engine | [raw sql] {}
2023-06-27 11:18:02,648 INFO sqlalchemy.engine.Engine SHOW CREATE TABLE `customer_credit`.`customer_credit_result_csig`
2023-06-27 11:18:02 | INFO | sqlalchemy.engine.Engine | SHOW CREATE TABLE `customer_credit`.`customer_credit_result_csig`
2023-06-27 11:18:02,648 INFO sqlalchemy.engine.Engine [raw sql] {}
2023-06-27 11:18:02 | INFO | sqlalchemy.engine.Engine | [raw sql] {}
2023-06-27 11:18:02,652 INFO sqlalchemy.engine.Engine SHOW CREATE TABLE `customer_credit`.`vendor_label`
2023-06-27 11:18:02 | INFO | sqlalchemy.engine.Engine | SHOW CREATE TABLE `customer_credit`.`vendor_label`
2023-06-27 11:18:02,652 INFO sqlalchemy.engine.Engine [raw sql] {}
2023-06-27 11:18:02 | INFO | sqlalchemy.engine.Engine | [raw sql] {}
2023-06-27 11:18:02 | INFO | webserver | stream out start!
2023-06-27 11:18:02 | INFO | sentence_transformers.SentenceTransformer | Load pretrained SentenceTransformer: /data/big_models/text2vec-large-chinese
2023-06-27 11:18:02 | WARNING | sentence_transformers.SentenceTransformer | No sentence-transformers model found with name /data/big_models/text2vec-large-chinese. Creating a new one with MEAN pooling.
2023-06-27 11:18:04 | INFO | sentence_transformers.SentenceTransformer | Use pytorch device: cuda
2023-06-27 11:18:04 | INFO | chromadb.telemetry.posthog | Anonymized telemetry enabled. See https://docs.trychroma.com/telemetry for more information.
2023-06-27 11:18:04 | INFO | chromadb | Running Chroma using direct local API.
2023-06-27 11:18:04 | WARNING | chromadb | Using embedded DuckDB with persistence: data will be stored in: /data/strangeli_code/DB-GPT/pilot/data/customer_credit_profile.vectordb
2023-06-27 11:18:04 | INFO | chromadb.db.duckdb | No existing DB found in /data/strangeli_code/DB-GPT/pilot/data/customer_credit_profile.vectordb, skipping load
2023-06-27 11:18:04 | INFO | chromadb.db.duckdb | No existing DB found in /data/strangeli_code/DB-GPT/pilot/data/customer_credit_profile.vectordb, skipping load
2023-06-27 11:18:04 | INFO | stdout |   ChromaStore similar search
2023-06-27 11:18:04 | INFO | LOGGER | ChromaStore similar search
Batches:   0%|                                                                                                                | 0/1 [00:00<?, ?it/s]
Batches: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 68.27it/s]
2023-06-27 11:18:04 | ERROR | stderr | 
2023-06-27 11:18:04 | INFO | stdout | Traceback (most recent call last):
2023-06-27 11:18:04 | INFO | stdout |   File "/data/strangeli_code/DB-GPT/pilot/server/webserver.py", line 324, in http_bot
2023-06-27 11:18:04 | INFO | stdout |     response = chat.stream_call()
2023-06-27 11:18:04 | INFO | stdout |   File "/data/strangeli_code/DB-GPT/pilot/scene/base_chat.py", line 138, in stream_call
2023-06-27 11:18:04 | INFO | stdout |     payload = self.__call_base()
2023-06-27 11:18:04 | INFO | stdout |   File "/data/strangeli_code/DB-GPT/pilot/scene/base_chat.py", line 106, in __call_base
2023-06-27 11:18:04 | INFO | stdout |     input_values = self.generate_input_values()
2023-06-27 11:18:04 | INFO | stdout |   File "/data/strangeli_code/DB-GPT/pilot/scene/chat_db/professional_qa/chat.py", line 55, in generate_input_values
2023-06-27 11:18:04 | INFO | stdout |     table_info = client.get_db_summary(
2023-06-27 11:18:04 | INFO | stdout |   File "/data/strangeli_code/DB-GPT/pilot/summary/db_summary_client.py", line 81, in get_db_summary
2023-06-27 11:18:04 | INFO | stdout |     table_docs = knowledge_embedding_client.similar_search(query, topk)
2023-06-27 11:18:04 | INFO | stdout |   File "/data/strangeli_code/DB-GPT/pilot/source_embedding/knowledge_embedding.py", line 80, in similar_search
2023-06-27 11:18:04 | INFO | stdout |     ans = vector_client.similar_search(text, topk)
2023-06-27 11:18:04 | INFO | stdout |   File "/data/strangeli_code/DB-GPT/pilot/vector_store/connector.py", line 24, in similar_search
2023-06-27 11:18:04 | INFO | stdout |     return self.client.similar_search(docs, topk)
2023-06-27 11:18:04 | INFO | stdout |   File "/data/strangeli_code/DB-GPT/pilot/vector_store/chroma_store.py", line 24, in similar_search
2023-06-27 11:18:04 | INFO | stdout |     return self.vector_store_client.similarity_search(text, topk)
2023-06-27 11:18:04 | INFO | stdout |   File "/data/conda_packages/anaconda3/envs/dbgpt/lib/python3.10/site-packages/langchain/vectorstores/chroma.py", line 138, in similarity_search
2023-06-27 11:18:04 | INFO | stdout |     docs_and_scores = self.similarity_search_with_score(query, k, filter=filter)
2023-06-27 11:18:04 | INFO | stdout |   File "/data/conda_packages/anaconda3/envs/dbgpt/lib/python3.10/site-packages/langchain/vectorstores/chroma.py", line 184, in similarity_search_with_score
2023-06-27 11:18:04 | INFO | stdout |     results = self._collection.query(
2023-06-27 11:18:04 | INFO | stdout |   File "/data/conda_packages/anaconda3/envs/dbgpt/lib/python3.10/site-packages/chromadb/api/models/Collection.py", line 219, in query
2023-06-27 11:18:04 | INFO | stdout |     return self._client._query(
2023-06-27 11:18:04 | INFO | stdout |   File "/data/conda_packages/anaconda3/envs/dbgpt/lib/python3.10/site-packages/chromadb/api/local.py", line 408, in _query
2023-06-27 11:18:04 | INFO | stdout |     uuids, distances = self._db.get_nearest_neighbors(
2023-06-27 11:18:04 | INFO | stdout |   File "/data/conda_packages/anaconda3/envs/dbgpt/lib/python3.10/site-packages/chromadb/db/clickhouse.py", line 583, in get_nearest_neighbors
2023-06-27 11:18:04 | INFO | stdout |     uuids, distances = index.get_nearest_neighbors(embeddings, n_results, ids)
2023-06-27 11:18:04 | INFO | stdout |   File "/data/conda_packages/anaconda3/envs/dbgpt/lib/python3.10/site-packages/chromadb/db/index/hnswlib.py", line 230, in get_nearest_neighbors
2023-06-27 11:18:04 | INFO | stdout |     raise NoIndexException(
2023-06-27 11:18:04 | INFO | stdout | chromadb.errors.NoIndexException: Index not found, please create an instance before querying
Aries-ckt commented 1 year ago

bug如下 企业微信截图_16878357581834

2023-06-27 11:18:02 | INFO | webserver | add_text. ip: . len: 7
2023-06-27 11:18:02 | INFO | webserver | User message send!c55f4de4-1495-11ee-b579-5254002e4088,SQL生成与诊断,,新增知识库对话,DB问答,customer_credit,
2023-06-27 11:18:02 | INFO | stdout | chat scene:**chat_with_db_qa**
2023-06-27 11:18:02,619 INFO sqlalchemy.engine.Engine use `customer_credit`
2023-06-27 11:18:02 | INFO | sqlalchemy.engine.Engine | use `customer_credit`
2023-06-27 11:18:02,619 INFO sqlalchemy.engine.Engine [cached since 2665s ago] {}
2023-06-27 11:18:02 | INFO | sqlalchemy.engine.Engine | [cached since 2665s ago] {}
2023-06-27 11:18:02,623 INFO sqlalchemy.engine.Engine SHOW FULL TABLES FROM `customer_credit`
2023-06-27 11:18:02 | INFO | sqlalchemy.engine.Engine | SHOW FULL TABLES FROM `customer_credit`
2023-06-27 11:18:02,623 INFO sqlalchemy.engine.Engine [raw sql] {}
2023-06-27 11:18:02 | INFO | sqlalchemy.engine.Engine | [raw sql] {}
2023-06-27 11:18:02,626 INFO sqlalchemy.engine.Engine SHOW CREATE TABLE `customer_credit`.`cas_err_log`
2023-06-27 11:18:02 | INFO | sqlalchemy.engine.Engine | SHOW CREATE TABLE `customer_credit`.`cas_err_log`
2023-06-27 11:18:02,626 INFO sqlalchemy.engine.Engine [raw sql] {}
2023-06-27 11:18:02 | INFO | sqlalchemy.engine.Engine | [raw sql] {}
2023-06-27 11:18:02,630 INFO sqlalchemy.engine.Engine SHOW CREATE TABLE `customer_credit`.`cc_test`
2023-06-27 11:18:02 | INFO | sqlalchemy.engine.Engine | SHOW CREATE TABLE `customer_credit`.`cc_test`
2023-06-27 11:18:02,630 INFO sqlalchemy.engine.Engine [raw sql] {}
2023-06-27 11:18:02 | INFO | sqlalchemy.engine.Engine | [raw sql] {}
2023-06-27 11:18:02,632 INFO sqlalchemy.engine.Engine SHOW CREATE TABLE `customer_credit`.`contractrisklabel_err_log`
2023-06-27 11:18:02 | INFO | sqlalchemy.engine.Engine | SHOW CREATE TABLE `customer_credit`.`contractrisklabel_err_log`
2023-06-27 11:18:02,633 INFO sqlalchemy.engine.Engine [raw sql] {}
2023-06-27 11:18:02 | INFO | sqlalchemy.engine.Engine | [raw sql] {}
2023-06-27 11:18:02,636 INFO sqlalchemy.engine.Engine SHOW CREATE TABLE `customer_credit`.`customer_credit_index_ams`
2023-06-27 11:18:02 | INFO | sqlalchemy.engine.Engine | SHOW CREATE TABLE `customer_credit`.`customer_credit_index_ams`
2023-06-27 11:18:02,636 INFO sqlalchemy.engine.Engine [raw sql] {}
2023-06-27 11:18:02 | INFO | sqlalchemy.engine.Engine | [raw sql] {}
2023-06-27 11:18:02,639 INFO sqlalchemy.engine.Engine SHOW CREATE TABLE `customer_credit`.`customer_credit_index_csig`
2023-06-27 11:18:02 | INFO | sqlalchemy.engine.Engine | SHOW CREATE TABLE `customer_credit`.`customer_credit_index_csig`
2023-06-27 11:18:02,640 INFO sqlalchemy.engine.Engine [raw sql] {}
2023-06-27 11:18:02 | INFO | sqlalchemy.engine.Engine | [raw sql] {}
2023-06-27 11:18:02,644 INFO sqlalchemy.engine.Engine SHOW CREATE TABLE `customer_credit`.`customer_credit_result_ams`
2023-06-27 11:18:02 | INFO | sqlalchemy.engine.Engine | SHOW CREATE TABLE `customer_credit`.`customer_credit_result_ams`
2023-06-27 11:18:02,644 INFO sqlalchemy.engine.Engine [raw sql] {}
2023-06-27 11:18:02 | INFO | sqlalchemy.engine.Engine | [raw sql] {}
2023-06-27 11:18:02,648 INFO sqlalchemy.engine.Engine SHOW CREATE TABLE `customer_credit`.`customer_credit_result_csig`
2023-06-27 11:18:02 | INFO | sqlalchemy.engine.Engine | SHOW CREATE TABLE `customer_credit`.`customer_credit_result_csig`
2023-06-27 11:18:02,648 INFO sqlalchemy.engine.Engine [raw sql] {}
2023-06-27 11:18:02 | INFO | sqlalchemy.engine.Engine | [raw sql] {}
2023-06-27 11:18:02,652 INFO sqlalchemy.engine.Engine SHOW CREATE TABLE `customer_credit`.`vendor_label`
2023-06-27 11:18:02 | INFO | sqlalchemy.engine.Engine | SHOW CREATE TABLE `customer_credit`.`vendor_label`
2023-06-27 11:18:02,652 INFO sqlalchemy.engine.Engine [raw sql] {}
2023-06-27 11:18:02 | INFO | sqlalchemy.engine.Engine | [raw sql] {}
2023-06-27 11:18:02 | INFO | webserver | stream out start!
2023-06-27 11:18:02 | INFO | sentence_transformers.SentenceTransformer | Load pretrained SentenceTransformer: /data/big_models/text2vec-large-chinese
2023-06-27 11:18:02 | WARNING | sentence_transformers.SentenceTransformer | No sentence-transformers model found with name /data/big_models/text2vec-large-chinese. Creating a new one with MEAN pooling.
2023-06-27 11:18:04 | INFO | sentence_transformers.SentenceTransformer | Use pytorch device: cuda
2023-06-27 11:18:04 | INFO | chromadb.telemetry.posthog | Anonymized telemetry enabled. See https://docs.trychroma.com/telemetry for more information.
2023-06-27 11:18:04 | INFO | chromadb | Running Chroma using direct local API.
2023-06-27 11:18:04 | WARNING | chromadb | Using embedded DuckDB with persistence: data will be stored in: /data/strangeli_code/DB-GPT/pilot/data/customer_credit_profile.vectordb
2023-06-27 11:18:04 | INFO | chromadb.db.duckdb | No existing DB found in /data/strangeli_code/DB-GPT/pilot/data/customer_credit_profile.vectordb, skipping load
2023-06-27 11:18:04 | INFO | chromadb.db.duckdb | No existing DB found in /data/strangeli_code/DB-GPT/pilot/data/customer_credit_profile.vectordb, skipping load
2023-06-27 11:18:04 | INFO | stdout |   ChromaStore similar search
2023-06-27 11:18:04 | INFO | LOGGER | ChromaStore similar search
Batches:   0%|                                                                                                                | 0/1 [00:00<?, ?it/s]
Batches: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 68.27it/s]
2023-06-27 11:18:04 | ERROR | stderr | 
2023-06-27 11:18:04 | INFO | stdout | Traceback (most recent call last):
2023-06-27 11:18:04 | INFO | stdout |   File "/data/strangeli_code/DB-GPT/pilot/server/webserver.py", line 324, in http_bot
2023-06-27 11:18:04 | INFO | stdout |     response = chat.stream_call()
2023-06-27 11:18:04 | INFO | stdout |   File "/data/strangeli_code/DB-GPT/pilot/scene/base_chat.py", line 138, in stream_call
2023-06-27 11:18:04 | INFO | stdout |     payload = self.__call_base()
2023-06-27 11:18:04 | INFO | stdout |   File "/data/strangeli_code/DB-GPT/pilot/scene/base_chat.py", line 106, in __call_base
2023-06-27 11:18:04 | INFO | stdout |     input_values = self.generate_input_values()
2023-06-27 11:18:04 | INFO | stdout |   File "/data/strangeli_code/DB-GPT/pilot/scene/chat_db/professional_qa/chat.py", line 55, in generate_input_values
2023-06-27 11:18:04 | INFO | stdout |     table_info = client.get_db_summary(
2023-06-27 11:18:04 | INFO | stdout |   File "/data/strangeli_code/DB-GPT/pilot/summary/db_summary_client.py", line 81, in get_db_summary
2023-06-27 11:18:04 | INFO | stdout |     table_docs = knowledge_embedding_client.similar_search(query, topk)
2023-06-27 11:18:04 | INFO | stdout |   File "/data/strangeli_code/DB-GPT/pilot/source_embedding/knowledge_embedding.py", line 80, in similar_search
2023-06-27 11:18:04 | INFO | stdout |     ans = vector_client.similar_search(text, topk)
2023-06-27 11:18:04 | INFO | stdout |   File "/data/strangeli_code/DB-GPT/pilot/vector_store/connector.py", line 24, in similar_search
2023-06-27 11:18:04 | INFO | stdout |     return self.client.similar_search(docs, topk)
2023-06-27 11:18:04 | INFO | stdout |   File "/data/strangeli_code/DB-GPT/pilot/vector_store/chroma_store.py", line 24, in similar_search
2023-06-27 11:18:04 | INFO | stdout |     return self.vector_store_client.similarity_search(text, topk)
2023-06-27 11:18:04 | INFO | stdout |   File "/data/conda_packages/anaconda3/envs/dbgpt/lib/python3.10/site-packages/langchain/vectorstores/chroma.py", line 138, in similarity_search
2023-06-27 11:18:04 | INFO | stdout |     docs_and_scores = self.similarity_search_with_score(query, k, filter=filter)
2023-06-27 11:18:04 | INFO | stdout |   File "/data/conda_packages/anaconda3/envs/dbgpt/lib/python3.10/site-packages/langchain/vectorstores/chroma.py", line 184, in similarity_search_with_score
2023-06-27 11:18:04 | INFO | stdout |     results = self._collection.query(
2023-06-27 11:18:04 | INFO | stdout |   File "/data/conda_packages/anaconda3/envs/dbgpt/lib/python3.10/site-packages/chromadb/api/models/Collection.py", line 219, in query
2023-06-27 11:18:04 | INFO | stdout |     return self._client._query(
2023-06-27 11:18:04 | INFO | stdout |   File "/data/conda_packages/anaconda3/envs/dbgpt/lib/python3.10/site-packages/chromadb/api/local.py", line 408, in _query
2023-06-27 11:18:04 | INFO | stdout |     uuids, distances = self._db.get_nearest_neighbors(
2023-06-27 11:18:04 | INFO | stdout |   File "/data/conda_packages/anaconda3/envs/dbgpt/lib/python3.10/site-packages/chromadb/db/clickhouse.py", line 583, in get_nearest_neighbors
2023-06-27 11:18:04 | INFO | stdout |     uuids, distances = index.get_nearest_neighbors(embeddings, n_results, ids)
2023-06-27 11:18:04 | INFO | stdout |   File "/data/conda_packages/anaconda3/envs/dbgpt/lib/python3.10/site-packages/chromadb/db/index/hnswlib.py", line 230, in get_nearest_neighbors
2023-06-27 11:18:04 | INFO | stdout |     raise NoIndexException(
2023-06-27 11:18:04 | INFO | stdout | chromadb.errors.NoIndexException: Index not found, please create an instance before querying

Do you see these logs "db summary embedding success" when you start the project, please check if {your_db}_profile.vectordb exists in pilot/data directory?

UTimeStrange commented 1 year ago

bug如下 企业微信截图_16878357581834

2023-06-27 11:18:02 | INFO | webserver | add_text. ip: . len: 7
2023-06-27 11:18:02 | INFO | webserver | User message send!c55f4de4-1495-11ee-b579-5254002e4088,SQL生成与诊断,,新增知识库对话,DB问答,customer_credit,
2023-06-27 11:18:02 | INFO | stdout | chat scene:**chat_with_db_qa**
2023-06-27 11:18:02,619 INFO sqlalchemy.engine.Engine use `customer_credit`
2023-06-27 11:18:02 | INFO | sqlalchemy.engine.Engine | use `customer_credit`
2023-06-27 11:18:02,619 INFO sqlalchemy.engine.Engine [cached since 2665s ago] {}
2023-06-27 11:18:02 | INFO | sqlalchemy.engine.Engine | [cached since 2665s ago] {}
2023-06-27 11:18:02,623 INFO sqlalchemy.engine.Engine SHOW FULL TABLES FROM `customer_credit`
2023-06-27 11:18:02 | INFO | sqlalchemy.engine.Engine | SHOW FULL TABLES FROM `customer_credit`
2023-06-27 11:18:02,623 INFO sqlalchemy.engine.Engine [raw sql] {}
2023-06-27 11:18:02 | INFO | sqlalchemy.engine.Engine | [raw sql] {}
2023-06-27 11:18:02,626 INFO sqlalchemy.engine.Engine SHOW CREATE TABLE `customer_credit`.`cas_err_log`
2023-06-27 11:18:02 | INFO | sqlalchemy.engine.Engine | SHOW CREATE TABLE `customer_credit`.`cas_err_log`
2023-06-27 11:18:02,626 INFO sqlalchemy.engine.Engine [raw sql] {}
2023-06-27 11:18:02 | INFO | sqlalchemy.engine.Engine | [raw sql] {}
2023-06-27 11:18:02,630 INFO sqlalchemy.engine.Engine SHOW CREATE TABLE `customer_credit`.`cc_test`
2023-06-27 11:18:02 | INFO | sqlalchemy.engine.Engine | SHOW CREATE TABLE `customer_credit`.`cc_test`
2023-06-27 11:18:02,630 INFO sqlalchemy.engine.Engine [raw sql] {}
2023-06-27 11:18:02 | INFO | sqlalchemy.engine.Engine | [raw sql] {}
2023-06-27 11:18:02,632 INFO sqlalchemy.engine.Engine SHOW CREATE TABLE `customer_credit`.`contractrisklabel_err_log`
2023-06-27 11:18:02 | INFO | sqlalchemy.engine.Engine | SHOW CREATE TABLE `customer_credit`.`contractrisklabel_err_log`
2023-06-27 11:18:02,633 INFO sqlalchemy.engine.Engine [raw sql] {}
2023-06-27 11:18:02 | INFO | sqlalchemy.engine.Engine | [raw sql] {}
2023-06-27 11:18:02,636 INFO sqlalchemy.engine.Engine SHOW CREATE TABLE `customer_credit`.`customer_credit_index_ams`
2023-06-27 11:18:02 | INFO | sqlalchemy.engine.Engine | SHOW CREATE TABLE `customer_credit`.`customer_credit_index_ams`
2023-06-27 11:18:02,636 INFO sqlalchemy.engine.Engine [raw sql] {}
2023-06-27 11:18:02 | INFO | sqlalchemy.engine.Engine | [raw sql] {}
2023-06-27 11:18:02,639 INFO sqlalchemy.engine.Engine SHOW CREATE TABLE `customer_credit`.`customer_credit_index_csig`
2023-06-27 11:18:02 | INFO | sqlalchemy.engine.Engine | SHOW CREATE TABLE `customer_credit`.`customer_credit_index_csig`
2023-06-27 11:18:02,640 INFO sqlalchemy.engine.Engine [raw sql] {}
2023-06-27 11:18:02 | INFO | sqlalchemy.engine.Engine | [raw sql] {}
2023-06-27 11:18:02,644 INFO sqlalchemy.engine.Engine SHOW CREATE TABLE `customer_credit`.`customer_credit_result_ams`
2023-06-27 11:18:02 | INFO | sqlalchemy.engine.Engine | SHOW CREATE TABLE `customer_credit`.`customer_credit_result_ams`
2023-06-27 11:18:02,644 INFO sqlalchemy.engine.Engine [raw sql] {}
2023-06-27 11:18:02 | INFO | sqlalchemy.engine.Engine | [raw sql] {}
2023-06-27 11:18:02,648 INFO sqlalchemy.engine.Engine SHOW CREATE TABLE `customer_credit`.`customer_credit_result_csig`
2023-06-27 11:18:02 | INFO | sqlalchemy.engine.Engine | SHOW CREATE TABLE `customer_credit`.`customer_credit_result_csig`
2023-06-27 11:18:02,648 INFO sqlalchemy.engine.Engine [raw sql] {}
2023-06-27 11:18:02 | INFO | sqlalchemy.engine.Engine | [raw sql] {}
2023-06-27 11:18:02,652 INFO sqlalchemy.engine.Engine SHOW CREATE TABLE `customer_credit`.`vendor_label`
2023-06-27 11:18:02 | INFO | sqlalchemy.engine.Engine | SHOW CREATE TABLE `customer_credit`.`vendor_label`
2023-06-27 11:18:02,652 INFO sqlalchemy.engine.Engine [raw sql] {}
2023-06-27 11:18:02 | INFO | sqlalchemy.engine.Engine | [raw sql] {}
2023-06-27 11:18:02 | INFO | webserver | stream out start!
2023-06-27 11:18:02 | INFO | sentence_transformers.SentenceTransformer | Load pretrained SentenceTransformer: /data/big_models/text2vec-large-chinese
2023-06-27 11:18:02 | WARNING | sentence_transformers.SentenceTransformer | No sentence-transformers model found with name /data/big_models/text2vec-large-chinese. Creating a new one with MEAN pooling.
2023-06-27 11:18:04 | INFO | sentence_transformers.SentenceTransformer | Use pytorch device: cuda
2023-06-27 11:18:04 | INFO | chromadb.telemetry.posthog | Anonymized telemetry enabled. See https://docs.trychroma.com/telemetry for more information.
2023-06-27 11:18:04 | INFO | chromadb | Running Chroma using direct local API.
2023-06-27 11:18:04 | WARNING | chromadb | Using embedded DuckDB with persistence: data will be stored in: /data/strangeli_code/DB-GPT/pilot/data/customer_credit_profile.vectordb
2023-06-27 11:18:04 | INFO | chromadb.db.duckdb | No existing DB found in /data/strangeli_code/DB-GPT/pilot/data/customer_credit_profile.vectordb, skipping load
2023-06-27 11:18:04 | INFO | chromadb.db.duckdb | No existing DB found in /data/strangeli_code/DB-GPT/pilot/data/customer_credit_profile.vectordb, skipping load
2023-06-27 11:18:04 | INFO | stdout |   ChromaStore similar search
2023-06-27 11:18:04 | INFO | LOGGER | ChromaStore similar search
Batches:   0%|                                                                                                                | 0/1 [00:00<?, ?it/s]
Batches: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 68.27it/s]
2023-06-27 11:18:04 | ERROR | stderr | 
2023-06-27 11:18:04 | INFO | stdout | Traceback (most recent call last):
2023-06-27 11:18:04 | INFO | stdout |   File "/data/strangeli_code/DB-GPT/pilot/server/webserver.py", line 324, in http_bot
2023-06-27 11:18:04 | INFO | stdout |     response = chat.stream_call()
2023-06-27 11:18:04 | INFO | stdout |   File "/data/strangeli_code/DB-GPT/pilot/scene/base_chat.py", line 138, in stream_call
2023-06-27 11:18:04 | INFO | stdout |     payload = self.__call_base()
2023-06-27 11:18:04 | INFO | stdout |   File "/data/strangeli_code/DB-GPT/pilot/scene/base_chat.py", line 106, in __call_base
2023-06-27 11:18:04 | INFO | stdout |     input_values = self.generate_input_values()
2023-06-27 11:18:04 | INFO | stdout |   File "/data/strangeli_code/DB-GPT/pilot/scene/chat_db/professional_qa/chat.py", line 55, in generate_input_values
2023-06-27 11:18:04 | INFO | stdout |     table_info = client.get_db_summary(
2023-06-27 11:18:04 | INFO | stdout |   File "/data/strangeli_code/DB-GPT/pilot/summary/db_summary_client.py", line 81, in get_db_summary
2023-06-27 11:18:04 | INFO | stdout |     table_docs = knowledge_embedding_client.similar_search(query, topk)
2023-06-27 11:18:04 | INFO | stdout |   File "/data/strangeli_code/DB-GPT/pilot/source_embedding/knowledge_embedding.py", line 80, in similar_search
2023-06-27 11:18:04 | INFO | stdout |     ans = vector_client.similar_search(text, topk)
2023-06-27 11:18:04 | INFO | stdout |   File "/data/strangeli_code/DB-GPT/pilot/vector_store/connector.py", line 24, in similar_search
2023-06-27 11:18:04 | INFO | stdout |     return self.client.similar_search(docs, topk)
2023-06-27 11:18:04 | INFO | stdout |   File "/data/strangeli_code/DB-GPT/pilot/vector_store/chroma_store.py", line 24, in similar_search
2023-06-27 11:18:04 | INFO | stdout |     return self.vector_store_client.similarity_search(text, topk)
2023-06-27 11:18:04 | INFO | stdout |   File "/data/conda_packages/anaconda3/envs/dbgpt/lib/python3.10/site-packages/langchain/vectorstores/chroma.py", line 138, in similarity_search
2023-06-27 11:18:04 | INFO | stdout |     docs_and_scores = self.similarity_search_with_score(query, k, filter=filter)
2023-06-27 11:18:04 | INFO | stdout |   File "/data/conda_packages/anaconda3/envs/dbgpt/lib/python3.10/site-packages/langchain/vectorstores/chroma.py", line 184, in similarity_search_with_score
2023-06-27 11:18:04 | INFO | stdout |     results = self._collection.query(
2023-06-27 11:18:04 | INFO | stdout |   File "/data/conda_packages/anaconda3/envs/dbgpt/lib/python3.10/site-packages/chromadb/api/models/Collection.py", line 219, in query
2023-06-27 11:18:04 | INFO | stdout |     return self._client._query(
2023-06-27 11:18:04 | INFO | stdout |   File "/data/conda_packages/anaconda3/envs/dbgpt/lib/python3.10/site-packages/chromadb/api/local.py", line 408, in _query
2023-06-27 11:18:04 | INFO | stdout |     uuids, distances = self._db.get_nearest_neighbors(
2023-06-27 11:18:04 | INFO | stdout |   File "/data/conda_packages/anaconda3/envs/dbgpt/lib/python3.10/site-packages/chromadb/db/clickhouse.py", line 583, in get_nearest_neighbors
2023-06-27 11:18:04 | INFO | stdout |     uuids, distances = index.get_nearest_neighbors(embeddings, n_results, ids)
2023-06-27 11:18:04 | INFO | stdout |   File "/data/conda_packages/anaconda3/envs/dbgpt/lib/python3.10/site-packages/chromadb/db/index/hnswlib.py", line 230, in get_nearest_neighbors
2023-06-27 11:18:04 | INFO | stdout |     raise NoIndexException(
2023-06-27 11:18:04 | INFO | stdout | chromadb.errors.NoIndexException: Index not found, please create an instance before querying

Do you see these logs "db summary embedding success" when you start the project, please check if {your_db}_profile.vectordb exists in pilot/data directory?

the folder {xxx}_profile.vectordb does exist,but there is nothing in it.

Aries-ckt commented 1 year ago

bug如下 企业微信截图_16878357581834

2023-06-27 11:18:02 | INFO | webserver | add_text. ip: . len: 7
2023-06-27 11:18:02 | INFO | webserver | User message send!c55f4de4-1495-11ee-b579-5254002e4088,SQL生成与诊断,,新增知识库对话,DB问答,customer_credit,
2023-06-27 11:18:02 | INFO | stdout | chat scene:**chat_with_db_qa**
2023-06-27 11:18:02,619 INFO sqlalchemy.engine.Engine use `customer_credit`
2023-06-27 11:18:02 | INFO | sqlalchemy.engine.Engine | use `customer_credit`
2023-06-27 11:18:02,619 INFO sqlalchemy.engine.Engine [cached since 2665s ago] {}
2023-06-27 11:18:02 | INFO | sqlalchemy.engine.Engine | [cached since 2665s ago] {}
2023-06-27 11:18:02,623 INFO sqlalchemy.engine.Engine SHOW FULL TABLES FROM `customer_credit`
2023-06-27 11:18:02 | INFO | sqlalchemy.engine.Engine | SHOW FULL TABLES FROM `customer_credit`
2023-06-27 11:18:02,623 INFO sqlalchemy.engine.Engine [raw sql] {}
2023-06-27 11:18:02 | INFO | sqlalchemy.engine.Engine | [raw sql] {}
2023-06-27 11:18:02,626 INFO sqlalchemy.engine.Engine SHOW CREATE TABLE `customer_credit`.`cas_err_log`
2023-06-27 11:18:02 | INFO | sqlalchemy.engine.Engine | SHOW CREATE TABLE `customer_credit`.`cas_err_log`
2023-06-27 11:18:02,626 INFO sqlalchemy.engine.Engine [raw sql] {}
2023-06-27 11:18:02 | INFO | sqlalchemy.engine.Engine | [raw sql] {}
2023-06-27 11:18:02,630 INFO sqlalchemy.engine.Engine SHOW CREATE TABLE `customer_credit`.`cc_test`
2023-06-27 11:18:02 | INFO | sqlalchemy.engine.Engine | SHOW CREATE TABLE `customer_credit`.`cc_test`
2023-06-27 11:18:02,630 INFO sqlalchemy.engine.Engine [raw sql] {}
2023-06-27 11:18:02 | INFO | sqlalchemy.engine.Engine | [raw sql] {}
2023-06-27 11:18:02,632 INFO sqlalchemy.engine.Engine SHOW CREATE TABLE `customer_credit`.`contractrisklabel_err_log`
2023-06-27 11:18:02 | INFO | sqlalchemy.engine.Engine | SHOW CREATE TABLE `customer_credit`.`contractrisklabel_err_log`
2023-06-27 11:18:02,633 INFO sqlalchemy.engine.Engine [raw sql] {}
2023-06-27 11:18:02 | INFO | sqlalchemy.engine.Engine | [raw sql] {}
2023-06-27 11:18:02,636 INFO sqlalchemy.engine.Engine SHOW CREATE TABLE `customer_credit`.`customer_credit_index_ams`
2023-06-27 11:18:02 | INFO | sqlalchemy.engine.Engine | SHOW CREATE TABLE `customer_credit`.`customer_credit_index_ams`
2023-06-27 11:18:02,636 INFO sqlalchemy.engine.Engine [raw sql] {}
2023-06-27 11:18:02 | INFO | sqlalchemy.engine.Engine | [raw sql] {}
2023-06-27 11:18:02,639 INFO sqlalchemy.engine.Engine SHOW CREATE TABLE `customer_credit`.`customer_credit_index_csig`
2023-06-27 11:18:02 | INFO | sqlalchemy.engine.Engine | SHOW CREATE TABLE `customer_credit`.`customer_credit_index_csig`
2023-06-27 11:18:02,640 INFO sqlalchemy.engine.Engine [raw sql] {}
2023-06-27 11:18:02 | INFO | sqlalchemy.engine.Engine | [raw sql] {}
2023-06-27 11:18:02,644 INFO sqlalchemy.engine.Engine SHOW CREATE TABLE `customer_credit`.`customer_credit_result_ams`
2023-06-27 11:18:02 | INFO | sqlalchemy.engine.Engine | SHOW CREATE TABLE `customer_credit`.`customer_credit_result_ams`
2023-06-27 11:18:02,644 INFO sqlalchemy.engine.Engine [raw sql] {}
2023-06-27 11:18:02 | INFO | sqlalchemy.engine.Engine | [raw sql] {}
2023-06-27 11:18:02,648 INFO sqlalchemy.engine.Engine SHOW CREATE TABLE `customer_credit`.`customer_credit_result_csig`
2023-06-27 11:18:02 | INFO | sqlalchemy.engine.Engine | SHOW CREATE TABLE `customer_credit`.`customer_credit_result_csig`
2023-06-27 11:18:02,648 INFO sqlalchemy.engine.Engine [raw sql] {}
2023-06-27 11:18:02 | INFO | sqlalchemy.engine.Engine | [raw sql] {}
2023-06-27 11:18:02,652 INFO sqlalchemy.engine.Engine SHOW CREATE TABLE `customer_credit`.`vendor_label`
2023-06-27 11:18:02 | INFO | sqlalchemy.engine.Engine | SHOW CREATE TABLE `customer_credit`.`vendor_label`
2023-06-27 11:18:02,652 INFO sqlalchemy.engine.Engine [raw sql] {}
2023-06-27 11:18:02 | INFO | sqlalchemy.engine.Engine | [raw sql] {}
2023-06-27 11:18:02 | INFO | webserver | stream out start!
2023-06-27 11:18:02 | INFO | sentence_transformers.SentenceTransformer | Load pretrained SentenceTransformer: /data/big_models/text2vec-large-chinese
2023-06-27 11:18:02 | WARNING | sentence_transformers.SentenceTransformer | No sentence-transformers model found with name /data/big_models/text2vec-large-chinese. Creating a new one with MEAN pooling.
2023-06-27 11:18:04 | INFO | sentence_transformers.SentenceTransformer | Use pytorch device: cuda
2023-06-27 11:18:04 | INFO | chromadb.telemetry.posthog | Anonymized telemetry enabled. See https://docs.trychroma.com/telemetry for more information.
2023-06-27 11:18:04 | INFO | chromadb | Running Chroma using direct local API.
2023-06-27 11:18:04 | WARNING | chromadb | Using embedded DuckDB with persistence: data will be stored in: /data/strangeli_code/DB-GPT/pilot/data/customer_credit_profile.vectordb
2023-06-27 11:18:04 | INFO | chromadb.db.duckdb | No existing DB found in /data/strangeli_code/DB-GPT/pilot/data/customer_credit_profile.vectordb, skipping load
2023-06-27 11:18:04 | INFO | chromadb.db.duckdb | No existing DB found in /data/strangeli_code/DB-GPT/pilot/data/customer_credit_profile.vectordb, skipping load
2023-06-27 11:18:04 | INFO | stdout |   ChromaStore similar search
2023-06-27 11:18:04 | INFO | LOGGER | ChromaStore similar search
Batches:   0%|                                                                                                                | 0/1 [00:00<?, ?it/s]
Batches: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 68.27it/s]
2023-06-27 11:18:04 | ERROR | stderr | 
2023-06-27 11:18:04 | INFO | stdout | Traceback (most recent call last):
2023-06-27 11:18:04 | INFO | stdout |   File "/data/strangeli_code/DB-GPT/pilot/server/webserver.py", line 324, in http_bot
2023-06-27 11:18:04 | INFO | stdout |     response = chat.stream_call()
2023-06-27 11:18:04 | INFO | stdout |   File "/data/strangeli_code/DB-GPT/pilot/scene/base_chat.py", line 138, in stream_call
2023-06-27 11:18:04 | INFO | stdout |     payload = self.__call_base()
2023-06-27 11:18:04 | INFO | stdout |   File "/data/strangeli_code/DB-GPT/pilot/scene/base_chat.py", line 106, in __call_base
2023-06-27 11:18:04 | INFO | stdout |     input_values = self.generate_input_values()
2023-06-27 11:18:04 | INFO | stdout |   File "/data/strangeli_code/DB-GPT/pilot/scene/chat_db/professional_qa/chat.py", line 55, in generate_input_values
2023-06-27 11:18:04 | INFO | stdout |     table_info = client.get_db_summary(
2023-06-27 11:18:04 | INFO | stdout |   File "/data/strangeli_code/DB-GPT/pilot/summary/db_summary_client.py", line 81, in get_db_summary
2023-06-27 11:18:04 | INFO | stdout |     table_docs = knowledge_embedding_client.similar_search(query, topk)
2023-06-27 11:18:04 | INFO | stdout |   File "/data/strangeli_code/DB-GPT/pilot/source_embedding/knowledge_embedding.py", line 80, in similar_search
2023-06-27 11:18:04 | INFO | stdout |     ans = vector_client.similar_search(text, topk)
2023-06-27 11:18:04 | INFO | stdout |   File "/data/strangeli_code/DB-GPT/pilot/vector_store/connector.py", line 24, in similar_search
2023-06-27 11:18:04 | INFO | stdout |     return self.client.similar_search(docs, topk)
2023-06-27 11:18:04 | INFO | stdout |   File "/data/strangeli_code/DB-GPT/pilot/vector_store/chroma_store.py", line 24, in similar_search
2023-06-27 11:18:04 | INFO | stdout |     return self.vector_store_client.similarity_search(text, topk)
2023-06-27 11:18:04 | INFO | stdout |   File "/data/conda_packages/anaconda3/envs/dbgpt/lib/python3.10/site-packages/langchain/vectorstores/chroma.py", line 138, in similarity_search
2023-06-27 11:18:04 | INFO | stdout |     docs_and_scores = self.similarity_search_with_score(query, k, filter=filter)
2023-06-27 11:18:04 | INFO | stdout |   File "/data/conda_packages/anaconda3/envs/dbgpt/lib/python3.10/site-packages/langchain/vectorstores/chroma.py", line 184, in similarity_search_with_score
2023-06-27 11:18:04 | INFO | stdout |     results = self._collection.query(
2023-06-27 11:18:04 | INFO | stdout |   File "/data/conda_packages/anaconda3/envs/dbgpt/lib/python3.10/site-packages/chromadb/api/models/Collection.py", line 219, in query
2023-06-27 11:18:04 | INFO | stdout |     return self._client._query(
2023-06-27 11:18:04 | INFO | stdout |   File "/data/conda_packages/anaconda3/envs/dbgpt/lib/python3.10/site-packages/chromadb/api/local.py", line 408, in _query
2023-06-27 11:18:04 | INFO | stdout |     uuids, distances = self._db.get_nearest_neighbors(
2023-06-27 11:18:04 | INFO | stdout |   File "/data/conda_packages/anaconda3/envs/dbgpt/lib/python3.10/site-packages/chromadb/db/clickhouse.py", line 583, in get_nearest_neighbors
2023-06-27 11:18:04 | INFO | stdout |     uuids, distances = index.get_nearest_neighbors(embeddings, n_results, ids)
2023-06-27 11:18:04 | INFO | stdout |   File "/data/conda_packages/anaconda3/envs/dbgpt/lib/python3.10/site-packages/chromadb/db/index/hnswlib.py", line 230, in get_nearest_neighbors
2023-06-27 11:18:04 | INFO | stdout |     raise NoIndexException(
2023-06-27 11:18:04 | INFO | stdout | chromadb.errors.NoIndexException: Index not found, please create an instance before querying

Do you see these logs "db summary embedding success" when you start the project, please check if {your_db}_profile.vectordb exists in pilot/data directory?

the folder {xxx}_profile.vectordb does exist,but there is nothing in it.

you can delete all dir in pilot/data, then reboot the webserver.py and try again.

finyone commented 1 year ago

the folder {xxx}_profile.vectordb does exist, and files do exist. but when query ,also give me “chromadb.errors.NoIndexException: Index not found, please create an instance before querying”

850tomato commented 1 year ago

同问

csunny commented 11 months ago

抱歉，早期版本更新比较快， 最新的版本解决了这些问题。 可以试试最新的v0.4.3版本