langgenius / dify

Dify is an open-source LLM app development platform. Dify's intuitive interface combines AI workflow, RAG pipeline, agent capabilities, model management, observability features and more, letting you quickly go from prototype to production.
https://dify.ai
Other
51.49k stars 7.44k forks source link

"Errors with 'mixed retrieval' Mode: Inconsistent 'vector' Errors" #7031

Closed mengdahuang closed 3 months ago

mengdahuang commented 3 months ago

Self Checks

Dify version

0.6.16

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

Dear Dify Support Team,

After I upgraded to the latest version and changed the knowledge base from 'vector retrieval' to 'mixed retrieval', some questions returned errors, with the web interface indicating 'vector'. iShot_2024-08-07_09 17 59 iShot_2024-08-07_09 23 42 iShot_2024-08-07_09 17 42 However, when I switched it back to 'vector retrieval', everything returned to normal. iShot_2024-08-07_09 16 59

PS:Not all questions return the 'vector' error when using 'mixed retrieval'; some work normally. Please see the detailed logs below.

error log

api-1         | 2024-08-07 01:01:10,083.083 ERROR [Dummy-533] [hit_testing.py:82] - Hit testing failed.
api-1         | Traceback (most recent call last):
api-1         |   File "/app/api/controllers/console/datasets/hit_testing.py", line 56, in post
api-1         |     response = HitTestingService.retrieve(
api-1         |   File "/app/api/services/hit_testing_service.py", line 41, in retrieve
api-1         |     all_documents = RetrievalService.retrieve(retrival_method=retrieval_model.get('search_method', 'semantic_search'),
api-1         |   File "/app/api/core/rag/datasource/retrieval_service.py", line 99, in retrieve
nginx-1       | 172.20.0.1 - - [07/Aug/2024:01:01:10 +0000] "POST /console/api/datasets/d0bde6c1-b41d-45c9-a16f-0bf632c6c70d/hit-testing HTTP/1.1" 500 72 "https://dify.demo.com/datasets/d0bde6c1-b41d-45c9-a16f-0bf632c6c70d/hitTesting" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0.0.0 Safari/537.36 Edg/127.0.0.0" "45.91.81.43"
api-1         |     all_documents = data_post_processor.invoke(
api-1         |   File "/app/api/core/rag/data_post_processor/data_post_processor.py", line 27, in invoke
api-1         |     documents = self.rerank_runner.run(query, documents, score_threshold, top_n, user)
api-1         |   File "/app/api/core/rag/rerank/weight_rerank.py", line 47, in run
api-1         |     query_vector_scores = self._calculate_cosine(self.tenant_id, query, documents, self.weights.vector_setting)
api-1         |   File "/app/api/core/rag/rerank/weight_rerank.py", line 162, in _calculate_cosine
api-1         |     content_vector = document.metadata['vector']
api-1         | KeyError: 'vector'
api-1         | 2024-08-07 01:01:10,084.084 ERROR [Dummy-533] [app.py:838] - Exception on /console/api/datasets/d0bde6c1-b41d-45c9-a16f-0bf632c6c70d/hit-testing [POST]
api-1         | Traceback (most recent call last):
api-1         |   File "/app/api/controllers/console/datasets/hit_testing.py", line 56, in post
api-1         |     response = HitTestingService.retrieve(
api-1         |   File "/app/api/services/hit_testing_service.py", line 41, in retrieve
api-1         |     all_documents = RetrievalService.retrieve(retrival_method=retrieval_model.get('search_method', 'semantic_search'),
api-1         |   File "/app/api/core/rag/datasource/retrieval_service.py", line 99, in retrieve
api-1         |     all_documents = data_post_processor.invoke(
api-1         |   File "/app/api/core/rag/data_post_processor/data_post_processor.py", line 27, in invoke
api-1         |     documents = self.rerank_runner.run(query, documents, score_threshold, top_n, user)
api-1         |   File "/app/api/core/rag/rerank/weight_rerank.py", line 47, in run
api-1         |     query_vector_scores = self._calculate_cosine(self.tenant_id, query, documents, self.weights.vector_setting)
api-1         |   File "/app/api/core/rag/rerank/weight_rerank.py", line 162, in _calculate_cosine
api-1         |     content_vector = document.metadata['vector']
api-1         | KeyError: 'vector'
api-1         |
api-1         | During handling of the above exception, another exception occurred:
api-1         |
api-1         | Traceback (most recent call last):
api-1         |   File "/app/api/.venv/lib/python3.10/site-packages/flask/app.py", line 880, in full_dispatch_request
api-1         |     rv = self.dispatch_request()
api-1         |   File "/app/api/.venv/lib/python3.10/site-packages/flask/app.py", line 865, in dispatch_request
api-1         |     return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)  # type: ignore[no-any-return]
api-1         |   File "/app/api/.venv/lib/python3.10/site-packages/flask_restful/__init__.py", line 489, in wrapper
api-1         |     resp = resource(*args, **kwargs)
api-1         |   File "/app/api/.venv/lib/python3.10/site-packages/flask/views.py", line 110, in view
api-1         |     return current_app.ensure_sync(self.dispatch_request)(**kwargs)  # type: ignore[no-any-return]
api-1         |   File "/app/api/.venv/lib/python3.10/site-packages/flask_restful/__init__.py", line 604, in dispatch_request
api-1         |     resp = meth(*args, **kwargs)
api-1         |   File "/app/api/controllers/console/setup.py", line 75, in decorated
api-1         |     return view(*args, **kwargs)
api-1         |   File "/app/api/libs/login.py", line 91, in decorated_view
api-1         |     return current_app.ensure_sync(func)(*args, **kwargs)
api-1         |   File "/app/api/controllers/console/wraps.py", line 22, in decorated
api-1         |     return view(*args, **kwargs)
api-1         |   File "/app/api/controllers/console/datasets/hit_testing.py", line 83, in post
api-1         |     raise InternalServerError(str(e))
api-1         | werkzeug.exceptions.InternalServerError: 500 Internal Server Error: 'vector'
nginx-1       | 172.20.0.1 - - [07/Aug/2024:01:03:18 +0000] "GET /console/api/workspaces/current/default-model?model_type=rerank HTTP/1.1" 200 606 "https://dify.demo.com/datasets/d0bde6c1-b41d-45c9-a16f-0bf632c6c70d/hitTesting" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0.0.0 Safari/537.36 Edg/127.0.0.0" "45.91.81.43"
nginx-1       | 172.20.0.1 - - [07/Aug/2024:01:03:18 +0000] "GET /console/api/workspaces/current/models/model-types/rerank HTTP/1.1" 200 1744 "https://dify.demo.com/datasets/d0bde6c1-b41d-45c9-a16f-0bf632c6c70d/hitTesting" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0.0.0 Safari/537.36 Edg/127.0.0.0" "45.91.81.43"
db-1          | 2024-08-07 09:03:25.085 CST [28] LOG:  checkpoint starting: time
db-1          | 2024-08-07 09:03:25.662 CST [28] LOG:  checkpoint complete: wrote 6 buffers (0.0%); 0 WAL file(s) added, 0 removed, 0 recycled; write=0.515 s, sync=0.017 s, total=0.577 s; sync files=5, longest=0.011 s, average=0.004 s; distance=12 kB, estimate=444 kB
api-1         | 2024-08-07 01:05:15,537.537 INFO [Thread-535 (embedding_search)] [_client.py:1026] - HTTP Request: POST https://az-eastus.openai.azure.com/openai/deployments/text-embedding-3-large/embeddings?api-version=2024-02-15-preview "HTTP/1.1 200 OK"
nginx-1       | 172.20.0.1 - - [07/Aug/2024:01:05:15 +0000] "POST /console/api/datasets/d0bde6c1-b41d-45c9-a16f-0bf632c6c70d/hit-testing HTTP/1.1" 200 150 "https://dify.demo.com/datasets/d0bde6c1-b41d-45c9-a16f-0bf632c6c70d/hitTesting" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0.0.0 Safari/537.36 Edg/127.0.0.0" "45.91.81.43"
nginx-1       | 172.20.0.1 - - [07/Aug/2024:01:05:15 +0000] "GET /console/api/datasets/d0bde6c1-b41d-45c9-a16f-0bf632c6c70d/queries?limit=10&page=1 HTTP/1.1" 200 2843 "https://dify.demo.com/datasets/d0bde6c1-b41d-45c9-a16f-0bf632c6c70d/hitTesting" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0.0.0 Safari/537.36 Edg/127.0.0.0" "45.91.81.43"
nginx-1       | 172.20.0.1 - - [07/Aug/2024:01:05:18 +0000] "GET /console/api/workspaces/current/default-model?model_type=rerank HTTP/1.1" 200 606 "https://dify.demo.com/datasets/d0bde6c1-b41d-45c9-a16f-0bf632c6c70d/hitTesting" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0.0.0 Safari/537.36 Edg/127.0.0.0" "45.91.81.43"
nginx-1       | 172.20.0.1 - - [07/Aug/2024:01:05:18 +0000] "GET /console/api/workspaces/current/models/model-types/rerank HTTP/1.1" 200 1744 "https://dify.demo.com/datasets/d0bde6c1-b41d-45c9-a16f-0bf632c6c70d/hitTesting" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0.0.0 Safari/537.36 Edg/127.0.0.0" "45.91.81.43"
nginx-1       | 172.20.0.1 - - [07/Aug/2024:01:05:22 +0000] "POST /console/api/datasets/d0bde6c1-b41d-45c9-a16f-0bf632c6c70d/hit-testing HTTP/1.1" 200 7527 "https://dify.demo.com/datasets/d0bde6c1-b41d-45c9-a16f-0bf632c6c70d/hitTesting" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0.0.0 Safari/537.36 Edg/127.0.0.0" "45.91.81.43"
nginx-1       | 172.20.0.1 - - [07/Aug/2024:01:05:22 +0000] "GET /console/api/datasets/d0bde6c1-b41d-45c9-a16f-0bf632c6c70d/queries?limit=10&page=1 HTTP/1.1" 200 2900 "https://dify.demo.com/datasets/d0bde6c1-b41d-45c9-a16f-0bf632c6c70d/hitTesting" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0.0.0 Safari/537.36 Edg/127.0.0.0" "45.91.81.43"
api-1         | 2024-08-07 01:06:10,348.348 ERROR [Dummy-542] [hit_testing.py:82] - Hit testing failed.
api-1         | Traceback (most recent call last):
api-1         |   File "/app/api/controllers/console/datasets/hit_testing.py", line 56, in post
api-1         |     response = HitTestingService.retrieve(
api-1         |   File "/app/api/services/hit_testing_service.py", line 41, in retrieve
api-1         |     all_documents = RetrievalService.retrieve(retrival_method=retrieval_model.get('search_method', 'semantic_search'),
api-1         |   File "/app/api/core/rag/datasource/retrieval_service.py", line 99, in retrieve
api-1         |     all_documents = data_post_processor.invoke(
api-1         |   File "/app/api/core/rag/data_post_processor/data_post_processor.py", line 27, in invoke
api-1         |     documents = self.rerank_runner.run(query, documents, score_threshold, top_n, user)
api-1         |   File "/app/api/core/rag/rerank/weight_rerank.py", line 47, in run
api-1         |     query_vector_scores = self._calculate_cosine(self.tenant_id, query, documents, self.weights.vector_setting)
api-1         |   File "/app/api/core/rag/rerank/weight_rerank.py", line 162, in _calculate_cosine
api-1         |     content_vector = document.metadata['vector']
api-1         | KeyError: 'vector'
api-1         | 2024-08-07 01:06:10,349.349 ERROR [Dummy-542] [app.py:838] - Exception on /console/api/datasets/d0bde6c1-b41d-45c9-a16f-0bf632c6c70d/hit-testing [POST]
api-1         | Traceback (most recent call last):
api-1         |   File "/app/api/controllers/console/datasets/hit_testing.py", line 56, in post
api-1         |     response = HitTestingService.retrieve(
api-1         |   File "/app/api/services/hit_testing_service.py", line 41, in retrieve
api-1         |     all_documents = RetrievalService.retrieve(retrival_method=retrieval_model.get('search_method', 'semantic_search'),
api-1         |   File "/app/api/core/rag/datasource/retrieval_service.py", line 99, in retrieve
api-1         |     all_documents = data_post_processor.invoke(
api-1         |   File "/app/api/core/rag/data_post_processor/data_post_processor.py", line 27, in invoke
api-1         |     documents = self.rerank_runner.run(query, documents, score_threshold, top_n, user)
api-1         |   File "/app/api/core/rag/rerank/weight_rerank.py", line 47, in run
api-1         |     query_vector_scores = self._calculate_cosine(self.tenant_id, query, documents, self.weights.vector_setting)
api-1         |   File "/app/api/core/rag/rerank/weight_rerank.py", line 162, in _calculate_cosine
api-1         |     content_vector = document.metadata['vector']
api-1         | KeyError: 'vector'
api-1         |
api-1         | During handling of the above exception, another exception occurred:
nginx-1       | 172.20.0.1 - - [07/Aug/2024:01:06:10 +0000] "POST /console/api/datasets/d0bde6c1-b41d-45c9-a16f-0bf632c6c70d/hit-testing HTTP/1.1" 500 72 "https://dify.demo.com/datasets/d0bde6c1-b41d-45c9-a16f-0bf632c6c70d/hitTesting" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0.0.0 Safari/537.36 Edg/127.0.0.0" "45.91.81.43"
api-1         |
api-1         | Traceback (most recent call last):
api-1         |   File "/app/api/.venv/lib/python3.10/site-packages/flask/app.py", line 880, in full_dispatch_request
api-1         |     rv = self.dispatch_request()
api-1         |   File "/app/api/.venv/lib/python3.10/site-packages/flask/app.py", line 865, in dispatch_request
api-1         |     return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)  # type: ignore[no-any-return]
api-1         |   File "/app/api/.venv/lib/python3.10/site-packages/flask_restful/__init__.py", line 489, in wrapper
api-1         |     resp = resource(*args, **kwargs)
api-1         |   File "/app/api/.venv/lib/python3.10/site-packages/flask/views.py", line 110, in view
api-1         |     return current_app.ensure_sync(self.dispatch_request)(**kwargs)  # type: ignore[no-any-return]
api-1         |   File "/app/api/.venv/lib/python3.10/site-packages/flask_restful/__init__.py", line 604, in dispatch_request
api-1         |     resp = meth(*args, **kwargs)
api-1         |   File "/app/api/controllers/console/setup.py", line 75, in decorated
api-1         |     return view(*args, **kwargs)
api-1         |   File "/app/api/libs/login.py", line 91, in decorated_view
api-1         |     return current_app.ensure_sync(func)(*args, **kwargs)
api-1         |   File "/app/api/controllers/console/wraps.py", line 22, in decorated
api-1         |     return view(*args, **kwargs)
api-1         |   File "/app/api/controllers/console/datasets/hit_testing.py", line 83, in post
api-1         |     raise InternalServerError(str(e))
api-1         | werkzeug.exceptions.InternalServerError: 500 Internal Server Error: 'vector'
nginx-1       | 172.20.0.1 - - [07/Aug/2024:01:07:01 +0000] "GET /apps?_rsc=115mj HTTP/1.1" 200 135 "https://dify.demo.com/datasets/d0bde6c1-b41d-45c9-a16f-0bf632c6c70d/hitTesting" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0.0.0 Safari/537.36 Edg/127.0.0.0" "45.91.81.43"

Thanks!

✔️ Expected Behavior

No response

❌ Actual Behavior

No response

dosubot[bot] commented 3 months ago

The 'KeyError: vector' error when switching from 'vector retrieval' to 'mixed retrieval' in Dify version 0.6.16 is likely due to a missing or misconfigured setting related to the vector search component within the mixed retrieval configuration. The mixed retrieval method requires both full-text and vector search configurations. If the vector search settings are not properly defined or missing, it can lead to this error [1].

To resolve this issue, ensure that the vector search settings are correctly configured. Here are the specific configurations required for the vector search component within the mixed retrieval method:

  1. TopK: This filters the text fragments most similar to the user’s query. The default value is 3.
  2. Rerank model: Ensure the API key for the Rerank model is configured in the “Model Provider” page and the “Rerank model” is enabled in the retrieval settings.
  3. Vector Store Configuration: Configure the type of vector store to use. Supported values include weaviate, qdrant, milvus, relyt, pgvector, chroma, opensearch, tidb_vector, oracle, and tencent. Depending on the selected vector store, additional configurations such as endpoint URLs, API keys, and other specific settings are required.

Example configuration for Weaviate:

VECTOR_STORE=weaviate
WEAVIATE_ENDPOINT=http://weaviate:8080
WEAVIATE_API_KEY=WVF5YThaHlkYwhGUSmCRgsX3tD5ngdN8pkih

Example configuration for Qdrant:

VECTOR_STORE=qdrant
QDRANT_URL=http://qdrant:6333
QDRANT_API_KEY=difyai123456
QDRANT_CLIENT_TIMEOUT=20
QDRANT_GRPC_ENABLED=false
QDRANT_GRPC_PORT=6334

Additionally, ensure that the rerank model is selected and valid. If the rerank model is not selected, the system will notify you to configure it before saving the settings [1][2].

By verifying and correctly setting these configurations, you should be able to resolve the 'KeyError: vector' issue when using the mixed retrieval method.

To continue talking to Dosu, mention @dosu.

crazywoola commented 3 months ago

Link https://github.com/langgenius/dify/pull/7012

mengdahuang commented 3 months ago

Link #7012

@crazywoola Thank you for the reply. My Docker was pulled to the latest version yesterday evening. May I ask if I only need to do a docker pull again to fix this problem?

crazywoola commented 3 months ago

Link #7012

@crazywoola Thank you for the reply. My Docker was pulled to the latest version yesterday evening. May I ask if I only need to do a docker pull again to fix this problem?

Not sure if this issue is resolved by this commit. I need to check it with @JohnJyong .

Do a docker pull won't fix this problem, because we haven't release a new version, I think this should remains in latest docker images.

crazywoola commented 3 months ago

BTW, what vector db you are using? This might be a issue related to vector db. I think you can change to another one in the mean time.

mengdahuang commented 3 months ago

BTW, what vector db you are using? This might be a issue related to vector db. I think you can change to another one in the mean time.

Currently, I am using "VECTOR_STORE=weaviate " (Default value)

mengdahuang commented 3 months ago

Link #7012

@crazywoola Thank you for the reply. My Docker was pulled to the latest version yesterday evening. May I ask if I only need to do a docker pull again to fix this problem?

Not sure if this issue is resolved by this commit. I need to check it with @JohnJyong .

Do a docker pull won't fix this problem, because we haven't release a new version, I think this should remains in latest docker images.

All right, thanks for your kindly help.

crazywoola commented 3 months ago

We have fixed this in #7051 , please wait for our new releases.