search

QuivrHQ / quivr

Open-source RAG Framework for building GenAI Second Brains 🧠 Build productivity assistant (RAG) ⚡️🤖 Chat with your docs (PDF, CSV, ...) & apps using Langchain, GPT 3.5 / 4 turbo, Private, Anthropic, VertexAI, Ollama, LLMs, Groq that you can share with users ! Efficient retrieval augmented generation framework
https://quivr.com
Other
36.15k stars 3.51k forks source link

[Bug]: blank response with ollama backend #2694

Closed travisgu closed 4 days ago

travisgu commented 3 months ago

What happened?

When I tried to use Quivr with Ollam qwen:72b model, the chat is already empty. There is no error showing in the log or web page. I checked the superbase chat history table the assistant text is empty:

image

Relevant log output

backend-core  | INFO:     127.0.0.1:51044 - "GET /healthz HTTP/1.1" 200 OK
backend-core  | INFO:     127.0.0.1:51088 - "GET /healthz HTTP/1.1" 200 OK
beat          | [2024-06-20 08:42:00,000: INFO/MainProcess] Scheduler: Sending due task process_premium_users (check_if_is_premium_user)
beat          | [2024-06-20 08:42:00,003: INFO/MainProcess] Scheduler: Sending due task process_sync_active (process_sync_active)
worker        | [2024-06-20 08:42:00,004: INFO/MainProcess] Task check_if_is_premium_user[ed7d1da4-7965-4b03-aa6c-d08f6d1b4953] received
worker        | [2024-06-20 08:42:00,009: INFO/MainProcess] Task process_sync_active[3e087e91-ba04-484a-8f1c-0d363f90dcc2] received
worker        | [2024-06-20 08:42:00,050: INFO/ForkPoolWorker-3] HTTP Request: GET http://host.docker.internal:54321/rest/v1/syncs_active?select=%2A&last_synced=lt.2024-06-20T02%3A42%3A00.012042 "HTTP/1.1 200 OK"
worker        | [2024-06-20 08:42:00,059: INFO/ForkPoolWorker-3] HTTP Request: GET http://host.docker.internal:54321/rest/v1/syncs_active?select=%2A&force_sync=eq.True "HTTP/1.1 200 OK"
worker        | [2024-06-20 08:42:00,062: INFO/ForkPoolWorker-3] Task process_sync_active[3e087e91-ba04-484a-8f1c-0d363f90dcc2] succeeded in 0.05039690800185781s: None
worker        | [2024-06-20 08:42:00,842: INFO/ForkPoolWorker-2] HTTP Request: GET http://host.docker.internal:54321/rest/v1/subscriptions?select=%2A&current_period_end=gt.2024-06-20%2010%3A42%3A00.006442 "HTTP/1.1 200 OK"
worker        | [2024-06-20 08:42:01,493: INFO/ForkPoolWorker-2] HTTP Request: GET http://host.docker.internal:54321/rest/v1/customers?select=%2A&order=created.desc "HTTP/1.1 200 OK"
worker        | [2024-06-20 08:42:01,501: INFO/ForkPoolWorker-2] HTTP Request: GET http://host.docker.internal:54321/rest/v1/product_to_features?select=%2A "HTTP/1.1 200 OK"
worker        | [2024-06-20 08:42:01,508: INFO/ForkPoolWorker-2] HTTP Request: GET http://host.docker.internal:54321/rest/v1/users?select=id&email=eq.admin%40quivr.app "HTTP/1.1 200 OK"
worker        | [2024-06-20 08:42:01,516: INFO/ForkPoolWorker-2] Task check_if_is_premium_user[ed7d1da4-7965-4b03-aa6c-d08f6d1b4953] succeeded in 1.5096762990033312s: True
backend-core  | INFO:     172.26.0.1:56690 - "GET /user HTTP/1.1" 200 OK
backend-core  | INFO:     172.26.0.1:56692 - "GET /chat HTTP/1.1" 200 OK
backend-core  | INFO:     172.26.0.1:56696 - "GET /user/identity HTTP/1.1" 200 OK
backend-core  | INFO:     172.26.0.1:56694 - "GET /onboarding HTTP/1.1" 200 OK
backend-core  | INFO:     172.26.0.1:56698 - "GET /brains/a0950bc4-0e84-4bb9-aecb-2aac255061da/ HTTP/1.1" 200 OK
backend-core  | INFO:     172.26.0.1:56694 - "GET /brains/ HTTP/1.1" 200 OK
backend-core  | INFO:     172.26.0.1:56822 - "GET /onboarding HTTP/1.1" 200 OK
backend-core  | INFO:     172.26.0.1:56818 - "GET /user HTTP/1.1" 200 OK
backend-core  | INFO:     172.26.0.1:56828 - "GET /user/identity HTTP/1.1" 200 OK
backend-core  | INFO:     172.26.0.1:56824 - "GET /chat HTTP/1.1" 200 OK
backend-core  | INFO:     172.26.0.1:56828 - "GET /brains/ HTTP/1.1" 200 OK
backend-core  | INFO:     172.26.0.1:56826 - "GET /brains/a0950bc4-0e84-4bb9-aecb-2aac255061da/ HTTP/1.1" 200 OK
backend-core  | INFO:     127.0.0.1:51286 - "GET /healthz HTTP/1.1" 200 OK
backend-core  | INFO:     172.26.0.1:56826 - "OPTIONS /chat/79148a96-9cb3-490f-ab32-c848b71eda35/question/stream?brain_id=a0950bc4-0e84-4bb9-aecb-2aac255061da HTTP/1.1" 200 OK
backend-core  | INFO:     172.26.0.1:56824 - "GET /user HTTP/1.1" 200 OK
backend-core  | INFO:     172.26.0.1:56828 - "GET /onboarding HTTP/1.1" 200 OK
backend-core  | INFO:     172.26.0.1:56818 - "GET /user/identity HTTP/1.1" 200 OK
backend-core  | INFO:     172.26.0.1:56826 - "POST /chat/79148a96-9cb3-490f-ab32-c848b71eda35/question/stream?brain_id=a0950bc4-0e84-4bb9-aecb-2aac255061da HTTP/1.1" 200 OK
backend-core  | INFO:     127.0.0.1:51386 - "GET /healthz HTTP/1.1" 200 OK
beat          | [2024-06-20 08:43:00,000: INFO/MainProcess] Scheduler: Sending due task process_sync_active (process_sync_active)
worker        | [2024-06-20 08:43:00,004: INFO/MainProcess] Task process_sync_active[be12573f-e929-4138-ba8a-90ece81ffcd6] received
beat          | [2024-06-20 08:43:00,005: INFO/MainProcess] Scheduler: Sending due task process_premium_users (check_if_is_premium_user)
worker        | [2024-06-20 08:43:00,009: INFO/MainProcess] Task check_if_is_premium_user[7cbdd17c-492e-4c4e-a5cb-fea70cdfd3a1] received
worker        | [2024-06-20 08:43:00,016: INFO/ForkPoolWorker-2] HTTP Request: GET http://host.docker.internal:54321/rest/v1/syncs_active?select=%2A&last_synced=lt.2024-06-20T02%3A43%3A00.006841 "HTTP/1.1 200 OK"
worker        | [2024-06-20 08:43:00,043: INFO/ForkPoolWorker-2] HTTP Request: GET http://host.docker.internal:54321/rest/v1/syncs_active?select=%2A&force_sync=eq.True "HTTP/1.1 200 OK"
worker        | [2024-06-20 08:43:00,045: INFO/ForkPoolWorker-2] Task process_sync_active[be12573f-e929-4138-ba8a-90ece81ffcd6] succeeded in 0.03935996099971817s: None
redis         | 1:M 20 Jun 2024 08:43:00.096 * 100 changes in 300 seconds. Saving...
redis         | 1:M 20 Jun 2024 08:43:00.097 * Background saving started by pid 25
redis         | 25:C 20 Jun 2024 08:43:00.102 * DB saved on disk
redis         | 25:C 20 Jun 2024 08:43:00.102 * Fork CoW for RDB: current 0 MB, peak 0 MB, average 0 MB
redis         | 1:M 20 Jun 2024 08:43:00.197 * Background saving terminated with success
worker        | [2024-06-20 08:43:00,709: INFO/ForkPoolWorker-3] HTTP Request: GET http://host.docker.internal:54321/rest/v1/subscriptions?select=%2A&current_period_end=gt.2024-06-20%2010%3A43%3A00.012100 "HTTP/1.1 200 OK"
worker        | [2024-06-20 08:43:01,276: INFO/ForkPoolWorker-3] HTTP Request: GET http://host.docker.internal:54321/rest/v1/customers?select=%2A&order=created.desc "HTTP/1.1 200 OK"
worker        | [2024-06-20 08:43:01,283: INFO/ForkPoolWorker-3] HTTP Request: GET http://host.docker.internal:54321/rest/v1/product_to_features?select=%2A "HTTP/1.1 200 OK"
worker        | [2024-06-20 08:43:01,289: INFO/ForkPoolWorker-3] HTTP Request: GET http://host.docker.internal:54321/rest/v1/users?select=id&email=eq.admin%40quivr.app "HTTP/1.1 200 OK"
worker        | [2024-06-20 08:43:01,290: INFO/ForkPoolWorker-3] Task check_if_is_premium_user[7cbdd17c-492e-4c4e-a5cb-fea70cdfd3a1] succeeded in 1.278936637998413s: True
backend-core  | INFO:     172.26.0.1:57000 - "GET /onboarding HTTP/1.1" 200 OK
backend-core  | INFO:     172.26.0.1:57004 - "GET /user HTTP/1.1" 200 OK
backend-core  | INFO:     172.26.0.1:57006 - "GET /brains/a0950bc4-0e84-4bb9-aecb-2aac255061da/ HTTP/1.1" 200 OK
backend-core  | INFO:     172.26.0.1:56826 - "GET /user/credits HTTP/1.1" 200 OK
backend-core  | INFO:     172.26.0.1:57002 - "GET /user/identity HTTP/1.1" 200 OK
backend-core  | INFO:     127.0.0.1:51460 - "GET /healthz HTTP/1.1" 200 OK
backend-core  | INFO:     127.0.0.1:51522 - "GET /healthz HTTP/1.1" 200 OK
beat          | [2024-06-20 08:44:00,000: INFO/MainProcess] Scheduler: Sending due task process_premium_users (check_if_is_premium_user)
beat          | [2024-06-20 08:44:00,004: INFO/MainProcess] Scheduler: Sending due task process_sync_active (process_sync_active)

Twitter / LinkedIn details

No response

dosubot[bot] commented 3 months ago

Hey there, @travisgu! I'm here to assist you with any issues you're experiencing while using Quivr. I can help you solve bugs, answer questions, and even guide you on how to become a contributor to the project. Let's get started!

To resolve the issue of blank chat responses when using the Ollam qwen:72b model in Quivr, ensure that the prompt and assistant columns in the Supabase chat history table are being correctly populated. Here are the steps to address this:

  1. Check the generate_stream and generate_answer methods: Ensure these methods are correctly saving the responses to the database.

  2. Update the save_answer method: Ensure this method correctly updates the prompt and assistant columns in the Supabase chat history table.

Here is an example of how you might update the save_answer method:

def save_answer(self, question, response_tokens, streamed_chat_history, save_answer):
    # Combine response tokens into a single response
    response = ''.join(response_tokens)

    # Update the chat history with the response
    streamed_chat_history.assistant = response

    # Save the chat history to the database
    if save_answer:
        chat_service.save_chat_history(
            chat_id=question.chat_id,
            question=question.question,
            answer=response,
            prompt=streamed_chat_history.prompt,
            assistant=streamed_chat_history.assistant
        )
  1. Verify .env configuration: Ensure the OLLAMA_API_BASE_URL is uncommented and correctly pointing to the local URL for the Ollama API:
# LOCAL
OLLAMA_API_BASE_URL=http://host.docker.internal:11434 # Uncomment to activate ollama. This is the local url for the ollama api
  1. Add additional logging: If the issue persists, add logging to trace where the data might be getting lost or not being saved correctly.

Ensure that the generate_answer and generate_stream methods in the backend/modules/brain/qa_headless.py file are correctly handling the chat history updates and saving the responses to the database [1][2][3].

To continue talking to Dosu, mention @dosu.

travisgu commented 3 months ago

I didn't change any code and I was running quivr with the docker image. The url configuration is correct. I can see the prompt in the Ollama log file and Ollama return 200 for the API call. @dosu

mkhludnev commented 3 months ago

what's the model name you use? I find that it should be prefixed with ollama/

travisgu commented 3 months ago

what's the model name you use? I find that it should be prefixed with ollama/

right，the model name is “ollama/qwen:72b”

travisgu commented 3 months ago

btw, I am using the quivr with docker-compose on my laptop, I am not sure if this is cause.

github-actions[bot] commented 1 week ago

Thanks for your contributions, we'll be closing this issue as it has gone stale. Feel free to reopen if you'd like to continue the discussion.