search

danswer-ai / danswer

Gen-AI Chat for Teams - Think ChatGPT if it had access to your team's unique knowledge.
https://danswer.ai
Other
10.82k stars 1.36k forks source link

Trying to use Ollama with Danswer AI #998

Open voarsh2 opened 10 months ago

voarsh2 commented 10 months ago

I go to the Ollama endpoint:

Ollama is running

I see calls to Ollama (Ollama output):


2024/01/25 02:10:40 ext_server_common.go:158: loaded 0 images
2024-01-25T02:10:40.393669268Z [GIN] 2024/01/25 - 02:10:40 | 200 |        17m53s |    10.42.182.32 | POST     "/api/generate"
[GIN] 2024/01/25 - 02:12:20 | 200 |        19m30s |    10.42.182.32 | POST     "/api/generate"
2024/01/25 02:12:20 ext_server_common.go:158: loaded 0 images
[GIN] 2024/01/25 - 02:14:48 | 200 |        21m58s |    10.42.182.32 | POST     "/api/generate"

API server output:


01/25/2024 02:12:08 AM     query_backend.py 129 : Validating query: How do I use helm rollback?
INFO:     10.42.182.39:50596 - "POST /query/stream-query-validation HTTP/1.1" 200 OK
01/25/2024 02:12:08 AM     query_backend.py 142 : Received query for one shot answer with quotes: How do I use helm rollback?
INFO:     10.42.182.39:50584 - "POST /query/stream-answer-with-quote HTTP/1.1" 200 OK
01/25/2024 02:12:09 AM            timing.py  30 : retrieval_preprocessing took 0.10846328735351562 seconds
01/25/2024 02:12:09 AM            timing.py  30 : embed_query took 0.05180001258850098 seconds
01/25/2024 02:12:14 AM            timing.py  30 : doc_index_retrieval took 5.70013952255249 seconds
2024-01-25T02:12:14.701182580Z 01/25/2024 02:12:14 AM     search_runner.py  51 : Top links from hybrid search: https://confluence.hosttweb.com/display/~voarsh/Helm+Rollback, https://confluence.hosttweb.com/display/~voarsh/Helm+Rollback, https://confluence.hosttweb.com/display/~voarsh/Harbor, https://github.com/PostHog/posthog/pull/19958, https://github.com/PostHog/posthog/pull/19736, https://confluence.atlassian.com/jirasoftwareserver0906/using-your-scrum-backlog-1217302509.html, https://confluence.atlassian.com/jirasoftwareserver0911/using-your-scrum-backlog-1318887867.html, https://confluence.hosttweb.com/display/~voarsh/SonarQube, https://confluence.atlassian.com/jirasoftwareserver0906/using-your-scrum-backlog-1217302509.html, https://confluence.atlassian.com/jirasoftwareserver0911/using-your-scrum-backlog-1318887867.html, https://confluence.atlassian.com/adminjiraserver0904/customizing-email-content-in-jira-server-1188768613.html, https://confluence.atlassian.com/adminjiraserver0904/customizing-email-content-in-jira-server-1188768613.html, https://github.com/PostHog/posthog/pull/18994, https://confluence.hosttweb.com/pages/viewpage.action?pageId=104697187, https://github.com/PostHog/posthog/pull/19605, https://github.com/PostHog/posthog/pull/19471, https://github.com/PostHog/posthog/pull/19707, https://github.com/PostHog/posthog/pull/19569, https://confluence.atlassian.com/jirasoftwareserver0905/building-a-backlog-1207175037.html, https://confluence.atlassian.com/jirasoftwareserver0905/building-a-backlog-1207175037.html, https://confluence.atlassian.com/jirasoftwareserver0906/building-a-backlog-1217302508.html, https://github.com/PostHog/posthog/pull/19743, https://confluence.atlassian.com/jirasoftwareserver0906/building-a-backlog-1217302508.html, https://github.com/PostHog/posthog/pull/19914, https://confluence.atlassian.com/adminjiraserver0904/getting-started-with-jira-data-cent
er-on-aws-1188767785.html, https://github.com/PostHog/posthog/pull/19914, https://confluence.atlassian.com/jirasoftwareserver0906/using-your-scrum-backlog-1217302509.html, https://confluence.atlassian.com/jirasoftwareserver0911/using-your-scrum-backlog-1318887867.html, https://confluence.atlassian.com/adminjiraserver0904/customizing-email-content-in-jira-server-1188768613.html, https://confluence.atlassian.com/adminjiraserver0904/connecting-to-crowd-or-another-jira-application-for-user-management-1188767900.html, https://confluence.atlassian.com/jirasoftwareserver/using-your-kanban-backlog-938845378.html, https://confluence.atlassian.com/jirasoftwareserver0905/using-your-kanban-backlog-1207175056.html, https://github.com/PostHog/posthog/pull/19700, https://confluence.atlassian.com/adminjiraserver0904/connecting-to-crowd-or-another-jira-application-for-user-management-1188767900.html, https://confluence.atlassian.com/jirasoftwareserver0905/installing-jira-software-1207174853.html, https://confluence.atlassian.com/jirasoftwareserver0905/running-sprints-in-a-scrum-project-1207175114.html, https://github.com/PostHog/posthog/pull/19700, https://github.com/PostHog/posthog/pull/17026, https://github.com/PostHog/posthog/pull/19416, https://github.com/PostHog/posthog/pull/19914, https://github.com/PostHog/posthog/pull/19946, https://confluence.atlassian.com/jirasoftwareserver0906/using-your-scrum-backlog-1217302509.html, https://github.com/PostHog/posthog/pull/19910, https://github.com/PostHog/posthog/pull/19654, https://confluence.atlassian.com/jirasoftwareserver0911/using-your-scrum-backlog-1318887867.html, https://github.com/PostHog/posthog/pull/19868, https://github.com/PostHog/posthog/pull/19652, https://confluence.atlassian.com/jirasoftwareserver0905/plan-your-work-in-advanced-roadmaps-1207175333.html, https://github.com/PostHog/posthog/pull/19506, https://confluence.atlassian.com/adminjiraserver0904/getting-started-with-jira-data-center-on-aws-1188767785.html
INFO:     10.42.182.39:49358 - "GET /health HTTP/1.1" 200 OK
INFO:     10.42.182.39:57704 - "GET /health HTTP/1.1" 200 OK
INFO:     10.42.235.193:41824 - "GET /auth/type HTTP/1.1" 200 OK
INFO:     10.42.235.193:41832 - "GET /manage/me HTTP/1.1" 200 OK
INFO:     10.42.235.193:41874 - "GET /chat/get-user-chat-sessions HTTP/1.1" 200 OK
INFO:     10.42.235.193:41856 - "GET /manage/document-set HTTP/1.1" 200 OK
INFO:     10.42.235.193:41846 - "GET /manage/connector HTTP/1.1" 200 OK
INFO:     10.42.235.193:41882 - "GET /query/valid-tags HTTP/1.1" 200 OK
INFO:     10.42.235.193:41870 - "GET /persona?include_default=true HTTP/1.1" 200 OK
INFO:     10.42.182.39:42466 - "GET /health HTTP/1.1" 200 OK
INFO:     10.42.182.39:42470 - "HEAD /manage/admin/genai-api-key/validate HTTP/1.1" 200 OK
2024-01-25T02:13:18.390421576Z INFO:     10.42.182.39:42472 - "HEAD /manage/admin/genai-api-key/validate HTTP/1.1" 200 OK
INFO:     10.42.235.193:41824 - "GET /auth/type HTTP/1.1" 200 OK
INFO:     10.42.235.193:41874 - "GET /chat/get-user-chat-sessions HTTP/1.1" 200 OK
INFO:     10.42.235.193:41832 - "GET /manage/me HTTP/1.1" 200 OK
INFO:     10.42.235.193:41856 - "GET /manage/document-set HTTP/1.1" 200 OK
INFO:     10.42.235.193:41870 - "GET /persona?include_default=true HTTP/1.1" 200 OK
INFO:     10.42.235.193:41846 - "GET /manage/connector HTTP/1.1" 200 OK
INFO:     10.42.235.193:41882 - "GET /query/valid-tags HTTP/1.1" 200 OK
INFO:     10.42.235.193:41824 - "GET /auth/type HTTP/1.1" 200 OK
INFO:     10.42.235.193:41832 - "GET /manage/me HTTP/1.1" 200 OK
INFO:     10.42.235.193:41846 - "GET /manage/connector HTTP/1.1" 200 OK
INFO:     10.42.235.193:41856 - "GET /manage/document-set HTTP/1.1" 200 OK
INFO:     10.42.235.193:41874 - "GET /chat/get-user-chat-sessions HTTP/1.1" 200 OK
INFO:     10.42.235.193:41882 - "GET /query/valid-tags HTTP/1.1" 200 OK
INFO:     10.42.235.193:41870 - "GET /persona?include_default=true HTTP/1.1" 200 OK
INFO:     10.42.182.39:42482 - "GET /chat/get-chat-session/6 HTTP/1.1" 200 OK
01/25/2024 02:13:23 AM      chat_backend.py 170 : Received new chat message: How do I use helm rollback?
2024-01-25T02:13:23.323563144Z INFO:     10.42.182.39:42494 - "POST /chat/send-message HTTP/1.1" 200 OK
INFO:     10.42.182.39:36718 - "GET /health HTTP/1.1" 200 OK
INFO:     10.42.182.39:49774 - "GET /health HTTP/1.1" 200 OK

Danswer replies within chat:

Unexpected token '<', "<!DOCTYPE "... is not valid JSON

Search shows no docs, but that's clearly not true.......

Unexpected token '<', "<!DOCTYPE "... is not valid JSON

Danswer config:

# Auth Setting, also check the secrets file
  AUTH_TYPE: "basic"  # Change this for production uses unless Danswer is only accessible behind VPN
  SESSION_EXPIRE_TIME_SECONDS: "1286400"  # 1 Day Default
  VALID_EMAIL_DOMAINS: ""  # Can be something like danswer.ai, as an extra double-check
  SMTP_SERVER: "" # For sending verification emails, if unspecified then defaults to 'smtp.gmail.com'
  SMTP_PORT: "" # For sending verification emails, if unspecified then defaults to '587'
  SMTP_USER: ""  # 'your-email@company.com'
  SMTP_PASS: ""  # 'your-gmail-password'
  EMAIL_FROM: ""  # 'your-email@company.com' SMTP_USER missing used instead
  # Gen AI Settings
  GEN_AI_MODEL_PROVIDER: "ollama"
  GEN_AI_MODEL_VERSION: "llama2"  # Use GPT-4 if you have it
  FAST_GEN_AI_MODEL_VERSION: "" # gpt-3.5-turbo
  GEN_AI_API_KEY: ""
  GEN_AI_API_ENDPOINT: "http://192.168.100.103:30999"
  GEN_AI_API_VERSION: ""
  GEN_AI_LLM_PROVIDER_TYPE: ""
  QA_TIMEOUT: "1200"
  NUM_DOCUMENT_TOKENS_FED_TO_GENERATIVE_MODEL: ""
  DISABLE_LLM_FILTER_EXTRACTION: "True"
  DISABLE_LLM_CHUNK_FILTER: "True"
  DISABLE_LLM_CHOOSE_SEARCH: ""
  # Query Options
  DOC_TIME_DECAY: ""
  HYBRID_ALPHA: ""
  EDIT_KEYWORD_QUERY: ""
  MULTILINGUAL_QUERY_EXPANSION: ""
  QA_PROMPT_OVERRIDE: "weak"
  # Other Services
  POSTGRES_HOST: "relational-db-service"
  VESPA_HOST: "document-index-service"
  # Don't change the NLP models unless you know what you're doing
  DOCUMENT_ENCODER_MODEL: ""
  NORMALIZE_EMBEDDINGS: ""
  ASYM_QUERY_PREFIX: ""
  ASYM_PASSAGE_PREFIX: ""
  ENABLE_RERANKING_REAL_TIME_FLOW: ""
  ENABLE_RERANKING_ASYNC_FLOW: ""
  MODEL_SERVER_HOST: ""
  MODEL_SERVER_PORT: ""
  INDEXING_MODEL_SERVER_HOST: ""
  MIN_THREADS_ML_MODELS: ""
  # Indexing Configs
  NUM_INDEXING_WORKERS: ""
  DASK_JOB_CLIENT_ENABLED: ""
  CONTINUE_ON_CONNECTOR_FAILURE: ""
  EXPERIMENTAL_CHECKPOINTING_ENABLED: ""
  CONFLUENCE_CONNECTOR_LABELS_TO_SKIP: ""
  GONG_CONNECTOR_START_TIME: ""
  NOTION_CONNECTOR_ENABLE_RECURSIVE_PAGE_LOOKUP: ""
  # DanswerBot SlackBot Configs
  DANSWER_BOT_SLACK_APP_TOKEN: ""
  DANSWER_BOT_SLACK_BOT_TOKEN: ""
  DANSWER_BOT_DISABLE_DOCS_ONLY_ANSWER: ""
  DANSWER_BOT_DISPLAY_ERROR_MSGS: ""
  DANSWER_BOT_RESPOND_EVERY_CHANNEL: ""
  DANSWER_BOT_DISABLE_COT: ""  # Currently unused
  NOTIFY_SLACKBOT_NO_ANSWER: ""
  # Logging
  # Optional Telemetry, please keep it on (nothing sensitive is collected)? <3
  # https://docs.danswer.dev/more/telemetry
  DISABLE_TELEMETRY: ""
  LOG_LEVEL: ""
  LOG_ALL_MODEL_INTERACTIONS: ""
  LOG_VESPA_TIMING_INFORMATION: ""
  # Shared or Non-backend Related
  INTERNAL_URL: "http://api-server-service:80"  # for web server
  WEB_DOMAIN: "DOMAIN"  # for web server and api server
  DOMAIN: "DOMAIN"  # for nginx

Seems I only get results with search, but not chat: image

audip commented 9 months ago

Can you try setting this environment variable?

# Use only 1 section from the documents and do not require quotes
QA_PROMPT_OVERRIDE=weak

Found it on this page