search

khoj-ai / khoj

Your AI second brain. Self-hostable. Get answers from the web or your docs. Build custom agents, schedule automations, do deep research. Turn any online or local LLM into your personal, autonomous AI (e.g gpt, claude, gemini, llama, qwen, mistral).
https://khoj.dev
GNU Affero General Public License v3.0
16.49k stars 802 forks source link

The local ollama cannot be connected #777

Closed mingLvft closed 2 months ago

mingLvft commented 6 months ago

微信截图_20240529094631 微信截图_20240529094546 The configuration is correct. The ollama container network communication is normal

mingLvft commented 6 months ago

微信截图_20240529094752 /api/chat?q=1&n=5&client=web&stream=true&conversation_id=8&region=null&city=null&country=null&timezone=null

maciekpoplawski commented 6 months ago

Hi! Please try adding /v1 at the end of Base URL Example: http://host.docker.internal:11434/v1 v1 means OpenAI API compatible mode Hope this helps! ❤️

mingLvft commented 6 months ago

Thank you for your answer, but it doesn't work. What is the problem? Other applications can connect to my ollama

mingLvft commented 6 months ago

Unable to respond to any action when asked

maciekpoplawski commented 6 months ago

I've only got one last idea. Maybe the space in API key is breaking something? Put it like "anystring"

yusfix commented 6 months ago

Change API key to "not-needed" (I am just guessing it works fine in my machine). Hope you find help

sabaimran commented 6 months ago

Echoing what the previous two users have said, you'll need for it to be /v1/ and perhaps replace with a string not containing a space in the API key.

Here's a link to our Ollama documentation: https://docs.khoj.dev/miscellaneous/ollama/. I've just tested it out recently.

mingLvft commented 5 months ago

It still doesn't work.

mingLvft commented 5 months ago

与前两位用户所说的相呼应，您需要将其/v1/替换为 API 密钥中不包含空格的字符串。

这是我们的 Ollama 文档的链接：https://docs.khoj.dev/miscellaneous/ollama/。我最近刚刚测试过它。

It still doesn't work.

saccohuo commented 5 months ago

Same problem on Windows. Self-host khoj.

PS C:\WINDOWS\system32> khoj --anonymous-mode
[00:49:27.364149] WARNING  khoj.routers.email: RESEND_API_KEY not set - email sending disabled               email.py:26
[00:49:27.410661] INFO     khoj: 🚒 Initializing Khoj v1.14.0                                                main.py:124
[00:49:27.412661] INFO     khoj: 📦 Initializing DB:                                                         main.py:125
                           Operations to perform:
                             Apply all migrations: admin, auth, contenttypes, database, django_apscheduler,
                           sessions
                           Running migrations:
                             No migrations to apply.
[00:49:27.421665] INFO     khoj: 🌘 Starting Khoj                                                            main.py:138
[00:49:27.623808] INFO     khoj.configure: 🚨 Khoj is not configured.                                   configure.py:198
                           Initializing it with a default config.
C:\Python\Python311\Lib\site-packages\huggingface_hub\file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
  warnings.warn(
[00:49:31.710370] INFO     khoj.configure: 📡 Telemetry enabled                                         configure.py:240
[00:49:31.716367] INFO     khoj: 🚒 Initializing Khoj v1.14.0                                                main.py:124
[00:49:31.717366] INFO     khoj: 📦 Initializing DB:                                                         main.py:125
                           Operations to perform:
                             Apply all migrations: admin, auth, contenttypes, database, django_apscheduler,
                           sessions
                           Running migrations:
                             No migrations to apply.
[00:49:31.720369] INFO     khoj: 🌘 Starting Khoj                                                            main.py:138
[00:49:31.776737] INFO     khoj.configure: 🚨 Khoj is not configured.                                   configure.py:198
                           Initializing it with a default config.
[00:49:35.645573] INFO     khoj.configure: 📡 Telemetry enabled                                         configure.py:240
[00:49:35.646572] INFO     khoj: 🌖 Khoj is ready to use                                                     main.py:190
[00:49:35.674664] INFO     uvicorn.error: Started server process [38712]                                    server.py:75
[00:49:35.676655] INFO     uvicorn.error: Waiting for application startup.                                      on.py:45
[00:49:35.681656] INFO     uvicorn.error: Application startup complete.                                         on.py:59
[00:49:35.683654] INFO     uvicorn.error: Uvicorn running on http://127.0.0.1:42110 (Press CTRL+C to quit) server.py:206
[00:50:19.377841] INFO     uvicorn.access: 127.0.0.1:55643 - "GET /api/chat/options HTTP/1.1" 200  httptools_impl.py:437
[00:50:19.384851] INFO     uvicorn.access: 127.0.0.1:55642 - "GET                                  httptools_impl.py:437
                           /api/chat/starters?client=desktop HTTP/1.1" 200
[00:50:19.460802] INFO     uvicorn.access: 127.0.0.1:55640 - "GET                                  httptools_impl.py:437
                           /api/chat/history?client=desktop&n=10 HTTP/1.1" 200
[00:50:19.489799] INFO     uvicorn.access: 127.0.0.1:55641 - "GET /api/chat/sessions HTTP/1.1" 200 httptools_impl.py:437
[00:50:19.519461] INFO     uvicorn.access: 127.0.0.1:55638 - "GET /api/v1/user?client=desktop      httptools_impl.py:437
                           HTTP/1.1" 200
[00:50:24.374164] INFO     uvicorn.access: 127.0.0.1:55672 - "GET                                  httptools_impl.py:437
                           /server/admin/database/openaiprocessorconversationconfig/?o=4 HTTP/1.1"
                           200
[00:51:27.172415] INFO     uvicorn.access: 127.0.0.1:55806 - "GET /api/v1/user?client=desktop      httptools_impl.py:437
                           HTTP/1.1" 200
[00:51:27.440925] INFO     khoj.configure: 📡 Uploading telemetry to                                    configure.py:353
                           https://khoj.beta.haletic.com/v1/telemetry...
[00:51:31.551088] INFO     uvicorn.access: 127.0.0.1:55816 - "GET /api/v1/user?client=desktop      httptools_impl.py:437
                           HTTP/1.1" 200
[00:51:31.570087] INFO     uvicorn.access: 127.0.0.1:55820 - "GET /api/chat/options HTTP/1.1" 200  httptools_impl.py:437
[00:51:31.572089] INFO     uvicorn.access: 127.0.0.1:55817 - "GET                                  httptools_impl.py:437
                           /api/chat/history?client=desktop&n=10 HTTP/1.1" 200
[00:51:31.575089] INFO     uvicorn.access: 127.0.0.1:55819 - "GET                                  httptools_impl.py:437
                           /api/chat/starters?client=desktop HTTP/1.1" 200
[00:51:31.657301] INFO     uvicorn.access: 127.0.0.1:55818 - "GET /api/chat/sessions HTTP/1.1" 200 httptools_impl.py:437
[00:51:35.745320] INFO     khoj.routers.api_chat: Chat request by default: hi                            api_chat.py:812
[00:53:31.828086] INFO     khoj.configure: 📡 Uploading telemetry to                                    configure.py:353
                           https://khoj.beta.haletic.com/v1/telemetry...
You are using the default legacy behaviour of the <class 'transformers.models.llama.tokenization_llama_fast.LlamaTokenizerFast'>. This is expected, and simply means that the `legacy` (previous) behavior will be used so nothing changes for you. If you want to use the new behaviour, set `legacy=False`. This should only be set if you understand what it means, and thoroughly read the reason why this was added as explained in https://github.com/huggingface/transformers/pull/24565
[00:54:02.803417] WARNING  khoj.processor.conversation.utils: Fallback to default chat model tokenizer:     utils.py:217
                           None.
                           Configure tokenizer for unsupported model: llama3:8b in Khoj settings to improve
                           context stuffing.
[00:54:02.810008] INFO     uvicorn.access: 127.0.0.1:55818 - "GET                                  httptools_impl.py:437
                           /api/chat?q=hi&n=5&client=web&stream=true&conversation_id=3&region=Cent
                           ral%20and%20Western&city=Central&country=Hong%20Kong&timezone=Asia/Hong
                           _Kong HTTP/1.1" 200
Exception in thread Thread-12 (llm_thread):
Traceback (most recent call last):
  File "C:\Python\Python311\Lib\threading.py", line 1038, in _bootstrap_inner
    self.run()
  File "C:\Python\Python311\Lib\threading.py", line 975, in run
    self._target(*self._args, **self._kwargs)
  File "C:\Python\Python311\Lib\site-packages\khoj\processor\conversation\openai\utils.py", line 114, in llm_thread
    chat = client.chat.completions.create(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Python\Python311\Lib\site-packages\openai\_utils\_utils.py", line 277, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "C:\Python\Python311\Lib\site-packages\openai\resources\chat\completions.py", line 606, in create
    return self._post(
           ^^^^^^^^^^^
  File "C:\Python\Python311\Lib\site-packages\openai\_base_client.py", line 1240, in post
    return cast(ResponseT, self.request(cast_to, opts, stream=stream, stream_cls=stream_cls))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Python\Python311\Lib\site-packages\openai\_base_client.py", line 921, in request
    return self._request(
           ^^^^^^^^^^^^^^
  File "C:\Python\Python311\Lib\site-packages\openai\_base_client.py", line 1005, in _request
    return self._retry_request(
           ^^^^^^^^^^^^^^^^^^^^
  File "C:\Python\Python311\Lib\site-packages\openai\_base_client.py", line 1053, in _retry_request
    return self._request(
           ^^^^^^^^^^^^^^
  File "C:\Python\Python311\Lib\site-packages\openai\_base_client.py", line 1005, in _request
    return self._retry_request(
           ^^^^^^^^^^^^^^^^^^^^
  File "C:\Python\Python311\Lib\site-packages\openai\_base_client.py", line 1053, in _retry_request
    return self._request(
           ^^^^^^^^^^^^^^
  File "C:\Python\Python311\Lib\site-packages\openai\_base_client.py", line 1020, in _request
    raise self._make_status_error_from_response(err.response) from None
openai.InternalServerError: Error code: 503
sabaimran commented 5 months ago

@saccohuo , could you verify that you're on the latest version of Khoj? You can run pip install khoj-assistant --upgrade.

Can you also verify the chat model name you've configured exactly matches the ollama model that you're running? Unfortunately the stack trace doesn't give all that much to go off of, except that indeed the error is happening when making the request to your ollama server.

saccohuo commented 5 months ago

@saccohuo , could you verify that you're on the latest version of Khoj? You can run pip install khoj-assistant --upgrade.@saccohuo，能否验证您是否在使用最新版本的Khoj？您可以运行pip install khoj-assistant --upgrade。

Can you also verify the chat model name you've configured exactly matches the ollama model that you're running? Unfortunately the stack trace doesn't give all that much to go off of, except that indeed the error is happening when making the request to your ollama server.您能否验证聊天模型名称是否与您正在运行的ollama模型完全匹配？不幸的是，堆栈跟踪没有提供太多信息，除了确实错误是在向ollama服务器发出请求时发生的。

@sabaimran After upgrade, the same error still.

sabaimran commented 2 months ago

@saccohuo , do you mind sharing your chat model options configuration for the ollama model? you can just post a screenshot.

sabaimran commented 2 months ago

Hey folks, since we haven't gotten an update in a while and I can't repro the issue, I'm going to close this for now. Feel free to comment here again if you encounter further problems!

Kolterdyx commented 1 week ago

Just to add to this:

I was trying to connect a khoj container I am running in a homelab, in a portainer instance, to an Ollama LXC I also host. I tried the url base as the docs mention, ending in /v1/, but that didn't work. I tried without it too, I tried going through the reverse http proxy or connecting directly, with and without SSL. Nothing worked. Then I tried with /v1 (notice the lack of a trailing slash), and that worked just fine!

sabaimran commented 1 week ago

Thanks for noting that, @Kolterdyx ! I'll update our documentation to fix that today.

EDIT: Actually, looking through our code, that seems a bit strange. Might be something particular to your server, because with trailing slash does work just fine with Khoj + my local Ollama.