Open mingLvft opened 3 months ago
The configuration is correct. The ollama container network communication is normal
/api/chat?q=1&n=5&client=web&stream=true&conversation_id=8®ion=null&city=null&country=null&timezone=null
Hi!
Please try adding /v1 at the end of Base URL
Example: http://host.docker.internal:11434/v1
v1 means OpenAI API compatible mode
Hope this helps! ❤️
Thank you for your answer, but it doesn't work. What is the problem? Other applications can connect to my ollama
Unable to respond to any action when asked
I've only got one last idea. Maybe the space in API key is breaking something? Put it like "anystring"
Change API key to "not-needed" (I am just guessing it works fine in my machine). Hope you find help
Echoing what the previous two users have said, you'll need for it to be /v1/
and perhaps replace with a string not containing a space in the API key.
Here's a link to our Ollama documentation: https://docs.khoj.dev/miscellaneous/ollama/. I've just tested it out recently.
It still doesn't work.
与前两位用户所说的相呼应,您需要将其
/v1/
替换为 API 密钥中不包含空格的字符串。这是我们的 Ollama 文档的链接:https://docs.khoj.dev/miscellaneous/ollama/。我最近刚刚测试过它。
It still doesn't work.
Same problem on Windows. Self-host khoj.
PS C:\WINDOWS\system32> khoj --anonymous-mode
[00:49:27.364149] WARNING khoj.routers.email: RESEND_API_KEY not set - email sending disabled email.py:26
[00:49:27.410661] INFO khoj: 🚒 Initializing Khoj v1.14.0 main.py:124
[00:49:27.412661] INFO khoj: 📦 Initializing DB: main.py:125
Operations to perform:
Apply all migrations: admin, auth, contenttypes, database, django_apscheduler,
sessions
Running migrations:
No migrations to apply.
[00:49:27.421665] INFO khoj: 🌘 Starting Khoj main.py:138
[00:49:27.623808] INFO khoj.configure: 🚨 Khoj is not configured. configure.py:198
Initializing it with a default config.
C:\Python\Python311\Lib\site-packages\huggingface_hub\file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
warnings.warn(
[00:49:31.710370] INFO khoj.configure: 📡 Telemetry enabled configure.py:240
[00:49:31.716367] INFO khoj: 🚒 Initializing Khoj v1.14.0 main.py:124
[00:49:31.717366] INFO khoj: 📦 Initializing DB: main.py:125
Operations to perform:
Apply all migrations: admin, auth, contenttypes, database, django_apscheduler,
sessions
Running migrations:
No migrations to apply.
[00:49:31.720369] INFO khoj: 🌘 Starting Khoj main.py:138
[00:49:31.776737] INFO khoj.configure: 🚨 Khoj is not configured. configure.py:198
Initializing it with a default config.
[00:49:35.645573] INFO khoj.configure: 📡 Telemetry enabled configure.py:240
[00:49:35.646572] INFO khoj: 🌖 Khoj is ready to use main.py:190
[00:49:35.674664] INFO uvicorn.error: Started server process [38712] server.py:75
[00:49:35.676655] INFO uvicorn.error: Waiting for application startup. on.py:45
[00:49:35.681656] INFO uvicorn.error: Application startup complete. on.py:59
[00:49:35.683654] INFO uvicorn.error: Uvicorn running on http://127.0.0.1:42110 (Press CTRL+C to quit) server.py:206
[00:50:19.377841] INFO uvicorn.access: 127.0.0.1:55643 - "GET /api/chat/options HTTP/1.1" 200 httptools_impl.py:437
[00:50:19.384851] INFO uvicorn.access: 127.0.0.1:55642 - "GET httptools_impl.py:437
/api/chat/starters?client=desktop HTTP/1.1" 200
[00:50:19.460802] INFO uvicorn.access: 127.0.0.1:55640 - "GET httptools_impl.py:437
/api/chat/history?client=desktop&n=10 HTTP/1.1" 200
[00:50:19.489799] INFO uvicorn.access: 127.0.0.1:55641 - "GET /api/chat/sessions HTTP/1.1" 200 httptools_impl.py:437
[00:50:19.519461] INFO uvicorn.access: 127.0.0.1:55638 - "GET /api/v1/user?client=desktop httptools_impl.py:437
HTTP/1.1" 200
[00:50:24.374164] INFO uvicorn.access: 127.0.0.1:55672 - "GET httptools_impl.py:437
/server/admin/database/openaiprocessorconversationconfig/?o=4 HTTP/1.1"
200
[00:51:27.172415] INFO uvicorn.access: 127.0.0.1:55806 - "GET /api/v1/user?client=desktop httptools_impl.py:437
HTTP/1.1" 200
[00:51:27.440925] INFO khoj.configure: 📡 Uploading telemetry to configure.py:353
https://khoj.beta.haletic.com/v1/telemetry...
[00:51:31.551088] INFO uvicorn.access: 127.0.0.1:55816 - "GET /api/v1/user?client=desktop httptools_impl.py:437
HTTP/1.1" 200
[00:51:31.570087] INFO uvicorn.access: 127.0.0.1:55820 - "GET /api/chat/options HTTP/1.1" 200 httptools_impl.py:437
[00:51:31.572089] INFO uvicorn.access: 127.0.0.1:55817 - "GET httptools_impl.py:437
/api/chat/history?client=desktop&n=10 HTTP/1.1" 200
[00:51:31.575089] INFO uvicorn.access: 127.0.0.1:55819 - "GET httptools_impl.py:437
/api/chat/starters?client=desktop HTTP/1.1" 200
[00:51:31.657301] INFO uvicorn.access: 127.0.0.1:55818 - "GET /api/chat/sessions HTTP/1.1" 200 httptools_impl.py:437
[00:51:35.745320] INFO khoj.routers.api_chat: Chat request by default: hi api_chat.py:812
[00:53:31.828086] INFO khoj.configure: 📡 Uploading telemetry to configure.py:353
https://khoj.beta.haletic.com/v1/telemetry...
You are using the default legacy behaviour of the <class 'transformers.models.llama.tokenization_llama_fast.LlamaTokenizerFast'>. This is expected, and simply means that the `legacy` (previous) behavior will be used so nothing changes for you. If you want to use the new behaviour, set `legacy=False`. This should only be set if you understand what it means, and thoroughly read the reason why this was added as explained in https://github.com/huggingface/transformers/pull/24565
[00:54:02.803417] WARNING khoj.processor.conversation.utils: Fallback to default chat model tokenizer: utils.py:217
None.
Configure tokenizer for unsupported model: llama3:8b in Khoj settings to improve
context stuffing.
[00:54:02.810008] INFO uvicorn.access: 127.0.0.1:55818 - "GET httptools_impl.py:437
/api/chat?q=hi&n=5&client=web&stream=true&conversation_id=3®ion=Cent
ral%20and%20Western&city=Central&country=Hong%20Kong&timezone=Asia/Hong
_Kong HTTP/1.1" 200
Exception in thread Thread-12 (llm_thread):
Traceback (most recent call last):
File "C:\Python\Python311\Lib\threading.py", line 1038, in _bootstrap_inner
self.run()
File "C:\Python\Python311\Lib\threading.py", line 975, in run
self._target(*self._args, **self._kwargs)
File "C:\Python\Python311\Lib\site-packages\khoj\processor\conversation\openai\utils.py", line 114, in llm_thread
chat = client.chat.completions.create(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Python\Python311\Lib\site-packages\openai\_utils\_utils.py", line 277, in wrapper
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "C:\Python\Python311\Lib\site-packages\openai\resources\chat\completions.py", line 606, in create
return self._post(
^^^^^^^^^^^
File "C:\Python\Python311\Lib\site-packages\openai\_base_client.py", line 1240, in post
return cast(ResponseT, self.request(cast_to, opts, stream=stream, stream_cls=stream_cls))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Python\Python311\Lib\site-packages\openai\_base_client.py", line 921, in request
return self._request(
^^^^^^^^^^^^^^
File "C:\Python\Python311\Lib\site-packages\openai\_base_client.py", line 1005, in _request
return self._retry_request(
^^^^^^^^^^^^^^^^^^^^
File "C:\Python\Python311\Lib\site-packages\openai\_base_client.py", line 1053, in _retry_request
return self._request(
^^^^^^^^^^^^^^
File "C:\Python\Python311\Lib\site-packages\openai\_base_client.py", line 1005, in _request
return self._retry_request(
^^^^^^^^^^^^^^^^^^^^
File "C:\Python\Python311\Lib\site-packages\openai\_base_client.py", line 1053, in _retry_request
return self._request(
^^^^^^^^^^^^^^
File "C:\Python\Python311\Lib\site-packages\openai\_base_client.py", line 1020, in _request
raise self._make_status_error_from_response(err.response) from None
openai.InternalServerError: Error code: 503
@saccohuo , could you verify that you're on the latest version of Khoj? You can run pip install khoj-assistant --upgrade
.
Can you also verify the chat model name you've configured exactly matches the ollama
model that you're running? Unfortunately the stack trace doesn't give all that much to go off of, except that indeed the error is happening when making the request to your ollama server.
@saccohuo , could you verify that you're on the latest version of Khoj? You can run
pip install khoj-assistant --upgrade
.@saccohuo,能否验证您是否在使用最新版本的Khoj?您可以运行pip install khoj-assistant --upgrade
。Can you also verify the chat model name you've configured exactly matches the
ollama
model that you're running? Unfortunately the stack trace doesn't give all that much to go off of, except that indeed the error is happening when making the request to your ollama server.您能否验证聊天模型名称是否与您正在运行的ollama
模型完全匹配?不幸的是,堆栈跟踪没有提供太多信息,除了确实错误是在向ollama服务器发出请求时发生的。
@sabaimran After upgrade, the same error still.
@saccohuo , do you mind sharing your chat model options configuration for the ollama model? you can just post a screenshot.