[FIX] The chat fails when querying with /notes while running LLAMA3 API

stevennt commented 4 months ago

Describe the bug

Problem: When querying notes: The chat interface is not responding. Backend gives out a lot of error messages.

The setup:

Self host (localhost)
LLAMA3 API

This works: /general How many articles are in Wikipedia?

This fails: How many articles are in Wikipedia?

When it fails: whenever it needs to query the notes. So specifically telling it not to (/general), then it works.

To Reproduce

Steps to reproduce the behavior:

Use LLAMA3 API
Upload a file
This chat will fail: How many articles are in Wikipedia?
This chat works: /general How many articles are in Wikipedia?

Screenshots

If applicable, add screenshots to help explain your problem.

Platform

Server:
- [ ] Cloud-Hosted (https://app.khoj.dev)
- [X] Self-Hosted Docker
- [ ] Self-Hosted Python package
- [ ] Self-Hosted source code
Client:
- [ ] Obsidian
- [ ] Emacs
- [ ] Desktop app
- [X] Web browser
- [ ] WhatsApp
OS:
- [ ] Windows
- [ ] macOS
- [X] Linux
- [ ] Android
- [ ] iOS

If self-hosted

Server Version [e.g. 1.0.1]: Docker from source, branch master at this commit: 3cfe5aabe5737bd9f9661a441784ff878eff4f23

Additional context

Add any other context about the problem here.

stevennt commented 4 months ago

When I add the OpenAi API (not the OpenAI-compatible LLAMA3 API), then it works properly.

So the issue appears when the /notes mode is used together with an OpenAi-compatible API.

sabaimran commented 4 months ago

Thanks for reporting @stevennt , I'll see if I can get a repro of the error on my end. Meanwhile, could you share any stack traces from your local server?

stevennt commented 4 months ago

Seems that it is something to do with the tokenizer: Configure tokenizer for unsupported model: llama3-70b-8192 in Khoj settings

myaiabnkhoj-server-1 myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 myaiabnkhoj-server-1 | myaiabnkhoj-server-1 myaiabnkhoj-server-1 | myaiabnkhoj-server-1 myaiabnkhoj-server-1 | myaiabnkhoj-server-1 myaiabnkhoj-server-1 | myaiabnkhoj-server-1 myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 myaiabnkhoj-server-1 myaiabnkhoj-server-1 | | [05:56:17.344098] DEBUG uvicorn.error: < TEXT 'How many protocol.py:1172 myaiabnkhoj-server-1 | items are in wikipedia' [31 bytes] myaiabnkhoj-server-1 | [05:56:17.348774] DEBUG uvicorn.error: > TEXT '{"type": protocol.py:1178 myaiabnkhoj-server-1 | "status", "message": myaiabnkhoj-server-1 | "**Understanding ...e": myaiabnkhoj-server-1 | "application/json"}' [125 bytes] myaiabnkhoj-server-1 | [05:56:17.355892] WARNING khoj.processor.conversation.utils: utils.py:217 myaiabnkhoj-server-1 | Fallback to default chat model myaiabnkhoj-server-1 | tokenizer: None. myaiabnkhoj-server-1 | Configure tokenizer for unsupported myaiabnkhoj-server-1 | model: llama3-70b-8192 in Khoj settings myaiabnkhoj-server-1 | to improve context stuffing. myaiabnkhoj-server-1 | [05:56:17.798986] DEBUG khoj.processor.conversation.openai before_sleep.py:65 myaiabnkhoj-server-1 | .utils: Retrying myaiabnkhoj-server-1 | khoj.processor.conversation.openai myaiabnkhoj-server-1 | .utils.completion_with_backoff in myaiabnkhoj-server-1 | 0.41305954873250217 seconds as it myaiabnkhoj-server-1 | raised BadRequestError: Error myaiabnkhoj-server-1 | code: 400 - {'error': {'message': myaiabnkhoj-server-1 | 'response_format does not support
streaming', 'type':
'invalid_request_error'}}.
| [05:56:18.498893] DEBUG khoj.processor.conversation.openai before_sleep.py:65 .utils: Retrying
khoj.processor.conversation.openai
.utils.completion_with_backoff in
0.6010106707150216 seconds as it
raised BadRequestError: Error
code: 400 - {'error': {'message':
'response_formatdoes not support myaiabnkhoj-server-1 | streaming', 'type': myaiabnkhoj-server-1 | 'invalid_request_error'}}. myaiabnkhoj-server-1 | [05:56:19.389462] DEBUG khoj.routers.helpers: Chat actor: helpers.py:175 myaiabnkhoj-server-1 | Infer information sources to refer: myaiabnkhoj-server-1 | 2.038 seconds myaiabnkhoj-server-1 | [05:56:19.395893] ERROR uvicorn.error: Exception in websockets_impl.py:188 myaiabnkhoj-server-1 | ASGI application myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | ╭─ Traceback (most recent ca─╮ myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │ myaiabnkhoj-server-1 | │ dist-packages/uvicorn/prot │ myaiabnkhoj-server-1 | │ ocols/websockets/websocket │ myaiabnkhoj-server-1 | │ s_impl.py:184 in run_asgi │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ 181 │ │ termination │ myaiabnkhoj-server-1 | │ 182 │ │ """ │ myaiabnkhoj-server-1 | │ 183 │ │ try: │ myaiabnkhoj-server-1 | │ ❱ 184 │ │ │ result = │ myaiabnkhoj-server-1 | │ 185 │ │ except BaseE │ myaiabnkhoj-server-1 | │ 186 │ │ │ self.clo │ myaiabnkhoj-server-1 | │ 187 │ │ │ msg = "E │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │ myaiabnkhoj-server-1 | │ dist-packages/uvicorn/midd │ myaiabnkhoj-server-1 | │ leware/proxy_headers.py:75 │ myaiabnkhoj-server-1 | │ in __call__ │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ 72 │ │ │ │ │ p │ myaiabnkhoj-server-1 | │ 73 │ │ │ │ │ s │ myaiabnkhoj-server-1 | │ 74 │ │ │ myaiabnkhoj-server-1 | │ ❱ 75 │ │ return await │ myaiabnkhoj-server-1 | │ 76 │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │ myaiabnkhoj-server-1 | │ dist-packages/fastapi/appl │ myaiabnkhoj-server-1 | │ ications.py:1054 in │ myaiabnkhoj-server-1 | │ __call__ │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ 1051 │ async def __cal │ myaiabnkhoj-server-1 | │ 1052 │ │ if self.roo │ myaiabnkhoj-server-1 | │ 1053 │ │ │ scope[" │ myaiabnkhoj-server-1 | │ ❱ 1054 │ │ await super │ myaiabnkhoj-server-1 | │ 1055 │ │ myaiabnkhoj-server-1 | │ 1056 │ def add_api_rou │ myaiabnkhoj-server-1 | │ 1057 │ │ self, │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │ myaiabnkhoj-server-1 | │ dist-packages/starlette/ap │ myaiabnkhoj-server-1 | │ plications.py:123 in │ myaiabnkhoj-server-1 | │ __call__ │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ 120 │ │ scope["app"] │ myaiabnkhoj-server-1 | │ 121 │ │ if self.midd │ myaiabnkhoj-server-1 | │ 122 │ │ │ self.mid │ myaiabnkhoj-server-1 | │ ❱ 123 │ │ await self.m │ myaiabnkhoj-server-1 | │ 124 │ │ myaiabnkhoj-server-1 | │ 125 │ def on_event(sel │ myaiabnkhoj-server-1 | │ 126 │ │ return self. │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │ myaiabnkhoj-server-1 | │ dist-packages/starlette/mi │ myaiabnkhoj-server-1 | │ ddleware/errors.py:151 in │ myaiabnkhoj-server-1 | │ __call__ │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ 148 │ │ myaiabnkhoj-server-1 | │ 149 │ async def __call │ myaiabnkhoj-server-1 | │ 150 │ │ if scope["ty │ myaiabnkhoj-server-1 | │ ❱ 151 │ │ │ await se │ myaiabnkhoj-server-1 | │ 152 │ │ │ return │ myaiabnkhoj-server-1 | │ 153 │ │ │ myaiabnkhoj-server-1 | │ 154 │ │ response_sta │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │ myaiabnkhoj-server-1 | │ dist-packages/starlette/mi │ myaiabnkhoj-server-1 | │ ddleware/sessions.py:85 in │ myaiabnkhoj-server-1 | │ __call__ │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ 82 │ │ │ │ │ h │ myaiabnkhoj-server-1 | │ 83 │ │ │ await sen │ myaiabnkhoj-server-1 | │ 84 │ │ │ myaiabnkhoj-server-1 | │ ❱ 85 │ │ await self.ap │ myaiabnkhoj-server-1 | │ 86 │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │ myaiabnkhoj-server-1 | │ dist-packages/starlette/mi │ myaiabnkhoj-server-1 | │ ddleware/authentication.py │ myaiabnkhoj-server-1 | │ :49 in __call__ │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ 46 │ │ if auth_resul │ myaiabnkhoj-server-1 | │ 47 │ │ │ auth_resu │ myaiabnkhoj-server-1 | │ 48 │ │ scope["auth"] │ myaiabnkhoj-server-1 | │ ❱ 49 │ │ await self.ap │ myaiabnkhoj-server-1 | │ 50 │ │ myaiabnkhoj-server-1 | │ 51 │ @staticmethod │ myaiabnkhoj-server-1 | │ 52 │ def default_on_er │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │ myaiabnkhoj-server-1 | │ dist-packages/starlette/mi │ myaiabnkhoj-server-1 | │ ddleware/cors.py:77 in │ myaiabnkhoj-server-1 | │ __call__ │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ 74 │ │ myaiabnkhoj-server-1 | │ 75 │ async def __call │ myaiabnkhoj-server-1 | │ 76 │ │ if scope["ty │ myaiabnkhoj-server-1 | │ ❱ 77 │ │ │ await se │ myaiabnkhoj-server-1 | │ 78 │ │ │ return │ myaiabnkhoj-server-1 | │ 79 │ │ │ myaiabnkhoj-server-1 | │ 80 │ │ method = sco │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │ myaiabnkhoj-server-1 | │ dist-packages/starlette/mi │ myaiabnkhoj-server-1 | │ ddleware/exceptions.py:65 │ myaiabnkhoj-server-1 | │ in __call__ │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ 62 │ │ else: │ myaiabnkhoj-server-1 | │ 63 │ │ │ conn = We │ myaiabnkhoj-server-1 | │ 64 │ │ │ myaiabnkhoj-server-1 | │ ❱ 65 │ │ await wrap_ap │ myaiabnkhoj-server-1 | │ 66 │ │ myaiabnkhoj-server-1 | │ 67 │ def http_exceptio │ myaiabnkhoj-server-1 | │ 68 │ │ assert isinst │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │ myaiabnkhoj-server-1 | │ dist-packages/starlette/_e │ myaiabnkhoj-server-1 | │ xception_handler.py:64 in │ myaiabnkhoj-server-1 | │ wrapped_app │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ 61 │ │ │ │ handl │ myaiabnkhoj-server-1 | │ 62 │ │ │ │ myaiabnkhoj-server-1 | │ 63 │ │ │ if handle │ myaiabnkhoj-server-1 | │ ❱ 64 │ │ │ │ raise │ myaiabnkhoj-server-1 | │ 65 │ │ │ │ myaiabnkhoj-server-1 | │ 66 │ │ │ if respon │ myaiabnkhoj-server-1 | │ 67 │ │ │ │ msg = │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │ myaiabnkhoj-server-1 | │ dist-packages/starlette/_e │ myaiabnkhoj-server-1 | │ xception_handler.py:53 in │ myaiabnkhoj-server-1 | │ wrapped_app │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ 50 │ │ │ await sen │ myaiabnkhoj-server-1 | │ 51 │ │ │ myaiabnkhoj-server-1 | │ 52 │ │ try: │ myaiabnkhoj-server-1 | │ ❱ 53 │ │ │ await app │ myaiabnkhoj-server-1 | │ 54 │ │ except Except │ myaiabnkhoj-server-1 | │ 55 │ │ │ handler = │ myaiabnkhoj-server-1 | │ 56 │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │ myaiabnkhoj-server-1 | │ dist-packages/starlette/ro │ myaiabnkhoj-server-1 | │ uting.py:756 in __call__ │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ 753 │ │ """ │ myaiabnkhoj-server-1 | │ 754 │ │ The main ent │ myaiabnkhoj-server-1 | │ 755 │ │ """ │ myaiabnkhoj-server-1 | │ ❱ 756 │ │ await self.m │ myaiabnkhoj-server-1 | │ 757 │ │ myaiabnkhoj-server-1 | │ 758 │ async def app(se │ myaiabnkhoj-server-1 | │ 759 │ │ assert scope │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │ myaiabnkhoj-server-1 | │ dist-packages/starlette/ro │ myaiabnkhoj-server-1 | │ uting.py:776 in app │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ 773 │ │ │ match, c │ myaiabnkhoj-server-1 | │ 774 │ │ │ if match │ myaiabnkhoj-server-1 | │ 775 │ │ │ │ scop │ myaiabnkhoj-server-1 | │ ❱ 776 │ │ │ │ awai │ myaiabnkhoj-server-1 | │ 777 │ │ │ │ retu │ myaiabnkhoj-server-1 | │ 778 │ │ │ elif mat │ myaiabnkhoj-server-1 | │ 779 │ │ │ │ part │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │ myaiabnkhoj-server-1 | │ dist-packages/starlette/ro │ myaiabnkhoj-server-1 | │ uting.py:373 in handle │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ 370 │ │ return URLPa │ myaiabnkhoj-server-1 | │ 371 │ │ myaiabnkhoj-server-1 | │ 372 │ async def handle │ myaiabnkhoj-server-1 | │ ❱ 373 │ │ await self.a │ myaiabnkhoj-server-1 | │ 374 │ │ myaiabnkhoj-server-1 | │ 375 │ def __eq__(self, │ myaiabnkhoj-server-1 | │ 376 │ │ return ( │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │ myaiabnkhoj-server-1 | │ dist-packages/starlette/ro │ myaiabnkhoj-server-1 | │ uting.py:96 in app │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ 93 │ │ async def ap │ myaiabnkhoj-server-1 | │ 94 │ │ │ await fu │ myaiabnkhoj-server-1 | │ 95 │ │ │ myaiabnkhoj-server-1 | │ ❱ 96 │ │ await wrap_a │ myaiabnkhoj-server-1 | │ 97 │ │ myaiabnkhoj-server-1 | │ 98 │ return app │ myaiabnkhoj-server-1 | │ 99 │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │ myaiabnkhoj-server-1 | │ dist-packages/starlette/_e │ myaiabnkhoj-server-1 | │ xception_handler.py:64 in │ myaiabnkhoj-server-1 | │ wrapped_app │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ 61 │ │ │ │ handl │ myaiabnkhoj-server-1 | │ 62 │ │ │ │ myaiabnkhoj-server-1 | │ 63 │ │ │ if handle │ myaiabnkhoj-server-1 | │ ❱ 64 │ │ │ │ raise │ myaiabnkhoj-server-1 | │ 65 │ │ │ │ myaiabnkhoj-server-1 | │ 66 │ │ │ if respon │ myaiabnkhoj-server-1 | │ 67 │ │ │ │ msg = │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │ myaiabnkhoj-server-1 | │ dist-packages/starlette/_e │ myaiabnkhoj-server-1 | │ xception_handler.py:53 in │ myaiabnkhoj-server-1 | │ wrapped_app │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ 50 │ │ │ await sen │ myaiabnkhoj-server-1 | │ 51 │ │ │ myaiabnkhoj-server-1 | │ 52 │ │ try: │ myaiabnkhoj-server-1 | │ ❱ 53 │ │ │ await app │ myaiabnkhoj-server-1 | │ 54 │ │ except Except │ myaiabnkhoj-server-1 | │ 55 │ │ │ handler = │ myaiabnkhoj-server-1 | │ 56 │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │ myaiabnkhoj-server-1 | │ dist-packages/starlette/ro │ myaiabnkhoj-server-1 | │ uting.py:94 in app │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ 91 │ │ session = We │ myaiabnkhoj-server-1 | │ 92 │ │ │ myaiabnkhoj-server-1 | │ 93 │ │ async def ap │ myaiabnkhoj-server-1 | │ ❱ 94 │ │ │ await fu │ myaiabnkhoj-server-1 | │ 95 │ │ │ myaiabnkhoj-server-1 | │ 96 │ │ await wrap_a │ myaiabnkhoj-server-1 | │ 97 │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │ myaiabnkhoj-server-1 | │ dist-packages/fastapi/rout │ myaiabnkhoj-server-1 | │ ing.py:348 in app │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ 345 │ │ │ if erro │ myaiabnkhoj-server-1 | │ 346 │ │ │ │ rai │ myaiabnkhoj-server-1 | │ 347 │ │ │ assert │ myaiabnkhoj-server-1 | │ ❱ 348 │ │ │ await d │ myaiabnkhoj-server-1 | │ 349 │ │ myaiabnkhoj-server-1 | │ 350 │ return app │ myaiabnkhoj-server-1 | │ 351 │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ /app/src/khoj/routers/api_ │ myaiabnkhoj-server-1 | │ chat.py:578 in │ myaiabnkhoj-server-1 | │ websocket_endpoint │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ 575 │ │ is_automate │ myaiabnkhoj-server-1 | │ 576 │ │ │ myaiabnkhoj-server-1 | │ 577 │ │ if conversa │ myaiabnkhoj-server-1 | │ ❱ 578 │ │ │ convers │ myaiabnkhoj-server-1 | │ is_automated_task) │ myaiabnkhoj-server-1 | │ 579 │ │ │ convers │ myaiabnkhoj-server-1 | │ conversation_comman │ myaiabnkhoj-server-1 | │ 580 │ │ │ await s │ myaiabnkhoj-server-1 | │ {conversation_comma │ myaiabnkhoj-server-1 | │ 581 │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ /app/src/khoj/routers/help │ myaiabnkhoj-server-1 | │ ers.py:250 in │ myaiabnkhoj-server-1 | │ aget_relevant_information_ │ myaiabnkhoj-server-1 | │ sources │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ 247 │ ) │ myaiabnkhoj-server-1 | │ 248 │ │ myaiabnkhoj-server-1 | │ 249 │ with timer("Cha │ myaiabnkhoj-server-1 | │ ❱ 250 │ │ response = │ myaiabnkhoj-server-1 | │ response_type="json │ myaiabnkhoj-server-1 | │ 251 │ │ myaiabnkhoj-server-1 | │ 252 │ try: │ myaiabnkhoj-server-1 | │ 253 │ │ response = │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ /app/src/khoj/routers/help │ myaiabnkhoj-server-1 | │ ers.py:547 in │ myaiabnkhoj-server-1 | │ send_message_to_model_wrap │ myaiabnkhoj-server-1 | │ per │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ 544 │ │ │ tokeniz │ myaiabnkhoj-server-1 | │ 545 │ │ ) │ myaiabnkhoj-server-1 | │ 546 │ │ │ myaiabnkhoj-server-1 | │ ❱ 547 │ │ openai_resp │ myaiabnkhoj-server-1 | │ 548 │ │ │ message │ myaiabnkhoj-server-1 | │ 549 │ │ │ api_key │ myaiabnkhoj-server-1 | │ 550 │ │ │ model=c │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ /app/src/khoj/processor/co │ myaiabnkhoj-server-1 | │ nversation/openai/gpt.py:9 │ myaiabnkhoj-server-1 | │ 8 in send_message_to_model │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ 95 │ """ │ myaiabnkhoj-server-1 | │ 96 │ │ myaiabnkhoj-server-1 | │ 97 │ # Get Response f │ myaiabnkhoj-server-1 | │ ❱ 98 │ return completio │ myaiabnkhoj-server-1 | │ 99 │ │ messages=mes │ myaiabnkhoj-server-1 | │ 100 │ │ model=model, │ myaiabnkhoj-server-1 | │ 101 │ │ openai_api_k │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │ myaiabnkhoj-server-1 | │ dist-packages/tenacity/__i │ myaiabnkhoj-server-1 | │ nit__.py:330 in wrapped_f │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ 327 │ │ │ f, funct │ myaiabnkhoj-server-1 | │ 328 │ │ ) │ myaiabnkhoj-server-1 | │ 329 │ │ def wrapped_ │ myaiabnkhoj-server-1 | │ ❱ 330 │ │ │ return s │ myaiabnkhoj-server-1 | │ 331 │ │ │ myaiabnkhoj-server-1 | │ 332 │ │ def retry_wi │ myaiabnkhoj-server-1 | │ 333 │ │ │ return s │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │ myaiabnkhoj-server-1 | │ dist-packages/tenacity/__i │ myaiabnkhoj-server-1 | │ nit__.py:467 in __call__ │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ 464 │ │ │ myaiabnkhoj-server-1 | │ 465 │ │ retry_state │ myaiabnkhoj-server-1 | │ 466 │ │ while True: │ myaiabnkhoj-server-1 | │ ❱ 467 │ │ │ do = sel │ myaiabnkhoj-server-1 | │ 468 │ │ │ if isins │ myaiabnkhoj-server-1 | │ 469 │ │ │ │ try: │ myaiabnkhoj-server-1 | │ 470 │ │ │ │ │ │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │ myaiabnkhoj-server-1 | │ dist-packages/tenacity/__i │ myaiabnkhoj-server-1 | │ nit__.py:368 in iter │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ 365 │ │ self._begin_ │ myaiabnkhoj-server-1 | │ 366 │ │ result = Non │ myaiabnkhoj-server-1 | │ 367 │ │ for action i │ myaiabnkhoj-server-1 | │ ❱ 368 │ │ │ result = │ myaiabnkhoj-server-1 | │ 369 │ │ return resul │ myaiabnkhoj-server-1 | │ 370 │ │ myaiabnkhoj-server-1 | │ 371 │ def _begin_iter( │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │ myaiabnkhoj-server-1 | │ dist-packages/tenacity/__i │ myaiabnkhoj-server-1 | │ nit__.py:410 in exc_check │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ 407 │ │ │ │ fut │ myaiabnkhoj-server-1 | │ 408 │ │ │ │ retr │ myaiabnkhoj-server-1 | │ 409 │ │ │ │ if s │ myaiabnkhoj-server-1 | │ ❱ 410 │ │ │ │ │ │ myaiabnkhoj-server-1 | │ 411 │ │ │ │ rais │ myaiabnkhoj-server-1 | │ 412 │ │ │ │ myaiabnkhoj-server-1 | │ 413 │ │ │ self._ad │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │ myaiabnkhoj-server-1 | │ dist-packages/tenacity/__i │ myaiabnkhoj-server-1 | │ nit__.py:183 in reraise │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ 180 │ │ myaiabnkhoj-server-1 | │ 181 │ def reraise(self │ myaiabnkhoj-server-1 | │ 182 │ │ if self.last │ myaiabnkhoj-server-1 | │ ❱ 183 │ │ │ raise se │ myaiabnkhoj-server-1 | │ 184 │ │ raise self │ myaiabnkhoj-server-1 | │ 185 │ │ myaiabnkhoj-server-1 | │ 186 │ def __str__(self │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ /usr/lib/python3.10/concur │ myaiabnkhoj-server-1 | │ rent/futures/_base.py:451 │ myaiabnkhoj-server-1 | │ in result │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ 448 │ │ │ │ if s │ myaiabnkhoj-server-1 | │ 449 │ │ │ │ │ │ myaiabnkhoj-server-1 | │ 450 │ │ │ │ elif │ myaiabnkhoj-server-1 | │ ❱ 451 │ │ │ │ │ │ myaiabnkhoj-server-1 | │ 452 │ │ │ │ │ myaiabnkhoj-server-1 | │ 453 │ │ │ │ self │ myaiabnkhoj-server-1 | │ 454 │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ /usr/lib/python3.10/concur │ myaiabnkhoj-server-1 | │ rent/futures/_base.py:403 │ myaiabnkhoj-server-1 | │ in __get_result │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ 400 │ def __get_result │ myaiabnkhoj-server-1 | │ 401 │ │ if self._exc │ myaiabnkhoj-server-1 | │ 402 │ │ │ try: │ myaiabnkhoj-server-1 | │ ❱ 403 │ │ │ │ rais │ myaiabnkhoj-server-1 | │ 404 │ │ │ finally: │ myaiabnkhoj-server-1 | │ 405 │ │ │ │ # Br │ myaiabnkhoj-server-1 | │ 406 │ │ │ │ self │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │ myaiabnkhoj-server-1 | │ dist-packages/tenacity/__i │ myaiabnkhoj-server-1 | │ nit__.py:470 in __call__ │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ 467 │ │ │ do = sel │ myaiabnkhoj-server-1 | │ 468 │ │ │ if isins │ myaiabnkhoj-server-1 | │ 469 │ │ │ │ try: │ myaiabnkhoj-server-1 | │ ❱ 470 │ │ │ │ │ │ myaiabnkhoj-server-1 | │ 471 │ │ │ │ exce │ myaiabnkhoj-server-1 | │ 472 │ │ │ │ │ │ myaiabnkhoj-server-1 | │ 473 │ │ │ │ else │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ /app/src/khoj/processor/co │ myaiabnkhoj-server-1 | │ nversation/openai/utils.py │ myaiabnkhoj-server-1 | │ :50 in │ myaiabnkhoj-server-1 | │ completion_with_backoff │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ 47 │ │ myaiabnkhoj-server-1 | │ 48 │ formatted_messag │ myaiabnkhoj-server-1 | │ in messages] │ myaiabnkhoj-server-1 | │ 49 │ │ myaiabnkhoj-server-1 | │ ❱ 50 │ chat = client.ch │ myaiabnkhoj-server-1 | │ 51 │ │ stream=True, │ myaiabnkhoj-server-1 | │ 52 │ │ messages=for │ myaiabnkhoj-server-1 | │ 53 │ │ model=model, │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │ myaiabnkhoj-server-1 | │ dist-packages/openai/_util │ myaiabnkhoj-server-1 | │ s/_utils.py:277 in wrapper │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ 274 │ │ │ │ │ │ myaiabnkhoj-server-1 | │ 275 │ │ │ │ │ │ myaiabnkhoj-server-1 | │ 276 │ │ │ │ rais │ myaiabnkhoj-server-1 | │ ❱ 277 │ │ │ return f │ myaiabnkhoj-server-1 | │ 278 │ │ │ myaiabnkhoj-server-1 | │ 279 │ │ return wrapp │ myaiabnkhoj-server-1 | │ 280 │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │ myaiabnkhoj-server-1 | │ dist-packages/openai/resou │ myaiabnkhoj-server-1 | │ rces/chat/completions.py:6 │ myaiabnkhoj-server-1 | │ 40 in create │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ 637 │ │ extra_body: │ myaiabnkhoj-server-1 | │ 638 │ │ timeout: fl │ myaiabnkhoj-server-1 | │ 639 │ ) -> ChatComple │ myaiabnkhoj-server-1 | │ ❱ 640 │ │ return self │ myaiabnkhoj-server-1 | │ 641 │ │ │ "/chat/ │ myaiabnkhoj-server-1 | │ 642 │ │ │ body=ma │ myaiabnkhoj-server-1 | │ 643 │ │ │ │ { │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │ myaiabnkhoj-server-1 | │ dist-packages/openai/_base │ myaiabnkhoj-server-1 | │ _client.py:1250 in post │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ 1247 │ │ opts = Fina │ myaiabnkhoj-server-1 | │ 1248 │ │ │ method= │ myaiabnkhoj-server-1 | │ **options │ myaiabnkhoj-server-1 | │ 1249 │ │ ) │ myaiabnkhoj-server-1 | │ ❱ 1250 │ │ return cast │ myaiabnkhoj-server-1 | │ stream_cls=stream_c │ myaiabnkhoj-server-1 | │ 1251 │ │ myaiabnkhoj-server-1 | │ 1252 │ def patch( │ myaiabnkhoj-server-1 | │ 1253 │ │ self, │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │ myaiabnkhoj-server-1 | │ dist-packages/openai/_base │ myaiabnkhoj-server-1 | │ _client.py:931 in request │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ 928 │ │ stream: boo │ myaiabnkhoj-server-1 | │ 929 │ │ stream_cls: │ myaiabnkhoj-server-1 | │ 930 │ ) -> ResponseT │ myaiabnkhoj-server-1 | │ ❱ 931 │ │ return self │ myaiabnkhoj-server-1 | │ 932 │ │ │ cast_to │ myaiabnkhoj-server-1 | │ 933 │ │ │ options │ myaiabnkhoj-server-1 | │ 934 │ │ │ stream= │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │ myaiabnkhoj-server-1 | │ dist-packages/openai/_base │ myaiabnkhoj-server-1 | │ _client.py:1030 in │ myaiabnkhoj-server-1 | │ _request │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ 1027 │ │ │ │ err │ myaiabnkhoj-server-1 | │ 1028 │ │ │ │ myaiabnkhoj-server-1 | │ 1029 │ │ │ log.deb │ myaiabnkhoj-server-1 | │ ❱ 1030 │ │ │ raise s │ myaiabnkhoj-server-1 | │ 1031 │ │ │ myaiabnkhoj-server-1 | │ 1032 │ │ return self │ myaiabnkhoj-server-1 | │ 1033 │ │ │ cast_to │ myaiabnkhoj-server-1 | ╰────────────────────────────╯ myaiabnkhoj-server-1 | BadRequestError: Error code: myaiabnkhoj-server-1 | 400 - {'error': {'message': myaiabnkhoj-server-1 | 'response_format does not
support streaming', 'type':
'invalid_request_error'}}
| [05:56:20.452685] DEBUG uvicorn.error: = connection is protocol.py:1227 CLOSING
| [05:56:20.453936] DEBUG uvicorn.error: > CLOSE 1000 (OK) [2 protocol.py:1178 bytes]
| [05:56:20.455200] DEBUG uvicorn.error: = connection is protocol.py:1497 CLOSED
| [05:56:20.457202] DEBUG uvicorn.error: ! failing connection protocol.py:1412 with code 1006
| [05:56:20.459085] ERROR uvicorn.error: closing handshake failed server.py:248 ╭─ Traceback (most recent call last) ─╮
│ /usr/local/lib/python3.10/dist-pack │
│ ages/websockets/legacy/server.py:24 │
│ 4 in handler │
│ │
│ 241 │ │ │ │ raise │
│ 242 │ │ │ │
│ 243 │ │ │ try: │
│ ❱ 244 │ │ │ │ await self.c │
│ 245 │ │ │ except Connectio │
│ 246 │ │ │ │ raise │
│ 247 │ │ │ except Exception │
│ │
│ /usr/local/lib/python3.10/dist-pack │
│ ages/websockets/legacy/protocol.py: │
│ 770 in close │
│ │
│ 767 │ │ """ │
│ 768 │ │ try: │
│ 769 │ │ │ async with async │
│ ❱ 770 │ │ │ │ await self.w │
│ 771 │ │ except asyncio.Timeo │
│ 772 │ │ │ # If the close f │
│ 773 │ │ │ # are full, the │
│ │
│ /usr/local/lib/python3.10/dist-pack │
│ ages/websockets/legacy/protocol.py: │
│ 1236 in write_close_frame │
│ │
│ 1233 │ │ │ │ data = close │
│ 1234 │ │ │ │
│ 1235 │ │ │ # 7.1.2. Start t │
│ ❱ 1236 │ │ │ await self.write │
│ 1237 │ │
│ 1238 │ async def keepalive_ping │
│ 1239 │ │ """ │
│ │
│ /usr/local/lib/python3.10/dist-pack │
│ ages/websockets/legacy/protocol.py: │
│ 1209 in write_frame │
│ │
│ 1206 │ │ │ │ f"Cannot wri │
│ 1207 │ │ │ ) │
│ 1208 │ │ self.write_frame_syn │
│ ❱ 1209 │ │ await self.drain() │
│ 1210 │ │
│ 1211 │ async def write_close_fr │
│ 1212 │ │ self, close: Close, │
│ │
│ /usr/local/lib/python3.10/dist-pack │
│ ages/websockets/legacy/protocol.py: │
│ 1198 in drain │
│ │
│ 1195 │ │ │ self.fail_connec │
│ 1196 │ │ │ # Wait until the │
│ 1197 │ │ │ # with the corre │
│ ❱ 1198 │ │ │ await self.ensur │
│ 1199 │ │
│ 1200 │ async def write_frame( │
│ 1201 │ │ self, fin: bool, opc │
│ │
│ /usr/local/lib/python3.10/dist-pack │
│ ages/websockets/legacy/protocol.py: │
│ 939 in ensure_open │
│ │
│ 936 │ │ │ │ return │
│ 937 │ │ │
│ 938 │ │ if self.state is Sta │
│ ❱ 939 │ │ │ raise self.conne │
│ 940 │ │ │
│ 941 │ │ if self.state is Sta │
│ 942 │ │ │ # If we started │
╰─────────────────────────────────────╯
ConnectionClosedError: sent 1000 (OK);
no close frame received
| [05:56:20.763183] INFO uvicorn.error: connection closed server.py:264 | [05:56:20.764402] DEBUG uvicorn.error: x half-closing TCP protocol.py:1319 connection

`

sabaimran commented 4 months ago

Awesome, this is really helpful, thanks for including it. Will see if I can get it resolved today.

sabaimran commented 4 months ago

Thanks for the helpful report @stevennt ! I've pushed a fix. It'll be included in the next release (~1 week). If you want the fix sooner and you're using the docker image, you can build from the pre tag when the image finishes building (~20 minutes from now).

stevennt commented 4 months ago

I just pulled the master branch which has the below commit fix. The issue still persists. @sabaimran

https://github.com/khoj-ai/khoj/commit/d3c0111121ec3d2d68ea0181a3a93ecd2641e676

sabaimran commented 4 months ago

How are you running the project? Straight from source? Or from the docker image?

You're seeing the exact same stack trace?

stevennt commented 4 months ago

What I did was to pull the source, run docker-compose build, docker-compose down -v, and docker-compose up. Then I setup the configs at server admin. First: then:

Maybe something needs to be set at this tokenizer field?

Below is the stack trace:

docker-compose up [+] Running 2/0 ⠿ Container myaiabnkhoj-database-1 Created 0.0s ⠿ Container myaiabnkhoj-server-1 Created 0.0s Attaching to myaiabnkhoj-database-1, myaiabnkhoj-server-1 myaiabnkhoj-database-1 | myaiabnkhoj-database-1 | PostgreSQL Database directory appears to contain a database; Skipping initialization myaiabnkhoj-database-1 | myaiabnkhoj-database-1 | 2024-06-21 07:58:14.007 UTC [1] LOG: starting PostgreSQL 15.4 (Debian 15.4-2.pgdg120+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 12.2.0-14) 12.2.0, 64-bit myaiabnkhoj-database-1 | 2024-06-21 07:58:14.007 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432 myaiabnkhoj-database-1 | 2024-06-21 07:58:14.007 UTC [1] LOG: listening on IPv6 address "::", port 5432 myaiabnkhoj-database-1 | 2024-06-21 07:58:14.173 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432" myaiabnkhoj-database-1 | 2024-06-21 07:58:14.367 UTC [28] LOG: database system was shut down at 2024-06-21 07:58:05 UTC myaiabnkhoj-database-1 | 2024-06-21 07:58:14.487 UTC [1] LOG: database system is ready to accept connections myaiabnkhoj-server-1 | [07:58:49.489205] WARNING khoj.routers.email: RESEND_API_KEY not email.py:26 myaiabnkhoj-server-1 | set - email sending disabled myaiabnkhoj-server-1 | [07:58:49.529753] INFO khoj: 🚒 Initializing ABNCopilot v0.0.0 main.py:124 myaiabnkhoj-server-1 | [07:58:49.531214] INFO khoj: 📦 Initializing DB: main.py:125 myaiabnkhoj-server-1 | Operations to perform: myaiabnkhoj-server-1 | Apply all migrations: admin, auth, myaiabnkhoj-server-1 | contenttypes, database, myaiabnkhoj-server-1 | django_apscheduler, sessions myaiabnkhoj-server-1 | Running migrations: myaiabnkhoj-server-1 | No migrations to apply. myaiabnkhoj-server-1 | [07:58:49.533447] DEBUG khoj: 🌍 Initializing Web Client: main.py:126 myaiabnkhoj-server-1 | 0 static files copied to myaiabnkhoj-server-1 | '/app/src/khoj/static', 201 unmodified. myaiabnkhoj-server-1 | [07:58:49.539414] INFO khoj: 🌘 Starting Khoj main.py:138 myaiabnkhoj-server-1 | [07:58:49.829809] INFO khoj.configure: 🚨 Khoj is not configure.py:198 myaiabnkhoj-server-1 | configured. myaiabnkhoj-server-1 | Initializing it with a default myaiabnkhoj-server-1 | config. myaiabnkhoj-server-1 | /usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py:1132: FutureWarning:resume_downloadis deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, useforce_download=True`. myaiabnkhoj-server-1 | warnings.warn(

myaiabnkhoj-server-1 | [08:00:03.609363] INFO khoj.database.adapters: No default init.py:558 myaiabnkhoj-server-1 | conversation config found, skipping
myaiabnkhoj-server-1 | default agent creation
myaiabnkhoj-server-1 | [08:00:03.610718] INFO khoj.configure: 📡 Telemetry enabled configure.py:240 myaiabnkhoj-server-1 | [08:00:03.611839] INFO khoj: 🌖 ABN Copilot is ready to use main.py:190 myaiabnkhoj-server-1 | [08:00:03.624946] INFO uvicorn.error: Started server process server.py:75 myaiabnkhoj-server-1 | [1]
myaiabnkhoj-server-1 | [08:00:03.626195] INFO uvicorn.error: Waiting for application on.py:45 myaiabnkhoj-server-1 | startup.
myaiabnkhoj-server-1 | [08:00:03.627583] INFO uvicorn.error: Application startup complete. on.py:59 myaiabnkhoj-server-1 | [08:00:03.629082] INFO uvicorn.error: Uvicorn running on server.py:206 myaiabnkhoj-server-1 | http://0.0.0.0:42110 (Press CTRL+C to
myaiabnkhoj-server-1 | quit)

myaiabnkhoj-server-1 | [08:04:47.153646] DEBUG uvicorn.error: < TEXT 'What are the protocol.py:1172 myaiabnkhoj-server-1 | number of articles in Wikipedia?'
myaiabnkhoj-server-1 | [45 bytes]
myaiabnkhoj-server-1 | [08:04:47.160158] DEBUG uvicorn.error: > TEXT '{"type": protocol.py:1178 myaiabnkhoj-server-1 | "status", "message":
myaiabnkhoj-server-1 | "**Understanding ...e":
myaiabnkhoj-server-1 | "application/json"}' [139 bytes]
myaiabnkhoj-server-1 | [08:04:49.544382] INFO khoj.configure: 📡 Uploading configure.py:353 myaiabnkhoj-server-1 | telemetry to
myaiabnkhoj-server-1 | https://khoj.beta.haletic.com/v1/tel
myaiabnkhoj-server-1 | emetry...
myaiabnkhoj-server-1 | [08:04:49.550244] DEBUG khoj.configure: Telemetry state: configure.py:354 myaiabnkhoj-server-1 | [{'telemetry_type': 'api',
myaiabnkhoj-server-1 | 'server_version': '0.0.0', 'os':
myaiabnkhoj-server-1 | 'Linux', 'timestamp': '2024-06-21
myaiabnkhoj-server-1 | 08:03:59', 'client_host':
myaiabnkhoj-server-1 | '172.26.0.1', 'user_agent':
myaiabnkhoj-server-1 | 'unknown', 'referer': 'unknown',
myaiabnkhoj-server-1 | 'host': 'unknown', 'server_id':
myaiabnkhoj-server-1 | '26485c08-2711-4a10-9a5f-b96e41e1e2a
myaiabnkhoj-server-1 | d', 'subscription_type': 'standard',
myaiabnkhoj-server-1 | 'is_recurring': False, 'client_id':
myaiabnkhoj-server-1 | 'default', 'api':
myaiabnkhoj-server-1 | 'get_all_filenames'},
myaiabnkhoj-server-1 | {'telemetry_type': 'api',
myaiabnkhoj-server-1 | 'server_version': '0.0.0', 'os':
myaiabnkhoj-server-1 | 'Linux', 'timestamp': '2024-06-21
myaiabnkhoj-server-1 | 08:03:59', 'client_host':
myaiabnkhoj-server-1 | '172.26.0.1', 'user_agent':
myaiabnkhoj-server-1 | 'Mozilla/5.0 (X11; Linux x86_64)
myaiabnkhoj-server-1 | AppleWebKit/537.36 (KHTML, like
myaiabnkhoj-server-1 | Gecko) Chrome/123.0.0.0
myaiabnkhoj-server-1 | Safari/537.36', 'referer':
myaiabnkhoj-server-1 | 'http://localhost:42110/', 'host':
myaiabnkhoj-server-1 | 'localhost:42110', 'server_id':
myaiabnkhoj-server-1 | '26485c08-2711-4a10-9a5f-b96e41e1e2a
myaiabnkhoj-server-1 | d', 'subscription_type': 'standard',
myaiabnkhoj-server-1 | 'is_recurring': False, 'client_id':
myaiabnkhoj-server-1 | 'default', 'api': 'chat_options'},
myaiabnkhoj-server-1 | {'telemetry_type': 'api',
myaiabnkhoj-server-1 | 'server_version': '0.0.0', 'os':
myaiabnkhoj-server-1 | 'Linux', 'timestamp': '2024-06-21
myaiabnkhoj-server-1 | 08:03:59', 'client_host':
myaiabnkhoj-server-1 | '172.26.0.1', 'user_agent':
myaiabnkhoj-server-1 | 'Mozilla/5.0 (X11; Linux x86_64)
myaiabnkhoj-server-1 | AppleWebKit/537.36 (KHTML, like
myaiabnkhoj-server-1 | Gecko) Chrome/123.0.0.0
myaiabnkhoj-server-1 | Safari/537.36', 'referer':
myaiabnkhoj-server-1 | 'http://localhost:42110/', 'host':
myaiabnkhoj-server-1 | 'localhost:42110', 'server_id':
myaiabnkhoj-server-1 | '26485c08-2711-4a10-9a5f-b96e41e1e2a
myaiabnkhoj-server-1 | d', 'subscription_type': 'standard',
myaiabnkhoj-server-1 | 'is_recurring': False, 'client_id':
myaiabnkhoj-server-1 | 'default', 'api': 'chat_sessions'},
myaiabnkhoj-server-1 | {'telemetry_type': 'api',
myaiabnkhoj-server-1 | 'server_version': '0.0.0', 'os':
myaiabnkhoj-server-1 | 'Linux', 'timestamp': '2024-06-21
myaiabnkhoj-server-1 | 08:03:59', 'client_host':
myaiabnkhoj-server-1 | '172.26.0.1', 'user_agent':
myaiabnkhoj-server-1 | 'Mozilla/5.0 (X11; Linux x86_64)
myaiabnkhoj-server-1 | AppleWebKit/537.36 (KHTML, like
myaiabnkhoj-server-1 | Gecko) Chrome/123.0.0.0
myaiabnkhoj-server-1 | Safari/537.36', 'referer':
myaiabnkhoj-server-1 | 'http://localhost:42110/', 'host':
myaiabnkhoj-server-1 | 'localhost:42110', 'server_id':
myaiabnkhoj-server-1 | '26485c08-2711-4a10-9a5f-b96e41e1e2a
myaiabnkhoj-server-1 | d', 'subscription_type': 'standard',
myaiabnkhoj-server-1 | 'is_recurring': False, 'client_id':
myaiabnkhoj-server-1 | 'default', 'api': 'chat_history',
myaiabnkhoj-server-1 | 'client': 'web'}, {'telemetry_type':
myaiabnkhoj-server-1 | 'api', 'server_version': '0.0.0',
myaiabnkhoj-server-1 | 'os': 'Linux', 'timestamp':
myaiabnkhoj-server-1 | '2024-06-21 08:04:15',
myaiabnkhoj-server-1 | 'client_host': '172.26.0.1',
myaiabnkhoj-server-1 | 'user_agent': 'Mozilla/5.0 (X11;
myaiabnkhoj-server-1 | Linux x86_64) AppleWebKit/537.36
myaiabnkhoj-server-1 | (KHTML, like Gecko) Chrome/123.0.0.0
myaiabnkhoj-server-1 | Safari/537.36', 'referer':
myaiabnkhoj-server-1 | 'http://localhost:42110/', 'host':
myaiabnkhoj-server-1 | 'localhost:42110', 'server_id':
myaiabnkhoj-server-1 | '26485c08-2711-4a10-9a5f-b96e41e1e2a
myaiabnkhoj-server-1 | d', 'subscription_type': 'standard',
myaiabnkhoj-server-1 | 'is_recurring': False, 'client_id':
myaiabnkhoj-server-1 | 'default', 'num_org': 0,
myaiabnkhoj-server-1 | 'num_markdown': 0, 'num_pdf': 1,
myaiabnkhoj-server-1 | 'num_plaintext': 0, 'num_docx': 0,
myaiabnkhoj-server-1 | 'api': 'index/update', 'client':
myaiabnkhoj-server-1 | 'web'}, {'telemetry_type': 'api',
myaiabnkhoj-server-1 | 'server_version': '0.0.0', 'os':
myaiabnkhoj-server-1 | 'Linux', 'timestamp': '2024-06-21
myaiabnkhoj-server-1 | 08:04:15', 'client_host':
myaiabnkhoj-server-1 | '172.26.0.1', 'user_agent':
myaiabnkhoj-server-1 | 'unknown', 'referer': 'unknown',
myaiabnkhoj-server-1 | 'host': 'unknown', 'server_id':
myaiabnkhoj-server-1 | '26485c08-2711-4a10-9a5f-b96e41e1e2a
myaiabnkhoj-server-1 | d', 'subscription_type': 'standard',
myaiabnkhoj-server-1 | 'is_recurring': False, 'client_id':
myaiabnkhoj-server-1 | 'default', 'api':
myaiabnkhoj-server-1 | 'get_all_filenames'}]
myaiabnkhoj-server-1 | You are using the default legacy behaviour of the <class 'transformers.models.llama.tokenization_llama_fast.LlamaTokenizerFast'>. This is expected, and simply means that the legacy (previous) behavior will be used so nothing changes for you. If you want to use the new behaviour, set legacy=False. This should only be set if you understand what it means, and thoroughly read the reason why this was added as explained in https://github.com/huggingface/transformers/pull/24565 myaiabnkhoj-server-1 | [08:04:52.298187] WARNING khoj.processor.conversation.utils: utils.py:217 myaiabnkhoj-server-1 | Fallback to default chat model
myaiabnkhoj-server-1 | tokenizer: None.
myaiabnkhoj-server-1 | Configure tokenizer for unsupported
myaiabnkhoj-server-1 | model: llama3-70b-8192 in Khoj settings
myaiabnkhoj-server-1 | to improve context stuffing.
myaiabnkhoj-server-1 | [08:04:52.682359] DEBUG khoj.processor.conversation.openai before_sleep.py:65 myaiabnkhoj-server-1 | .utils: Retrying
myaiabnkhoj-server-1 | khoj.processor.conversation.openai
myaiabnkhoj-server-1 | .utils.completion_with_backoff in
myaiabnkhoj-server-1 | 0.6461066858467513 seconds as it
myaiabnkhoj-server-1 | raised BadRequestError: Error
myaiabnkhoj-server-1 | code: 400 - {'error': {'message':
myaiabnkhoj-server-1 | 'response_formatdoes not support myaiabnkhoj-server-1 | streaming', 'type': myaiabnkhoj-server-1 | 'invalid_request_error'}}. myaiabnkhoj-server-1 | [08:04:53.536637] DEBUG khoj.processor.conversation.openai before_sleep.py:65 myaiabnkhoj-server-1 | .utils: Retrying myaiabnkhoj-server-1 | khoj.processor.conversation.openai myaiabnkhoj-server-1 | .utils.completion_with_backoff in myaiabnkhoj-server-1 | 1.0890849572597519 seconds as it myaiabnkhoj-server-1 | raised BadRequestError: Error myaiabnkhoj-server-1 | code: 400 - {'error': {'message': myaiabnkhoj-server-1 | 'response_format does not support
myaiabnkhoj-server-1 | streaming', 'type':
myaiabnkhoj-server-1 | 'invalid_request_error'}}.
myaiabnkhoj-server-1 | [08:04:54.883765] DEBUG khoj.routers.helpers: Chat actor: helpers.py:175 myaiabnkhoj-server-1 | Infer information sources to refer:
myaiabnkhoj-server-1 | 7.722 seconds
myaiabnkhoj-server-1 | [08:04:54.888881] ERROR uvicorn.error: Exception in websockets_impl.py:188 myaiabnkhoj-server-1 | ASGI application
myaiabnkhoj-server-1 |
myaiabnkhoj-server-1 | ╭─ Traceback (most recent ca─╮
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/uvicorn/prot │
myaiabnkhoj-server-1 | │ ocols/websockets/websocket │
myaiabnkhoj-server-1 | │ s_impl.py:184 in run_asgi │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 181 │ │ termination │
myaiabnkhoj-server-1 | │ 182 │ │ """ │
myaiabnkhoj-server-1 | │ 183 │ │ try: │
myaiabnkhoj-server-1 | │ ❱ 184 │ │ │ result = │
myaiabnkhoj-server-1 | │ 185 │ │ except BaseE │
myaiabnkhoj-server-1 | │ 186 │ │ │ self.clo │
myaiabnkhoj-server-1 | │ 187 │ │ │ msg = "E │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/uvicorn/midd │
myaiabnkhoj-server-1 | │ leware/proxy_headers.py:75 │
myaiabnkhoj-server-1 | │ in call │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 72 │ │ │ │ │ p │
myaiabnkhoj-server-1 | │ 73 │ │ │ │ │ s │
myaiabnkhoj-server-1 | │ 74 │ │ │
myaiabnkhoj-server-1 | │ ❱ 75 │ │ return await │
myaiabnkhoj-server-1 | │ 76 │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/fastapi/appl │
myaiabnkhoj-server-1 | │ ications.py:1054 in │
myaiabnkhoj-server-1 | │ call │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 1051 │ async def cal │
myaiabnkhoj-server-1 | │ 1052 │ │ if self.roo │
myaiabnkhoj-server-1 | │ 1053 │ │ │ scope[" │
myaiabnkhoj-server-1 | │ ❱ 1054 │ │ await super │
myaiabnkhoj-server-1 | │ 1055 │ │
myaiabnkhoj-server-1 | │ 1056 │ def add_api_rou │
myaiabnkhoj-server-1 | │ 1057 │ │ self, │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/starlette/ap │
myaiabnkhoj-server-1 | │ plications.py:123 in │
myaiabnkhoj-server-1 | │ call │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 120 │ │ scope["app"] │
myaiabnkhoj-server-1 | │ 121 │ │ if self.midd │
myaiabnkhoj-server-1 | │ 122 │ │ │ self.mid │
myaiabnkhoj-server-1 | │ ❱ 123 │ │ await self.m │
myaiabnkhoj-server-1 | │ 124 │ │
myaiabnkhoj-server-1 | │ 125 │ def on_event(sel │
myaiabnkhoj-server-1 | │ 126 │ │ return self. │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/starlette/mi │
myaiabnkhoj-server-1 | │ ddleware/errors.py:151 in │
myaiabnkhoj-server-1 | │ call │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 148 │ │
myaiabnkhoj-server-1 | │ 149 │ async def __call │
myaiabnkhoj-server-1 | │ 150 │ │ if scope["ty │
myaiabnkhoj-server-1 | │ ❱ 151 │ │ │ await se │
myaiabnkhoj-server-1 | │ 152 │ │ │ return │
myaiabnkhoj-server-1 | │ 153 │ │ │
myaiabnkhoj-server-1 | │ 154 │ │ response_sta │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/starlette/mi │
myaiabnkhoj-server-1 | │ ddleware/sessions.py:85 in │
myaiabnkhoj-server-1 | │ call │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 82 │ │ │ │ │ h │
myaiabnkhoj-server-1 | │ 83 │ │ │ await sen │
myaiabnkhoj-server-1 | │ 84 │ │ │
myaiabnkhoj-server-1 | │ ❱ 85 │ │ await self.ap │
myaiabnkhoj-server-1 | │ 86 │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/starlette/mi │
myaiabnkhoj-server-1 | │ ddleware/authentication.py │
myaiabnkhoj-server-1 | │ :49 in call │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 46 │ │ if auth_resul │
myaiabnkhoj-server-1 | │ 47 │ │ │ auth_resu │
myaiabnkhoj-server-1 | │ 48 │ │ scope["auth"] │
myaiabnkhoj-server-1 | │ ❱ 49 │ │ await self.ap │
myaiabnkhoj-server-1 | │ 50 │ │
myaiabnkhoj-server-1 | │ 51 │ @staticmethod │
myaiabnkhoj-server-1 | │ 52 │ def default_on_er │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/starlette/mi │
myaiabnkhoj-server-1 | │ ddleware/cors.py:77 in │
myaiabnkhoj-server-1 | │ call │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 74 │ │
myaiabnkhoj-server-1 | │ 75 │ async def call │
myaiabnkhoj-server-1 | │ 76 │ │ if scope["ty │
myaiabnkhoj-server-1 | │ ❱ 77 │ │ │ await se │
myaiabnkhoj-server-1 | │ 78 │ │ │ return │
myaiabnkhoj-server-1 | │ 79 │ │ │
myaiabnkhoj-server-1 | │ 80 │ │ method = sco │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/starlette/mi │
myaiabnkhoj-server-1 | │ ddleware/exceptions.py:65 │
myaiabnkhoj-server-1 | │ in call │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 62 │ │ else: │
myaiabnkhoj-server-1 | │ 63 │ │ │ conn = We │
myaiabnkhoj-server-1 | │ 64 │ │ │
myaiabnkhoj-server-1 | │ ❱ 65 │ │ await wrap_ap │
myaiabnkhoj-server-1 | │ 66 │ │
myaiabnkhoj-server-1 | │ 67 │ def http_exceptio │
myaiabnkhoj-server-1 | │ 68 │ │ assert isinst │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/starlette/_e │
myaiabnkhoj-server-1 | │ xception_handler.py:64 in │
myaiabnkhoj-server-1 | │ wrapped_app │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 61 │ │ │ │ handl │
myaiabnkhoj-server-1 | │ 62 │ │ │ │
myaiabnkhoj-server-1 | │ 63 │ │ │ if handle │
myaiabnkhoj-server-1 | │ ❱ 64 │ │ │ │ raise │
myaiabnkhoj-server-1 | │ 65 │ │ │ │
myaiabnkhoj-server-1 | │ 66 │ │ │ if respon │
myaiabnkhoj-server-1 | │ 67 │ │ │ │ msg = │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/starlette/_e │
myaiabnkhoj-server-1 | │ xception_handler.py:53 in │
myaiabnkhoj-server-1 | │ wrapped_app │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 50 │ │ │ await sen │
myaiabnkhoj-server-1 | │ 51 │ │ │
myaiabnkhoj-server-1 | │ 52 │ │ try: │
myaiabnkhoj-server-1 | │ ❱ 53 │ │ │ await app │
myaiabnkhoj-server-1 | │ 54 │ │ except Except │
myaiabnkhoj-server-1 | │ 55 │ │ │ handler = │
myaiabnkhoj-server-1 | │ 56 │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/starlette/ro │
myaiabnkhoj-server-1 | │ uting.py:756 in call │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 753 │ │ """ │
myaiabnkhoj-server-1 | │ 754 │ │ The main ent │
myaiabnkhoj-server-1 | │ 755 │ │ """ │
myaiabnkhoj-server-1 | │ ❱ 756 │ │ await self.m │
myaiabnkhoj-server-1 | │ 757 │ │
myaiabnkhoj-server-1 | │ 758 │ async def app(se │
myaiabnkhoj-server-1 | │ 759 │ │ assert scope │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/starlette/ro │
myaiabnkhoj-server-1 | │ uting.py:776 in app │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 773 │ │ │ match, c │
myaiabnkhoj-server-1 | │ 774 │ │ │ if match │
myaiabnkhoj-server-1 | │ 775 │ │ │ │ scop │
myaiabnkhoj-server-1 | │ ❱ 776 │ │ │ │ awai │
myaiabnkhoj-server-1 | │ 777 │ │ │ │ retu │
myaiabnkhoj-server-1 | │ 778 │ │ │ elif mat │
myaiabnkhoj-server-1 | │ 779 │ │ │ │ part │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/starlette/ro │
myaiabnkhoj-server-1 | │ uting.py:373 in handle │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 370 │ │ return URLPa │
myaiabnkhoj-server-1 | │ 371 │ │
myaiabnkhoj-server-1 | │ 372 │ async def handle │
myaiabnkhoj-server-1 | │ ❱ 373 │ │ await self.a │
myaiabnkhoj-server-1 | │ 374 │ │
myaiabnkhoj-server-1 | │ 375 │ def eq(self, │
myaiabnkhoj-server-1 | │ 376 │ │ return ( │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/starlette/ro │
myaiabnkhoj-server-1 | │ uting.py:96 in app │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 93 │ │ async def ap │
myaiabnkhoj-server-1 | │ 94 │ │ │ await fu │
myaiabnkhoj-server-1 | │ 95 │ │ │
myaiabnkhoj-server-1 | │ ❱ 96 │ │ await wrap_a │
myaiabnkhoj-server-1 | │ 97 │ │
myaiabnkhoj-server-1 | │ 98 │ return app │
myaiabnkhoj-server-1 | │ 99 │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/starlette/_e │
myaiabnkhoj-server-1 | │ xception_handler.py:64 in │
myaiabnkhoj-server-1 | │ wrapped_app │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 61 │ │ │ │ handl │
myaiabnkhoj-server-1 | │ 62 │ │ │ │
myaiabnkhoj-server-1 | │ 63 │ │ │ if handle │
myaiabnkhoj-server-1 | │ ❱ 64 │ │ │ │ raise │
myaiabnkhoj-server-1 | │ 65 │ │ │ │
myaiabnkhoj-server-1 | │ 66 │ │ │ if respon │
myaiabnkhoj-server-1 | │ 67 │ │ │ │ msg = │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/starlette/_e │
myaiabnkhoj-server-1 | │ xception_handler.py:53 in │
myaiabnkhoj-server-1 | │ wrapped_app │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 50 │ │ │ await sen │
myaiabnkhoj-server-1 | │ 51 │ │ │
myaiabnkhoj-server-1 | │ 52 │ │ try: │
myaiabnkhoj-server-1 | │ ❱ 53 │ │ │ await app │
myaiabnkhoj-server-1 | │ 54 │ │ except Except │
myaiabnkhoj-server-1 | │ 55 │ │ │ handler = │
myaiabnkhoj-server-1 | │ 56 │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/starlette/ro │
myaiabnkhoj-server-1 | │ uting.py:94 in app │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 91 │ │ session = We │
myaiabnkhoj-server-1 | │ 92 │ │ │
myaiabnkhoj-server-1 | │ 93 │ │ async def ap │
myaiabnkhoj-server-1 | │ ❱ 94 │ │ │ await fu │
myaiabnkhoj-server-1 | │ 95 │ │ │
myaiabnkhoj-server-1 | │ 96 │ │ await wrapa │
myaiabnkhoj-server-1 | │ 97 │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/fastapi/rout │
myaiabnkhoj-server-1 | │ ing.py:348 in app │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 345 │ │ │ if erro │
myaiabnkhoj-server-1 | │ 346 │ │ │ │ rai │
myaiabnkhoj-server-1 | │ 347 │ │ │ assert │
myaiabnkhoj-server-1 | │ ❱ 348 │ │ │ await d │
myaiabnkhoj-server-1 | │ 349 │ │
myaiabnkhoj-server-1 | │ 350 │ return app │
myaiabnkhoj-server-1 | │ 351 │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /app/src/khoj/routers/api │
myaiabnkhoj-server-1 | │ chat.py:593 in │
myaiabnkhoj-server-1 | │ websocket_endpoint │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 590 │ │ is_automate │
myaiabnkhoj-server-1 | │ 591 │ │ │
myaiabnkhoj-server-1 | │ 592 │ │ if conversa │
myaiabnkhoj-server-1 | │ ❱ 593 │ │ │ convers │
myaiabnkhoj-server-1 | │ is_automated_task) │
myaiabnkhoj-server-1 | │ 594 │ │ │ convers │
myaiabnkhoj-server-1 | │ conversation_comman │
myaiabnkhoj-server-1 | │ 595 │ │ │ await s │
myaiabnkhoj-server-1 | │ {conversation_comma │
myaiabnkhoj-server-1 | │ 596 │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /app/src/khoj/routers/help │
myaiabnkhoj-server-1 | │ ers.py:250 in │
myaiabnkhoj-server-1 | │ aget_relevantinformation │
myaiabnkhoj-server-1 | │ sources │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 247 │ ) │
myaiabnkhoj-server-1 | │ 248 │ │
myaiabnkhoj-server-1 | │ 249 │ with timer("Cha │
myaiabnkhoj-server-1 | │ ❱ 250 │ │ response = │
myaiabnkhoj-server-1 | │ response_type="json │
myaiabnkhoj-server-1 | │ 251 │ │
myaiabnkhoj-server-1 | │ 252 │ try: │
myaiabnkhoj-server-1 | │ 253 │ │ response = │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /app/src/khoj/routers/help │
myaiabnkhoj-server-1 | │ ers.py:547 in │
myaiabnkhoj-server-1 | │ send_message_to_model_wrap │
myaiabnkhoj-server-1 | │ per │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 544 │ │ │ tokeniz │
myaiabnkhoj-server-1 | │ 545 │ │ ) │
myaiabnkhoj-server-1 | │ 546 │ │ │
myaiabnkhoj-server-1 | │ ❱ 547 │ │ openai_resp │
myaiabnkhoj-server-1 | │ 548 │ │ │ message │
myaiabnkhoj-server-1 | │ 549 │ │ │ api_key │
myaiabnkhoj-server-1 | │ 550 │ │ │ model=c │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /app/src/khoj/processor/co │
myaiabnkhoj-server-1 | │ nversation/openai/gpt.py:9 │
myaiabnkhoj-server-1 | │ 8 in send_message_to_model │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 95 │ """ │
myaiabnkhoj-server-1 | │ 96 │ │
myaiabnkhoj-server-1 | │ 97 │ # Get Response f │
myaiabnkhoj-server-1 | │ ❱ 98 │ return completio │
myaiabnkhoj-server-1 | │ 99 │ │ messages=mes │
myaiabnkhoj-server-1 | │ 100 │ │ model=model, │
myaiabnkhoj-server-1 | │ 101 │ │ openai_api_k │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/tenacity/i │
myaiabnkhoj-server-1 | │ nit.py:330 in wrappedf │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 327 │ │ │ f, funct │
myaiabnkhoj-server-1 | │ 328 │ │ ) │
myaiabnkhoj-server-1 | │ 329 │ │ def wrapped │
myaiabnkhoj-server-1 | │ ❱ 330 │ │ │ return s │
myaiabnkhoj-server-1 | │ 331 │ │ │
myaiabnkhoj-server-1 | │ 332 │ │ def retry_wi │
myaiabnkhoj-server-1 | │ 333 │ │ │ return s │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/tenacity/i │
myaiabnkhoj-server-1 | │ nit.py:467 in call │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 464 │ │ │
myaiabnkhoj-server-1 | │ 465 │ │ retry_state │
myaiabnkhoj-server-1 | │ 466 │ │ while True: │
myaiabnkhoj-server-1 | │ ❱ 467 │ │ │ do = sel │
myaiabnkhoj-server-1 | │ 468 │ │ │ if isins │
myaiabnkhoj-server-1 | │ 469 │ │ │ │ try: │
myaiabnkhoj-server-1 | │ 470 │ │ │ │ │ │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/tenacity/i │
myaiabnkhoj-server-1 | │ nit.py:368 in iter │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 365 │ │ self.begin │
myaiabnkhoj-server-1 | │ 366 │ │ result = Non │
myaiabnkhoj-server-1 | │ 367 │ │ for action i │
myaiabnkhoj-server-1 | │ ❱ 368 │ │ │ result = │
myaiabnkhoj-server-1 | │ 369 │ │ return resul │
myaiabnkhoj-server-1 | │ 370 │ │
myaiabnkhoj-server-1 | │ 371 │ def _begin_iter( │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/tenacity/i │
myaiabnkhoj-server-1 | │ nit.py:410 in exc_check │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 407 │ │ │ │ fut │
myaiabnkhoj-server-1 | │ 408 │ │ │ │ retr │
myaiabnkhoj-server-1 | │ 409 │ │ │ │ if s │
myaiabnkhoj-server-1 | │ ❱ 410 │ │ │ │ │ │
myaiabnkhoj-server-1 | │ 411 │ │ │ │ rais │
myaiabnkhoj-server-1 | │ 412 │ │ │ │
myaiabnkhoj-server-1 | │ 413 │ │ │ self._ad │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/tenacity/i │
myaiabnkhoj-server-1 | │ nit.py:183 in reraise │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 180 │ │
myaiabnkhoj-server-1 | │ 181 │ def reraise(self │
myaiabnkhoj-server-1 | │ 182 │ │ if self.last │
myaiabnkhoj-server-1 | │ ❱ 183 │ │ │ raise se │
myaiabnkhoj-server-1 | │ 184 │ │ raise self │
myaiabnkhoj-server-1 | │ 185 │ │
myaiabnkhoj-server-1 | │ 186 │ def str(self │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/lib/python3.10/concur │
myaiabnkhoj-server-1 | │ rent/futures/_base.py:451 │
myaiabnkhoj-server-1 | │ in result │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 448 │ │ │ │ if s │
myaiabnkhoj-server-1 | │ 449 │ │ │ │ │ │
myaiabnkhoj-server-1 | │ 450 │ │ │ │ elif │
myaiabnkhoj-server-1 | │ ❱ 451 │ │ │ │ │ │
myaiabnkhoj-server-1 | │ 452 │ │ │ │ │
myaiabnkhoj-server-1 | │ 453 │ │ │ │ self │
myaiabnkhoj-server-1 | │ 454 │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/lib/python3.10/concur │
myaiabnkhoj-server-1 | │ rent/futures/_base.py:403 │
myaiabnkhoj-server-1 | │ in __get_result │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 400 │ def get_result │
myaiabnkhoj-server-1 | │ 401 │ │ if self._exc │
myaiabnkhoj-server-1 | │ 402 │ │ │ try: │
myaiabnkhoj-server-1 | │ ❱ 403 │ │ │ │ rais │
myaiabnkhoj-server-1 | │ 404 │ │ │ finally: │
myaiabnkhoj-server-1 | │ 405 │ │ │ │ # Br │
myaiabnkhoj-server-1 | │ 406 │ │ │ │ self │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/tenacity/i │
myaiabnkhoj-server-1 | │ nit.py:470 in call__ │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 467 │ │ │ do = sel │
myaiabnkhoj-server-1 | │ 468 │ │ │ if isins │
myaiabnkhoj-server-1 | │ 469 │ │ │ │ try: │
myaiabnkhoj-server-1 | │ ❱ 470 │ │ │ │ │ │
myaiabnkhoj-server-1 | │ 471 │ │ │ │ exce │
myaiabnkhoj-server-1 | │ 472 │ │ │ │ │ │
myaiabnkhoj-server-1 | │ 473 │ │ │ │ else │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /app/src/khoj/processor/co │
myaiabnkhoj-server-1 | │ nversation/openai/utils.py │
myaiabnkhoj-server-1 | │ :50 in │
myaiabnkhoj-server-1 | │ completion_with_backoff │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 47 │ │
myaiabnkhoj-server-1 | │ 48 │ formatted_messag │
myaiabnkhoj-server-1 | │ in messages] │
myaiabnkhoj-server-1 | │ 49 │ │
myaiabnkhoj-server-1 | │ ❱ 50 │ chat = client.ch │
myaiabnkhoj-server-1 | │ 51 │ │ stream=True, │
myaiabnkhoj-server-1 | │ 52 │ │ messages=for │
myaiabnkhoj-server-1 | │ 53 │ │ model=model, │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/openai/_util │
myaiabnkhoj-server-1 | │ s/_utils.py:277 in wrapper │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 274 │ │ │ │ │ │
myaiabnkhoj-server-1 | │ 275 │ │ │ │ │ │
myaiabnkhoj-server-1 | │ 276 │ │ │ │ rais │
myaiabnkhoj-server-1 | │ ❱ 277 │ │ │ return f │
myaiabnkhoj-server-1 | │ 278 │ │ │
myaiabnkhoj-server-1 | │ 279 │ │ return wrapp │
myaiabnkhoj-server-1 | │ 280 │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/openai/resou │
myaiabnkhoj-server-1 | │ rces/chat/completions.py:6 │
myaiabnkhoj-server-1 | │ 40 in create │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 637 │ │ extra_body: │
myaiabnkhoj-server-1 | │ 638 │ │ timeout: fl │
myaiabnkhoj-server-1 | │ 639 │ ) -> ChatComple │
myaiabnkhoj-server-1 | │ ❱ 640 │ │ return self │
myaiabnkhoj-server-1 | │ 641 │ │ │ "/chat/ │
myaiabnkhoj-server-1 | │ 642 │ │ │ body=ma │
myaiabnkhoj-server-1 | │ 643 │ │ │ │ { │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/openai/_base │
myaiabnkhoj-server-1 | │ _client.py:1250 in post │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 1247 │ │ opts = Fina │
myaiabnkhoj-server-1 | │ 1248 │ │ │ method= │
myaiabnkhoj-server-1 | │ **options │
myaiabnkhoj-server-1 | │ 1249 │ │ ) │
myaiabnkhoj-server-1 | │ ❱ 1250 │ │ return cast │
myaiabnkhoj-server-1 | │ stream_cls=stream_c │
myaiabnkhoj-server-1 | │ 1251 │ │
myaiabnkhoj-server-1 | │ 1252 │ def patch( │
myaiabnkhoj-server-1 | │ 1253 │ │ self, │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/openai/_base │
myaiabnkhoj-server-1 | │ _client.py:931 in request │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 928 │ │ stream: boo │
myaiabnkhoj-server-1 | │ 929 │ │ stream_cls: │
myaiabnkhoj-server-1 | │ 930 │ ) -> ResponseT │
myaiabnkhoj-server-1 | │ ❱ 931 │ │ return self │
myaiabnkhoj-server-1 | │ 932 │ │ │ cast_to │
myaiabnkhoj-server-1 | │ 933 │ │ │ options │
myaiabnkhoj-server-1 | │ 934 │ │ │ stream= │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/openai/_base │
myaiabnkhoj-server-1 | │ _client.py:1030 in │
myaiabnkhoj-server-1 | │ _request │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 1027 │ │ │ │ err │
myaiabnkhoj-server-1 | │ 1028 │ │ │ │
myaiabnkhoj-server-1 | │ 1029 │ │ │ log.deb │
myaiabnkhoj-server-1 | │ ❱ 1030 │ │ │ raise s │
myaiabnkhoj-server-1 | │ 1031 │ │ │
myaiabnkhoj-server-1 | │ 1032 │ │ return self │
myaiabnkhoj-server-1 | │ 1033 │ │ │ cast_to │
myaiabnkhoj-server-1 | ╰────────────────────────────╯
myaiabnkhoj-server-1 | BadRequestError: Error code:
myaiabnkhoj-server-1 | 400 - {'error': {'message':
myaiabnkhoj-server-1 | 'response_format` does not
myaiabnkhoj-server-1 | support streaming', 'type':
myaiabnkhoj-server-1 | 'invalid_request_error'}}
myaiabnkhoj-server-1 | [08:04:55.971712] DEBUG uvicorn.error: = connection is protocol.py:1227 myaiabnkhoj-server-1 | CLOSING
myaiabnkhoj-server-1 | [08:04:55.972998] DEBUG uvicorn.error: > CLOSE 1000 (OK) [2 protocol.py:1178 myaiabnkhoj-server-1 | bytes]
myaiabnkhoj-server-1 | [08:04:55.974311] DEBUG uvicorn.error: = connection is protocol.py:1497 myaiabnkhoj-server-1 | CLOSED
myaiabnkhoj-server-1 | [08:04:55.975695] DEBUG uvicorn.error: ! failing connection protocol.py:1412 myaiabnkhoj-server-1 | with code 1006
myaiabnkhoj-server-1 | [08:04:55.977672] ERROR uvicorn.error: closing handshake failed server.py:248 myaiabnkhoj-server-1 | ╭─ Traceback (most recent call last) ─╮
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/dist-pack │
myaiabnkhoj-server-1 | │ ages/websockets/legacy/server.py:24 │
myaiabnkhoj-server-1 | │ 4 in handler │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 241 │ │ │ │ raise │
myaiabnkhoj-server-1 | │ 242 │ │ │ │
myaiabnkhoj-server-1 | │ 243 │ │ │ try: │
myaiabnkhoj-server-1 | │ ❱ 244 │ │ │ │ await self.c │
myaiabnkhoj-server-1 | │ 245 │ │ │ except Connectio │
myaiabnkhoj-server-1 | │ 246 │ │ │ │ raise │
myaiabnkhoj-server-1 | │ 247 │ │ │ except Exception │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/dist-pack │
myaiabnkhoj-server-1 | │ ages/websockets/legacy/protocol.py: │
myaiabnkhoj-server-1 | │ 770 in close │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 767 │ │ """ │
myaiabnkhoj-server-1 | │ 768 │ │ try: │
myaiabnkhoj-server-1 | │ 769 │ │ │ async with async │
myaiabnkhoj-server-1 | │ ❱ 770 │ │ │ │ await self.w │
myaiabnkhoj-server-1 | │ 771 │ │ except asyncio.Timeo │
myaiabnkhoj-server-1 | │ 772 │ │ │ # If the close f │
myaiabnkhoj-server-1 | │ 773 │ │ │ # are full, the │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/dist-pack │
myaiabnkhoj-server-1 | │ ages/websockets/legacy/protocol.py: │
myaiabnkhoj-server-1 | │ 1236 in write_close_frame │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 1233 │ │ │ │ data = close │
myaiabnkhoj-server-1 | │ 1234 │ │ │ │
myaiabnkhoj-server-1 | │ 1235 │ │ │ # 7.1.2. Start t │
myaiabnkhoj-server-1 | │ ❱ 1236 │ │ │ await self.write │
myaiabnkhoj-server-1 | │ 1237 │ │
myaiabnkhoj-server-1 | │ 1238 │ async def keepalive_ping │
myaiabnkhoj-server-1 | │ 1239 │ │ """ │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/dist-pack │
myaiabnkhoj-server-1 | │ ages/websockets/legacy/protocol.py: │
myaiabnkhoj-server-1 | │ 1209 in write_frame │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 1206 │ │ │ │ f"Cannot wri │
myaiabnkhoj-server-1 | │ 1207 │ │ │ ) │
myaiabnkhoj-server-1 | │ 1208 │ │ self.write_frame_syn │
myaiabnkhoj-server-1 | │ ❱ 1209 │ │ await self.drain() │
myaiabnkhoj-server-1 | │ 1210 │ │
myaiabnkhoj-server-1 | │ 1211 │ async def write_close_fr │
myaiabnkhoj-server-1 | │ 1212 │ │ self, close: Close, │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/dist-pack │
myaiabnkhoj-server-1 | │ ages/websockets/legacy/protocol.py: │
myaiabnkhoj-server-1 | │ 1198 in drain │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 1195 │ │ │ self.fail_connec │
myaiabnkhoj-server-1 | │ 1196 │ │ │ # Wait until the │
myaiabnkhoj-server-1 | │ 1197 │ │ │ # with the corre │
myaiabnkhoj-server-1 | │ ❱ 1198 │ │ │ await self.ensur │
myaiabnkhoj-server-1 | │ 1199 │ │
myaiabnkhoj-server-1 | │ 1200 │ async def write_frame( │
myaiabnkhoj-server-1 | │ 1201 │ │ self, fin: bool, opc │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/dist-pack │
myaiabnkhoj-server-1 | │ ages/websockets/legacy/protocol.py: │
myaiabnkhoj-server-1 | │ 939 in ensure_open │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 936 │ │ │ │ return │
myaiabnkhoj-server-1 | │ 937 │ │ │
myaiabnkhoj-server-1 | │ 938 │ │ if self.state is Sta │
myaiabnkhoj-server-1 | │ ❱ 939 │ │ │ raise self.conne │
myaiabnkhoj-server-1 | │ 940 │ │ │
myaiabnkhoj-server-1 | │ 941 │ │ if self.state is Sta │
myaiabnkhoj-server-1 | │ 942 │ │ │ # If we started │
myaiabnkhoj-server-1 | ╰─────────────────────────────────────╯
myaiabnkhoj-server-1 | ConnectionClosedError: sent 1000 (OK);
myaiabnkhoj-server-1 | no close frame received
myaiabnkhoj-server-1 | [08:04:56.261760] INFO uvicorn.error: connection closed server.py:264 myaiabnkhoj-server-1 | [08:04:56.262973] DEBUG uvicorn.error: x half-closing TCP protocol.py:1319 myaiabnkhoj-server-1 | connection

`

stevennt commented 4 months ago

This doesn't work: What are the number of articles in Wikipedia?

This works: /general What are the number of articles in Wikipedia?

@sabaimran

stevennt commented 4 months ago

I also saw this in a console log:

BadRequestError: Error code:
400 - {'error': {'message':
'response_format` does not
support streaming', 'type':
'invalid_request_error'}}

sabaimran commented 4 months ago

Hmm interesting. You should run docker compose up --build when you're restarting the container after pulling from the latest changes. But you're right, this is likely a different issue. I'd assumed you were using ollama / llama3 and tested on that. The tokenizer shouldn't matter here. /general is still working, correct?

stevennt commented 4 months ago

API: I'm not using Ollama. I'm using a vendor-hosted LLAMA3 API. With that API: Yes /general works faultlessly but the moment I say /notes or /default it breaks (regardless of whether or not I have uploaded any document).

At the same time, OpenAI API works faultlessly with both /general and /notes.

Yes I do run docker-compose build with the revised code. Other test with text changes do show up, meaning the build process was ok.

@sabaimran

stevennt commented 4 months ago

Wrapping the API in LiteLLM:

myaiabnkhoj-server-1 | [01:05:12.918899] DEBUG uvicorn.error: < TEXT 'How can I protocol.py:1172 myaiabnkhoj-server-1 | talk to an LLM' [24 bytes]
myaiabnkhoj-server-1 | [01:05:12.928024] DEBUG uvicorn.error: > TEXT '{"type": protocol.py:1178 myaiabnkhoj-server-1 | "status", "message":
myaiabnkhoj-server-1 | "**Understanding ...e":
myaiabnkhoj-server-1 | "application/json"}' [118 bytes]
myaiabnkhoj-server-1 | [01:05:12.941103] WARNING khoj.processor.conversation.utils: utils.py:217 myaiabnkhoj-server-1 | Fallback to default chat model
myaiabnkhoj-server-1 | tokenizer: None.
myaiabnkhoj-server-1 | Configure tokenizer for unsupported
myaiabnkhoj-server-1 | model: llama3-70b-8192 in Khoj settings
myaiabnkhoj-server-1 | to improve context stuffing.
myaiabnkhoj-server-1 | [01:05:14.901754] DEBUG khoj.processor.conversation.openai before_sleep.py:65 myaiabnkhoj-server-1 | .utils: Retrying
myaiabnkhoj-server-1 | khoj.processor.conversation.openai
myaiabnkhoj-server-1 | .utils.completion_with_backoff in
myaiabnkhoj-server-1 | 0.8713305851005899 seconds as it
myaiabnkhoj-server-1 | raised BadRequestError: Error
myaiabnkhoj-server-1 | code: 400 - {'error': {'message':
myaiabnkhoj-server-1 | "litellm.BadRequestError:
myaiabnkhoj-server-1 | BadRequestError: GroqException -
myaiabnkhoj-server-1 | Error code: 400 - {'error':
myaiabnkhoj-server-1 | {'message': 'response_formatdoes myaiabnkhoj-server-1 | not support streaming', 'type': myaiabnkhoj-server-1 | 'invalid_request_error'}}", myaiabnkhoj-server-1 | 'type': None, 'param': None, myaiabnkhoj-server-1 | 'code': 400}}. myaiabnkhoj-server-1 | [01:05:16.811876] DEBUG khoj.processor.conversation.openai before_sleep.py:65 myaiabnkhoj-server-1 | .utils: Retrying myaiabnkhoj-server-1 | khoj.processor.conversation.openai myaiabnkhoj-server-1 | .utils.completion_with_backoff in myaiabnkhoj-server-1 | 0.04274018957486159 seconds as it myaiabnkhoj-server-1 | raised BadRequestError: Error myaiabnkhoj-server-1 | code: 400 - {'error': {'message': myaiabnkhoj-server-1 | "litellm.BadRequestError: myaiabnkhoj-server-1 | BadRequestError: GroqException - myaiabnkhoj-server-1 | Error code: 400 - {'error': myaiabnkhoj-server-1 | {'message': 'response_format does
myaiabnkhoj-server-1 | not support streaming', 'type':
myaiabnkhoj-server-1 | 'invalid_request_error'}}",
myaiabnkhoj-server-1 | 'type': None, 'param': None,
myaiabnkhoj-server-1 | 'code': 400}}.
myaiabnkhoj-server-1 | [01:05:17.897260] DEBUG khoj.routers.helpers: Chat actor: helpers.py:175 myaiabnkhoj-server-1 | Infer information sources to refer:
myaiabnkhoj-server-1 | 4.964 seconds
myaiabnkhoj-server-1 | [01:05:17.901853] ERROR uvicorn.error: Exception in websockets_impl.py:188 myaiabnkhoj-server-1 | ASGI application
myaiabnkhoj-server-1 |
myaiabnkhoj-server-1 | ╭─ Traceback (most recent ca─╮
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/uvicorn/prot │
myaiabnkhoj-server-1 | │ ocols/websockets/websocket │
myaiabnkhoj-server-1 | │ s_impl.py:184 in run_asgi │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 181 │ │ termination │
myaiabnkhoj-server-1 | │ 182 │ │ """ │
myaiabnkhoj-server-1 | │ 183 │ │ try: │
myaiabnkhoj-server-1 | │ ❱ 184 │ │ │ result = │
myaiabnkhoj-server-1 | │ 185 │ │ except BaseE │
myaiabnkhoj-server-1 | │ 186 │ │ │ self.clo │
myaiabnkhoj-server-1 | │ 187 │ │ │ msg = "E │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/uvicorn/midd │
myaiabnkhoj-server-1 | │ leware/proxy_headers.py:75 │
myaiabnkhoj-server-1 | │ in call │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 72 │ │ │ │ │ p │
myaiabnkhoj-server-1 | │ 73 │ │ │ │ │ s │
myaiabnkhoj-server-1 | │ 74 │ │ │
myaiabnkhoj-server-1 | │ ❱ 75 │ │ return await │
myaiabnkhoj-server-1 | │ 76 │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/fastapi/appl │
myaiabnkhoj-server-1 | │ ications.py:1054 in │
myaiabnkhoj-server-1 | │ call │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 1051 │ async def cal │
myaiabnkhoj-server-1 | │ 1052 │ │ if self.roo │
myaiabnkhoj-server-1 | │ 1053 │ │ │ scope[" │
myaiabnkhoj-server-1 | │ ❱ 1054 │ │ await super │
myaiabnkhoj-server-1 | │ 1055 │ │
myaiabnkhoj-server-1 | │ 1056 │ def add_api_rou │
myaiabnkhoj-server-1 | │ 1057 │ │ self, │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/starlette/ap │
myaiabnkhoj-server-1 | │ plications.py:123 in │
myaiabnkhoj-server-1 | │ call │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 120 │ │ scope["app"] │
myaiabnkhoj-server-1 | │ 121 │ │ if self.midd │
myaiabnkhoj-server-1 | │ 122 │ │ │ self.mid │
myaiabnkhoj-server-1 | │ ❱ 123 │ │ await self.m │
myaiabnkhoj-server-1 | │ 124 │ │
myaiabnkhoj-server-1 | │ 125 │ def on_event(sel │
myaiabnkhoj-server-1 | │ 126 │ │ return self. │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/starlette/mi │
myaiabnkhoj-server-1 | │ ddleware/errors.py:151 in │
myaiabnkhoj-server-1 | │ call │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 148 │ │
myaiabnkhoj-server-1 | │ 149 │ async def __call │
myaiabnkhoj-server-1 | │ 150 │ │ if scope["ty │
myaiabnkhoj-server-1 | │ ❱ 151 │ │ │ await se │
myaiabnkhoj-server-1 | │ 152 │ │ │ return │
myaiabnkhoj-server-1 | │ 153 │ │ │
myaiabnkhoj-server-1 | │ 154 │ │ response_sta │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/starlette/mi │
myaiabnkhoj-server-1 | │ ddleware/sessions.py:85 in │
myaiabnkhoj-server-1 | │ call │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 82 │ │ │ │ │ h │
myaiabnkhoj-server-1 | │ 83 │ │ │ await sen │
myaiabnkhoj-server-1 | │ 84 │ │ │
myaiabnkhoj-server-1 | │ ❱ 85 │ │ await self.ap │
myaiabnkhoj-server-1 | │ 86 │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /app/src/khoj/configure.py │
myaiabnkhoj-server-1 | │ :315 in call │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 312 │ │ async def │
myaiabnkhoj-server-1 | │ 313 │ │ │ if scope │
myaiabnkhoj-server-1 | │ 314 │ │ │ │ scop │
myaiabnkhoj-server-1 | │ ❱ 315 │ │ │ await se │
myaiabnkhoj-server-1 | │ 316 │ │ │
myaiabnkhoj-server-1 | │ 317 │ │ def init │
myaiabnkhoj-server-1 | │ 318 │ │ │ super(). │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/starlette/mi │
myaiabnkhoj-server-1 | │ ddleware/authentication.py │
myaiabnkhoj-server-1 | │ :49 in call │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 46 │ │ if auth_resul │
myaiabnkhoj-server-1 | │ 47 │ │ │ auth_resu │
myaiabnkhoj-server-1 | │ 48 │ │ scope["auth"] │
myaiabnkhoj-server-1 | │ ❱ 49 │ │ await self.ap │
myaiabnkhoj-server-1 | │ 50 │ │
myaiabnkhoj-server-1 | │ 51 │ @staticmethod │
myaiabnkhoj-server-1 | │ 52 │ def default_on_er │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/starlette/mi │
myaiabnkhoj-server-1 | │ ddleware/cors.py:77 in │
myaiabnkhoj-server-1 | │ call │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 74 │ │
myaiabnkhoj-server-1 | │ 75 │ async def call │
myaiabnkhoj-server-1 | │ 76 │ │ if scope["ty │
myaiabnkhoj-server-1 | │ ❱ 77 │ │ │ await se │
myaiabnkhoj-server-1 | │ 78 │ │ │ return │
myaiabnkhoj-server-1 | │ 79 │ │ │
myaiabnkhoj-server-1 | │ 80 │ │ method = sco │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/starlette/mi │
myaiabnkhoj-server-1 | │ ddleware/exceptions.py:65 │
myaiabnkhoj-server-1 | │ in call │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 62 │ │ else: │
myaiabnkhoj-server-1 | │ 63 │ │ │ conn = We │
myaiabnkhoj-server-1 | │ 64 │ │ │
myaiabnkhoj-server-1 | │ ❱ 65 │ │ await wrap_ap │
myaiabnkhoj-server-1 | │ 66 │ │
myaiabnkhoj-server-1 | │ 67 │ def http_exceptio │
myaiabnkhoj-server-1 | │ 68 │ │ assert isinst │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/starlette/_e │
myaiabnkhoj-server-1 | │ xception_handler.py:64 in │
myaiabnkhoj-server-1 | │ wrapped_app │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 61 │ │ │ │ handl │
myaiabnkhoj-server-1 | │ 62 │ │ │ │
myaiabnkhoj-server-1 | │ 63 │ │ │ if handle │
myaiabnkhoj-server-1 | │ ❱ 64 │ │ │ │ raise │
myaiabnkhoj-server-1 | │ 65 │ │ │ │
myaiabnkhoj-server-1 | │ 66 │ │ │ if respon │
myaiabnkhoj-server-1 | │ 67 │ │ │ │ msg = │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/starlette/_e │
myaiabnkhoj-server-1 | │ xception_handler.py:53 in │
myaiabnkhoj-server-1 | │ wrapped_app │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 50 │ │ │ await sen │
myaiabnkhoj-server-1 | │ 51 │ │ │
myaiabnkhoj-server-1 | │ 52 │ │ try: │
myaiabnkhoj-server-1 | │ ❱ 53 │ │ │ await app │
myaiabnkhoj-server-1 | │ 54 │ │ except Except │
myaiabnkhoj-server-1 | │ 55 │ │ │ handler = │
myaiabnkhoj-server-1 | │ 56 │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/starlette/ro │
myaiabnkhoj-server-1 | │ uting.py:756 in call │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 753 │ │ """ │
myaiabnkhoj-server-1 | │ 754 │ │ The main ent │
myaiabnkhoj-server-1 | │ 755 │ │ """ │
myaiabnkhoj-server-1 | │ ❱ 756 │ │ await self.m │
myaiabnkhoj-server-1 | │ 757 │ │
myaiabnkhoj-server-1 | │ 758 │ async def app(se │
myaiabnkhoj-server-1 | │ 759 │ │ assert scope │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/starlette/ro │
myaiabnkhoj-server-1 | │ uting.py:776 in app │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 773 │ │ │ match, c │
myaiabnkhoj-server-1 | │ 774 │ │ │ if match │
myaiabnkhoj-server-1 | │ 775 │ │ │ │ scop │
myaiabnkhoj-server-1 | │ ❱ 776 │ │ │ │ awai │
myaiabnkhoj-server-1 | │ 777 │ │ │ │ retu │
myaiabnkhoj-server-1 | │ 778 │ │ │ elif mat │
myaiabnkhoj-server-1 | │ 779 │ │ │ │ part │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/starlette/ro │
myaiabnkhoj-server-1 | │ uting.py:373 in handle │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 370 │ │ return URLPa │
myaiabnkhoj-server-1 | │ 371 │ │
myaiabnkhoj-server-1 | │ 372 │ async def handle │
myaiabnkhoj-server-1 | │ ❱ 373 │ │ await self.a │
myaiabnkhoj-server-1 | │ 374 │ │
myaiabnkhoj-server-1 | │ 375 │ def eq(self, │
myaiabnkhoj-server-1 | │ 376 │ │ return ( │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/starlette/ro │
myaiabnkhoj-server-1 | │ uting.py:96 in app │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 93 │ │ async def ap │
myaiabnkhoj-server-1 | │ 94 │ │ │ await fu │
myaiabnkhoj-server-1 | │ 95 │ │ │
myaiabnkhoj-server-1 | │ ❱ 96 │ │ await wrap_a │
myaiabnkhoj-server-1 | │ 97 │ │
myaiabnkhoj-server-1 | │ 98 │ return app │
myaiabnkhoj-server-1 | │ 99 │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/starlette/_e │
myaiabnkhoj-server-1 | │ xception_handler.py:64 in │
myaiabnkhoj-server-1 | │ wrapped_app │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 61 │ │ │ │ handl │
myaiabnkhoj-server-1 | │ 62 │ │ │ │
myaiabnkhoj-server-1 | │ 63 │ │ │ if handle │
myaiabnkhoj-server-1 | │ ❱ 64 │ │ │ │ raise │
myaiabnkhoj-server-1 | │ 65 │ │ │ │
myaiabnkhoj-server-1 | │ 66 │ │ │ if respon │
myaiabnkhoj-server-1 | │ 67 │ │ │ │ msg = │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/starlette/_e │
myaiabnkhoj-server-1 | │ xception_handler.py:53 in │
myaiabnkhoj-server-1 | │ wrapped_app │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 50 │ │ │ await sen │
myaiabnkhoj-server-1 | │ 51 │ │ │
myaiabnkhoj-server-1 | │ 52 │ │ try: │
myaiabnkhoj-server-1 | │ ❱ 53 │ │ │ await app │
myaiabnkhoj-server-1 | │ 54 │ │ except Except │
myaiabnkhoj-server-1 | │ 55 │ │ │ handler = │
myaiabnkhoj-server-1 | │ 56 │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/starlette/ro │
myaiabnkhoj-server-1 | │ uting.py:94 in app │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 91 │ │ session = We │
myaiabnkhoj-server-1 | │ 92 │ │ │
myaiabnkhoj-server-1 | │ 93 │ │ async def ap │
myaiabnkhoj-server-1 | │ ❱ 94 │ │ │ await fu │
myaiabnkhoj-server-1 | │ 95 │ │ │
myaiabnkhoj-server-1 | │ 96 │ │ await wrapa │
myaiabnkhoj-server-1 | │ 97 │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/fastapi/rout │
myaiabnkhoj-server-1 | │ ing.py:348 in app │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 345 │ │ │ if erro │
myaiabnkhoj-server-1 | │ 346 │ │ │ │ rai │
myaiabnkhoj-server-1 | │ 347 │ │ │ assert │
myaiabnkhoj-server-1 | │ ❱ 348 │ │ │ await d │
myaiabnkhoj-server-1 | │ 349 │ │
myaiabnkhoj-server-1 | │ 350 │ return app │
myaiabnkhoj-server-1 | │ 351 │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /app/src/khoj/routers/api │
myaiabnkhoj-server-1 | │ chat.py:593 in │
myaiabnkhoj-server-1 | │ websocket_endpoint │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 590 │ │ is_automate │
myaiabnkhoj-server-1 | │ 591 │ │ │
myaiabnkhoj-server-1 | │ 592 │ │ if conversa │
myaiabnkhoj-server-1 | │ ❱ 593 │ │ │ convers │
myaiabnkhoj-server-1 | │ is_automated_task) │
myaiabnkhoj-server-1 | │ 594 │ │ │ convers │
myaiabnkhoj-server-1 | │ conversation_comman │
myaiabnkhoj-server-1 | │ 595 │ │ │ await s │
myaiabnkhoj-server-1 | │ {conversation_comma │
myaiabnkhoj-server-1 | │ 596 │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /app/src/khoj/routers/help │
myaiabnkhoj-server-1 | │ ers.py:250 in │
myaiabnkhoj-server-1 | │ aget_relevantinformation │
myaiabnkhoj-server-1 | │ sources │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 247 │ ) │
myaiabnkhoj-server-1 | │ 248 │ │
myaiabnkhoj-server-1 | │ 249 │ with timer("Cha │
myaiabnkhoj-server-1 | │ ❱ 250 │ │ response = │
myaiabnkhoj-server-1 | │ response_type="json │
myaiabnkhoj-server-1 | │ 251 │ │
myaiabnkhoj-server-1 | │ 252 │ try: │
myaiabnkhoj-server-1 | │ 253 │ │ response = │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /app/src/khoj/routers/help │
myaiabnkhoj-server-1 | │ ers.py:547 in │
myaiabnkhoj-server-1 | │ send_message_to_model_wrap │
myaiabnkhoj-server-1 | │ per │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 544 │ │ │ tokeniz │
myaiabnkhoj-server-1 | │ 545 │ │ ) │
myaiabnkhoj-server-1 | │ 546 │ │ │
myaiabnkhoj-server-1 | │ ❱ 547 │ │ openai_resp │
myaiabnkhoj-server-1 | │ 548 │ │ │ message │
myaiabnkhoj-server-1 | │ 549 │ │ │ api_key │
myaiabnkhoj-server-1 | │ 550 │ │ │ model=c │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /app/src/khoj/processor/co │
myaiabnkhoj-server-1 | │ nversation/openai/gpt.py:9 │
myaiabnkhoj-server-1 | │ 8 in send_message_to_model │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 95 │ """ │
myaiabnkhoj-server-1 | │ 96 │ │
myaiabnkhoj-server-1 | │ 97 │ # Get Response f │
myaiabnkhoj-server-1 | │ ❱ 98 │ return completio │
myaiabnkhoj-server-1 | │ 99 │ │ messages=mes │
myaiabnkhoj-server-1 | │ 100 │ │ model=model, │
myaiabnkhoj-server-1 | │ 101 │ │ openai_api_k │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/tenacity/i │
myaiabnkhoj-server-1 | │ nit.py:330 in wrappedf │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 327 │ │ │ f, funct │
myaiabnkhoj-server-1 | │ 328 │ │ ) │
myaiabnkhoj-server-1 | │ 329 │ │ def wrapped │
myaiabnkhoj-server-1 | │ ❱ 330 │ │ │ return s │
myaiabnkhoj-server-1 | │ 331 │ │ │
myaiabnkhoj-server-1 | │ 332 │ │ def retry_wi │
myaiabnkhoj-server-1 | │ 333 │ │ │ return s │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/tenacity/i │
myaiabnkhoj-server-1 | │ nit.py:467 in call │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 464 │ │ │
myaiabnkhoj-server-1 | │ 465 │ │ retry_state │
myaiabnkhoj-server-1 | │ 466 │ │ while True: │
myaiabnkhoj-server-1 | │ ❱ 467 │ │ │ do = sel │
myaiabnkhoj-server-1 | │ 468 │ │ │ if isins │
myaiabnkhoj-server-1 | │ 469 │ │ │ │ try: │
myaiabnkhoj-server-1 | │ 470 │ │ │ │ │ │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/tenacity/i │
myaiabnkhoj-server-1 | │ nit.py:368 in iter │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 365 │ │ self.begin │
myaiabnkhoj-server-1 | │ 366 │ │ result = Non │
myaiabnkhoj-server-1 | │ 367 │ │ for action i │
myaiabnkhoj-server-1 | │ ❱ 368 │ │ │ result = │
myaiabnkhoj-server-1 | │ 369 │ │ return resul │
myaiabnkhoj-server-1 | │ 370 │ │
myaiabnkhoj-server-1 | │ 371 │ def _begin_iter( │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/tenacity/i │
myaiabnkhoj-server-1 | │ nit.py:410 in exc_check │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 407 │ │ │ │ fut │
myaiabnkhoj-server-1 | │ 408 │ │ │ │ retr │
myaiabnkhoj-server-1 | │ 409 │ │ │ │ if s │
myaiabnkhoj-server-1 | │ ❱ 410 │ │ │ │ │ │
myaiabnkhoj-server-1 | │ 411 │ │ │ │ rais │
myaiabnkhoj-server-1 | │ 412 │ │ │ │
myaiabnkhoj-server-1 | │ 413 │ │ │ self._ad │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/tenacity/i │
myaiabnkhoj-server-1 | │ nit.py:183 in reraise │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 180 │ │
myaiabnkhoj-server-1 | │ 181 │ def reraise(self │
myaiabnkhoj-server-1 | │ 182 │ │ if self.last │
myaiabnkhoj-server-1 | │ ❱ 183 │ │ │ raise se │
myaiabnkhoj-server-1 | │ 184 │ │ raise self │
myaiabnkhoj-server-1 | │ 185 │ │
myaiabnkhoj-server-1 | │ 186 │ def str(self │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/lib/python3.10/concur │
myaiabnkhoj-server-1 | │ rent/futures/_base.py:451 │
myaiabnkhoj-server-1 | │ in result │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 448 │ │ │ │ if s │
myaiabnkhoj-server-1 | │ 449 │ │ │ │ │ │
myaiabnkhoj-server-1 | │ 450 │ │ │ │ elif │
myaiabnkhoj-server-1 | │ ❱ 451 │ │ │ │ │ │
myaiabnkhoj-server-1 | │ 452 │ │ │ │ │
myaiabnkhoj-server-1 | │ 453 │ │ │ │ self │
myaiabnkhoj-server-1 | │ 454 │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/lib/python3.10/concur │
myaiabnkhoj-server-1 | │ rent/futures/_base.py:403 │
myaiabnkhoj-server-1 | │ in get_result │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 400 │ def get_result │
myaiabnkhoj-server-1 | │ 401 │ │ if self._exc │
myaiabnkhoj-server-1 | │ 402 │ │ │ try: │
myaiabnkhoj-server-1 | │ ❱ 403 │ │ │ │ rais │
myaiabnkhoj-server-1 | │ 404 │ │ │ finally: │
myaiabnkhoj-server-1 | │ 405 │ │ │ │ # Br │
myaiabnkhoj-server-1 | │ 406 │ │ │ │ self │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/tenacity/i │
myaiabnkhoj-server-1 | │ nit.py:470 in call__ │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 467 │ │ │ do = sel │
myaiabnkhoj-server-1 | │ 468 │ │ │ if isins │
myaiabnkhoj-server-1 | │ 469 │ │ │ │ try: │
myaiabnkhoj-server-1 | │ ❱ 470 │ │ │ │ │ │
myaiabnkhoj-server-1 | │ 471 │ │ │ │ exce │
myaiabnkhoj-server-1 | │ 472 │ │ │ │ │ │
myaiabnkhoj-server-1 | │ 473 │ │ │ │ else │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /app/src/khoj/processor/co │
myaiabnkhoj-server-1 | │ nversation/openai/utils.py │
myaiabnkhoj-server-1 | │ :50 in │
myaiabnkhoj-server-1 | │ completion_with_backoff │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 47 │ │
myaiabnkhoj-server-1 | │ 48 │ formatted_messag │
myaiabnkhoj-server-1 | │ in messages] │
myaiabnkhoj-server-1 | │ 49 │ │
myaiabnkhoj-server-1 | │ ❱ 50 │ chat = client.ch │
myaiabnkhoj-server-1 | │ 51 │ │ stream=True, │
myaiabnkhoj-server-1 | │ 52 │ │ messages=for │
myaiabnkhoj-server-1 | │ 53 │ │ model=model, │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/openai/_util │
myaiabnkhoj-server-1 | │ s/_utils.py:277 in wrapper │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 274 │ │ │ │ │ │
myaiabnkhoj-server-1 | │ 275 │ │ │ │ │ │
myaiabnkhoj-server-1 | │ 276 │ │ │ │ rais │
myaiabnkhoj-server-1 | │ ❱ 277 │ │ │ return f │
myaiabnkhoj-server-1 | │ 278 │ │ │
myaiabnkhoj-server-1 | │ 279 │ │ return wrapp │
myaiabnkhoj-server-1 | │ 280 │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/openai/resou │
myaiabnkhoj-server-1 | │ rces/chat/completions.py:6 │
myaiabnkhoj-server-1 | │ 40 in create │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 637 │ │ extra_body: │
myaiabnkhoj-server-1 | │ 638 │ │ timeout: fl │
myaiabnkhoj-server-1 | │ 639 │ ) -> ChatComple │
myaiabnkhoj-server-1 | │ ❱ 640 │ │ return self │
myaiabnkhoj-server-1 | │ 641 │ │ │ "/chat/ │
myaiabnkhoj-server-1 | │ 642 │ │ │ body=ma │
myaiabnkhoj-server-1 | │ 643 │ │ │ │ { │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/openai/_base │
myaiabnkhoj-server-1 | │ _client.py:1250 in post │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 1247 │ │ opts = Fina │
myaiabnkhoj-server-1 | │ 1248 │ │ │ method= │
myaiabnkhoj-server-1 | │ **options │
myaiabnkhoj-server-1 | │ 1249 │ │ ) │
myaiabnkhoj-server-1 | │ ❱ 1250 │ │ return cast │
myaiabnkhoj-server-1 | │ stream_cls=stream_c │
myaiabnkhoj-server-1 | │ 1251 │ │
myaiabnkhoj-server-1 | │ 1252 │ def patch( │
myaiabnkhoj-server-1 | │ 1253 │ │ self, │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/openai/_base │
myaiabnkhoj-server-1 | │ _client.py:931 in request │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 928 │ │ stream: boo │
myaiabnkhoj-server-1 | │ 929 │ │ stream_cls: │
myaiabnkhoj-server-1 | │ 930 │ ) -> ResponseT │
myaiabnkhoj-server-1 | │ ❱ 931 │ │ return self │
myaiabnkhoj-server-1 | │ 932 │ │ │ cast_to │
myaiabnkhoj-server-1 | │ 933 │ │ │ options │
myaiabnkhoj-server-1 | │ 934 │ │ │ stream= │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/openai/_base │
myaiabnkhoj-server-1 | │ _client.py:1030 in │
myaiabnkhoj-server-1 | │ _request │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 1027 │ │ │ │ err │
myaiabnkhoj-server-1 | │ 1028 │ │ │ │
myaiabnkhoj-server-1 | │ 1029 │ │ │ log.deb │
myaiabnkhoj-server-1 | │ ❱ 1030 │ │ │ raise s │
myaiabnkhoj-server-1 | │ 1031 │ │ │
myaiabnkhoj-server-1 | │ 1032 │ │ return self │
myaiabnkhoj-server-1 | │ 1033 │ │ │ cast_to │
myaiabnkhoj-server-1 | ╰────────────────────────────╯
myaiabnkhoj-server-1 | BadRequestError: Error code:
myaiabnkhoj-server-1 | 400 - {'error': {'message':
myaiabnkhoj-server-1 | "litellm.BadRequestError:
myaiabnkhoj-server-1 | BadRequestError: GroqException
myaiabnkhoj-server-1 | - Error code: 400 - {'error':
myaiabnkhoj-server-1 | {'message': 'response_format`
myaiabnkhoj-server-1 | does not support streaming',
myaiabnkhoj-server-1 | 'type':
myaiabnkhoj-server-1 | 'invalid_request_error'}}",
myaiabnkhoj-server-1 | 'type': None, 'param': None,
myaiabnkhoj-server-1 | 'code': 400}}
myaiabnkhoj-server-1 | [01:05:19.184406] DEBUG uvicorn.error: = connection is protocol.py:1227 myaiabnkhoj-server-1 | CLOSING
myaiabnkhoj-server-1 | [01:05:19.185674] DEBUG uvicorn.error: > CLOSE 1000 (OK) [2 protocol.py:1178 myaiabnkhoj-server-1 | bytes]
myaiabnkhoj-server-1 | [01:05:19.186874] DEBUG uvicorn.error: = connection is protocol.py:1497 myaiabnkhoj-server-1 | CLOSED
myaiabnkhoj-server-1 | [01:05:19.188183] DEBUG uvicorn.error: % sending keepalive protocol.py:1262 myaiabnkhoj-server-1 | ping
myaiabnkhoj-server-1 | [01:05:19.189827] DEBUG uvicorn.error: ! failing connection protocol.py:1412 myaiabnkhoj-server-1 | with code 1006
myaiabnkhoj-server-1 | [01:05:19.191496] ERROR uvicorn.error: closing handshake failed server.py:248 myaiabnkhoj-server-1 | ╭─ Traceback (most recent call last) ─╮
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/dist-pack │
myaiabnkhoj-server-1 | │ ages/websockets/legacy/server.py:24 │
myaiabnkhoj-server-1 | │ 4 in handler │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 241 │ │ │ │ raise │
myaiabnkhoj-server-1 | │ 242 │ │ │ │
myaiabnkhoj-server-1 | │ 243 │ │ │ try: │
myaiabnkhoj-server-1 | │ ❱ 244 │ │ │ │ await self.c │
myaiabnkhoj-server-1 | │ 245 │ │ │ except Connectio │
myaiabnkhoj-server-1 | │ 246 │ │ │ │ raise │
myaiabnkhoj-server-1 | │ 247 │ │ │ except Exception │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/dist-pack │
myaiabnkhoj-server-1 | │ ages/websockets/legacy/protocol.py: │
myaiabnkhoj-server-1 | │ 770 in close │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 767 │ │ """ │
myaiabnkhoj-server-1 | │ 768 │ │ try: │
myaiabnkhoj-server-1 | │ 769 │ │ │ async with async │
myaiabnkhoj-server-1 | │ ❱ 770 │ │ │ │ await self.w │
myaiabnkhoj-server-1 | │ 771 │ │ except asyncio.Timeo │
myaiabnkhoj-server-1 | │ 772 │ │ │ # If the close f │
myaiabnkhoj-server-1 | │ 773 │ │ │ # are full, the │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/dist-pack │
myaiabnkhoj-server-1 | │ ages/websockets/legacy/protocol.py: │
myaiabnkhoj-server-1 | │ 1236 in write_close_frame │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 1233 │ │ │ │ data = close │
myaiabnkhoj-server-1 | │ 1234 │ │ │ │
myaiabnkhoj-server-1 | │ 1235 │ │ │ # 7.1.2. Start t │
myaiabnkhoj-server-1 | │ ❱ 1236 │ │ │ await self.write │
myaiabnkhoj-server-1 | │ 1237 │ │
myaiabnkhoj-server-1 | │ 1238 │ async def keepalive_ping │
myaiabnkhoj-server-1 | │ 1239 │ │ """ │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/dist-pack │
myaiabnkhoj-server-1 | │ ages/websockets/legacy/protocol.py: │
myaiabnkhoj-server-1 | │ 1209 in write_frame │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 1206 │ │ │ │ f"Cannot wri │
myaiabnkhoj-server-1 | │ 1207 │ │ │ ) │
myaiabnkhoj-server-1 | │ 1208 │ │ self.write_frame_syn │
myaiabnkhoj-server-1 | │ ❱ 1209 │ │ await self.drain() │
myaiabnkhoj-server-1 | │ 1210 │ │
myaiabnkhoj-server-1 | │ 1211 │ async def write_close_fr │
myaiabnkhoj-server-1 | │ 1212 │ │ self, close: Close, │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/dist-pack │
myaiabnkhoj-server-1 | │ ages/websockets/legacy/protocol.py: │
myaiabnkhoj-server-1 | │ 1198 in drain │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 1195 │ │ │ self.fail_connec │
myaiabnkhoj-server-1 | │ 1196 │ │ │ # Wait until the │
myaiabnkhoj-server-1 | │ 1197 │ │ │ # with the corre │
myaiabnkhoj-server-1 | │ ❱ 1198 │ │ │ await self.ensur │
myaiabnkhoj-server-1 | │ 1199 │ │
myaiabnkhoj-server-1 | │ 1200 │ async def write_frame( │
myaiabnkhoj-server-1 | │ 1201 │ │ self, fin: bool, opc │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/dist-pack │
myaiabnkhoj-server-1 | │ ages/websockets/legacy/protocol.py: │
myaiabnkhoj-server-1 | │ 939 in ensure_open │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 936 │ │ │ │ return │
myaiabnkhoj-server-1 | │ 937 │ │ │
myaiabnkhoj-server-1 | │ 938 │ │ if self.state is Sta │
myaiabnkhoj-server-1 | │ ❱ 939 │ │ │ raise self.conne │
myaiabnkhoj-server-1 | │ 940 │ │ │
myaiabnkhoj-server-1 | │ 941 │ │ if self.state is Sta │
myaiabnkhoj-server-1 | │ 942 │ │ │ # If we started │
myaiabnkhoj-server-1 | ╰─────────────────────────────────────╯
myaiabnkhoj-server-1 | ConnectionClosedError: sent 1000 (OK);
myaiabnkhoj-server-1 | no close frame received
myaiabnkhoj-server-1 | [01:05:19.537140] INFO uvicorn.error: connection closed server.py:264 myaiabnkhoj-server-1 | [01:05:19.538533] DEBUG uvicorn.error: x half-closing TCP protocol.py:1319 myaiabnkhoj-server-1 | connection
myaiabnkhoj-database-1 | 2024-06-27 01:05:25.934 UTC [28] LOG: checkpoint starting: time myaiabnkhoj-database-1 | 2024-06-27 01:05:27.270 UTC [28] LOG: checkpoint complete: wrote 13 buffers (0.1%); 0 WAL file(s) added, 0 removed, 0 recycled; write=1.038 s, sync=0.145 s, total=1.336 s; sync files=12, longest=0.113 s, average=0.012 s; distance=15 kB, estimate=15 kB

Ok I tried to wrap that LLAMA3 API (via GROQ) behind LiteLLM as well to ensure it follows the OpenAi format. The error is the same. @sabaimran

stevennt commented 4 months ago

and here is the log from litellm backend: litellm-1 | 01:05:17 - LiteLLM Router:ERROR: router.py:2313 - An exception occurred - litellm.BadRequestError: BadRequestError: GroqException - Error code: 400 - {'error': {'message': 'response_formatdoes not support streaming', 'type': 'invalid_request_error'}} LiteLLM Retried: 1 times, LiteLLM Max Retries: 2 litellm-1 | 01:05:17 - LiteLLM Proxy:ERROR: proxy_server.py:3141 - litellm.proxy.proxy_server.chat_completion(): Exception occured - litellm.BadRequestError: BadRequestError: GroqException - Error code: 400 - {'error': {'message': 'response_format does not support streaming', 'type': 'invalid_request_error'}} LiteLLM Retried: 1 times, LiteLLM Max Retries: 2 litellm-1 | Traceback (most recent call last): litellm-1 | File "/usr/local/lib/python3.11/site-packages/litellm/llms/openai.py", line 968, in async_streaming litellm-1 | response = await openai_aclient.chat.completions.create( litellm-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ litellm-1 | File "/usr/local/lib/python3.11/site-packages/openai/resources/chat/completions.py", line 1181, in create litellm-1 | return await self._post( litellm-1 | ^^^^^^^^^^^^^^^^^ litellm-1 | File "/usr/local/lib/python3.11/site-packages/openai/_base_client.py", line 1790, in post litellm-1 | return await self.request(cast_to, opts, stream=stream, stream_cls=stream_cls) litellm-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ litellm-1 | File "/usr/local/lib/python3.11/site-packages/openai/_base_client.py", line 1493, in request litellm-1 | return await self._request( litellm-1 | ^^^^^^^^^^^^^^^^^^^^ litellm-1 | File "/usr/local/lib/python3.11/site-packages/openai/_base_client.py", line 1584, in _request litellm-1 | raise self._make_status_error_from_response(err.response) from None litellm-1 | openai.BadRequestError: Error code: 400 - {'error': {'message': 'response_formatdoes not support streaming', 'type': 'invalid_request_error'}} litellm-1 | litellm-1 | During handling of the above exception, another exception occurred: litellm-1 | litellm-1 | Traceback (most recent call last): litellm-1 | File "/usr/local/lib/python3.11/site-packages/litellm/main.py", line 379, in acompletion litellm-1 | response = await init_response litellm-1 | ^^^^^^^^^^^^^^^^^^^ litellm-1 | File "/usr/local/lib/python3.11/site-packages/litellm/llms/openai.py", line 991, in async_streaming litellm-1 | raise OpenAIError(status_code=e.status_code, message=str(e)) litellm-1 | litellm.llms.openai.OpenAIError: Error code: 400 - {'error': {'message': 'response_format does not support streaming', 'type': 'invalid_request_error'}} litellm-1 | litellm-1 | During handling of the above exception, another exception occurred: litellm-1 | litellm-1 | Traceback (most recent call last): litellm-1 | File "/usr/local/lib/python3.11/site-packages/litellm/proxy/proxy_server.py", line 3043, in chat_completion litellm-1 | responses = await llm_responses litellm-1 | ^^^^^^^^^^^^^^^^^^^ litellm-1 | File "/usr/local/lib/python3.11/site-packages/litellm/router.py", line 643, in acompletion litellm-1 | raise e litellm-1 | File "/usr/local/lib/python3.11/site-packages/litellm/router.py", line 631, in acompletion litellm-1 | response = await self.async_function_with_fallbacks(kwargs) litellm-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ litellm-1 | File "/usr/local/lib/python3.11/site-packages/litellm/router.py", line 2315, in async_function_with_fallbacks litellm-1 | raise original_exception litellm-1 | File "/usr/local/lib/python3.11/site-packages/litellm/router.py", line 2177, in async_function_with_fallbacks litellm-1 | raise e litellm-1 | File "/usr/local/lib/python3.11/site-packages/litellm/router.py", line 2160, in async_function_with_fallbacks litellm-1 | response = await self.async_function_with_retries(*args, *kwargs) litellm-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ litellm-1 | File "/usr/local/lib/python3.11/site-packages/litellm/router.py", line 2415, in async_function_with_retries litellm-1 | raise original_exception litellm-1 | File "/usr/local/lib/python3.11/site-packages/litellm/router.py", line 2337, in async_function_with_retries litellm-1 | response = await original_function(args, kwargs) litellm-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ litellm-1 | File "/usr/local/lib/python3.11/site-packages/litellm/router.py", line 773, in _acompletion litellm-1 | raise e litellm-1 | File "/usr/local/lib/python3.11/site-packages/litellm/router.py", line 745, in _acompletion litellm-1 | response = await _response litellm-1 | ^^^^^^^^^^^^^^^ litellm-1 | File "/usr/local/lib/python3.11/site-packages/litellm/utils.py", line 1442, in wrapper_async litellm-1 | raise e litellm-1 | File "/usr/local/lib/python3.11/site-packages/litellm/utils.py", line 1270, in wrapper_async litellm-1 | result = await original_function(*args, **kwargs) litellm-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ litellm-1 | File "/usr/local/lib/python3.11/site-packages/litellm/main.py", line 407, in acompletion litellm-1 | raise exception_type( litellm-1 | ^^^^^^^^^^^^^^^ litellm-1 | File "/usr/local/lib/python3.11/site-packages/litellm/utils.py", line 7503, in exception_type litellm-1 | raise e litellm-1 | File "/usr/local/lib/python3.11/site-packages/litellm/utils.py", line 5818, in exception_type litellm-1 | raise BadRequestError( litellm-1 | litellm.exceptions.BadRequestError: litellm.BadRequestError: BadRequestError: GroqException - Error code: 400 - {'error': {'message': 'response_format` does not support streaming', 'type': 'invalid_request_error'}} LiteLLM Retried: 1 times, LiteLLM Max Retries: 2 litellm-1 | litellm-1 | 01:05:17 - LiteLLM Proxy:ERROR: _common.py:120 - Giving up chat_completion(...) after 1 tries (litellm.proxy._types.ProxyException) litellm-1 | INFO: 172.19.0.11:60098 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request

@sabaimran

stevennt commented 4 months ago

Could it be something with the streaming settings under the /notes mode?

stevennt commented 4 months ago

@sabaimran @debanjum Not sure if this is just for me or any crucial step I'm missing, but so far: Only OpenAPI endpoint works when talking to notes. All other OpenAI-compatible endpoints fail when talking to notes and only work in /general mode.

I have tried:

GROQ API (llama3-70b-8192 model)
LLAMA3 served from LiteLLM
NousResearch/Hermes-2-Pro-Mistral-7B-GGUF

So far, this part looks the most suspicious in the log:

Btw, the backend has a myriad of configuration options (good job to Khoj for making it so configurable), but the backend documentation is not catching up with that :). Would be helpful to add them!

sabaimran commented 1 month ago

@stevennt , sorry for the long gap in responding to this! I see why this is happening. It's because the OpenAI API Compatible server you're using (in this case, via LiteLLM or Groq) fails when it sees a parameter it doesn't recognize. Specifically, it seems to be getting triggered by response_format, which allows is to enforce the JSON scheme with OpenAI.

I don't seem to be running into this error with any of the OpenAI API compatible servers we're using. For LiteLLM, I think you should be able to configure the setting for drop params to change the behavior of this failure.

stevennt commented 1 month ago

Hey @sabaimran excellent. It was indeed that drop_params that solved the problem. Khoj seems to work now with the LiteLLM & Groq combination. Thanks.

khoj-ai / khoj