[BUG]: can't use openai realtime api

japen0617 commented 3 weeks ago

How are you running AnythingLLM?

Local development

What happened?

I chose the gpt-4o-realtime model in the workspace. Then try to use it, but it shows the following error message Could not respond to message. 404 This is not a chat model and thus not supported in the v1/chat/completions endpoint. Did you mean to use v1/ completions

Are there known steps to reproduce?

No response

timothycarambat commented 3 weeks ago

Correct, dont use realtime as your model - we dont have support for it currently since it cannot handle chat message pairs

lwiart commented 1 week ago

Hello,

Same for me, I just installed AnythingLLM (1.6.9), chose OpenAI and pasted an API Key. But found why :

In the global settings for AnythingLLM : I chose OpenAI and gpt-4o-realtime-preview-....
Created a workspace in which I configured OpenAI and gpt-4o
When I was trying to chat in that workspace, no matter which model I choose, I got "Could not respond to message. 404 This is not a chat model and thus not supported in the v1/chat/completions endpoint. Did you mean to use v1/completions?"
Then I went back to the global settings for AnythingLLM and chose got-4o
Then chat worked in my workspace => it seems that the workspace config for model is not used and it's the global setting that gets used.

Normal ?

PS: other than that, AnythingLLM is awesome!

timothycarambat commented 6 days ago

While realtime is available in the dropdown (we dont filter) support for it needs to still be built out since it works quite differently from the traditional API - that is why it is broken. Realtime is pretty different from the normal API/chunk response.

In the interim, swap to just another model that works in the traditional sense (basically any model that isn't realtime)

Mintplex-Labs / anything-llm