search

baptisteArno / typebot.io

💬 Typebot is a powerful chatbot builder that you can self-host.
https://typebot.io
Other
7.31k stars 2.01k forks source link

OpenAI requires defaults to "gpt-3.5-turbo" with custom provider #1250

Closed mjp0 closed 8 months ago

mjp0 commented 8 months ago

The OpenAI block seems to default to "gpt-3.5-turbo" even when using a custom provider.

The block should probably skip validating the model if base URL has been changed because then it's probable that user may be using OpenAI compatible API endpoint to access other models.

baptisteArno commented 8 months ago

You're right! So what should be the default for custom provider? Is it working if you send the request without a model ID?

mjp0 commented 8 months ago

I think it should default to empty because that will throw an error when executing. I think we can safely assume that if someone is playing with custom provider, they probably know they need to specify the model as well. Only time when models would be same as OpenAI's selection would be when the user uses Azure OpenAI.

Right now, I can't actually use it because if I leave the model id empty, it will default to gpt-3.5-turbo. That also happens if I put anything else in the field. I'm guessing it validates the model id against OpenAI's model list endpoint, so it makes sense. In case you want to replicate what I tried, I tried with Together's OpenAI compatible endpoint and Qwen/Qwen1.5-72B-Chat model id.

baptisteArno commented 8 months ago

Ok, I thought that compatible endpoints would also have the "list models" API endpoint but it seems Together does not.

mjp0 commented 8 months ago

AFAIK, none of them (Together, DeepInfra, OpenRouter, etc.) do, so definitely shouldn't rely on that existing :)

baptisteArno commented 8 months ago

After diving a bit into this. I think that would be preferrable to deprecate the "Custom provider" option and implement a block for each service since the other services also don't have Assistant-related endpoints or Speech-related endpoints etc.. It impacts the UX.

So you need Together AI? Or should I focus on something else first?

mjp0 commented 8 months ago

Sure, you can implement block for each one but you will end up writing one for Together and then copy/pasting it over and over again for others. Together is nice and fast, OpenRouter has more edge case models and DeepInfra is the cheapest ;)

baptisteArno commented 8 months ago

1253 #1254 https://github.com/baptisteArno/typebot.io/issues/1266