search

janhq / jan

Jan is an open source alternative to ChatGPT that runs 100% offline on your computer. Multiple engine support (llama.cpp, TensorRT-LLM)
https://jan.ai/
GNU Affero General Public License v3.0
20.95k stars 1.21k forks source link

Discussion: [API model selection] #2600

Closed sfz009900 closed 2 months ago

sfz009900 commented 3 months ago

For example, I want to use the native API for a translation software, but the model selection of the translation software is only gpt3.5 or gpt4, what should I do :)?

Propheticus commented 3 months ago

My expectation: ask the translation software to implement a choice to use (local) Open AI stand-in APIs. Any local app that follows the API specification of Open AI, like Jan, could then work. But: commercial products with a SLA / service commitment will probably limit the options they offer to those that are of a high enough 'production quality' to limit the amount of support cases they get with issues and questions caused by 3rd party dependencies.

imtuyethan commented 2 months ago

For example, I want to use the native API for a translation software, but the model selection of the translation software is only gpt3.5 or gpt4, what should I do :)?

We are moving forward with more inference engines like OpenAI, Groq, Mistral, OpenRouter,... (see docs here: https://jan.ai/docs/remote-inference/openai)