janhq / jan

Jan is an open source alternative to ChatGPT that runs 100% offline on your computer. Multiple engine support (llama.cpp, TensorRT-LLM)
https://jan.ai/
GNU Affero General Public License v3.0
22.52k stars 1.3k forks source link

feat: Use OpenRouter Default Model instead of OpenRouter/auto #3110

Closed RONNCC closed 1 month ago

RONNCC commented 3 months ago

Is your feature request related to a problem? Please describe it

Currently Jan defaults to using "openrouter/auto" which costs money.

Describe the solution

It may be better if Jan instead does not pass a model parameter.

This would make OpenRouter fallback to the default model chosen in settings which could be a free model image or it also could also be set to openrouter/auto which would be the same as the current functionality.

Teachability, documentation, adoption, migration strategy

Users will be able to use this the same way as the current OpenRouter model.

What is the motivation / use case for changing the behavior?

So I can use the free models on OpenRouter

RONNCC commented 3 months ago

Change would just require removing "openrouter/auto" in the pertinent transformPayload function

tmobilon commented 3 months ago

Why not show a dropdown so the user can choose between all the available models in Openrouter?

RONNCC commented 3 months ago

@tmobilon Sounds great but seems more complicated to implement?

From my understanding reverting to the DefaultModel is the default behavior of the OpenRouter API if a model string is not explicitly provided in an API call.

image

https://openrouter.ai/docs/requests

RONNCC commented 2 months ago

@hahuyhoang411 wondering if y'all would be open to the above change? I can write up a PR?

ghz commented 2 months ago

+1 for adding a models selection dropdown

josemnbcamacho commented 1 month ago

Why not allow for all the OpenRouter available models? It is possible to list all the available models via API at openrouter.ai/api/v1/models

RONNCC commented 1 month ago

@josemnbcamacho @ghz those all require a lot more work and knowledge to interface with the Jan UI.

Van-QA commented 1 month ago

As of nightly Jan v0.5.2-603, the open router now redirected to default instead of auto