search

janhq / jan

Jan is an open source alternative to ChatGPT that runs 100% offline on your computer. Multiple engine support (llama.cpp, TensorRT-LLM)
https://jan.ai/
GNU Affero General Public License v3.0
23.74k stars 1.38k forks source link

bug: OpenRouter API max token limit too low in UI (1024) compared to model capability (128k) #4014

Closed imtuyethan closed 1 week ago

imtuyethan commented 1 week ago

Jan version

0.5.7 & 0.5.8

Describe the Bug

https://discord.com/channels/1107178041848909847/1305863466459992115 The OpenRouter API integration in Jan has a UI-enforced token limit of 1024 tokens, which is significantly lower than the actual model capabilities (128k for Qwen model). This artificial limitation prevents users from utilizing the model's full potential.

Current Workaround Manually edit the openrouter model.json file in the /models folder to adjust max_tokens parameter

Steps to Reproduce

  1. Connect to OpenRouter API through Jan
  2. Try to use Qwen model
  3. Check max token setting - limited to 1024 in UI
  4. Attempt to generate longer outputs - gets cut off due to token limit

Screenshots / Logs

-

What is your OS?