Fix tokens left example

ShipBit / slickgpt

SlickGPT is a light-weight "use-your-own-API-key" web client for the OpenAI API written in Svelte. It offers GPT-4 integration, a userless share feature and other superpowers.

https://slickgpt.vercel.app

MIT License

464 stars 97 forks source link

Fix tokens left example #97

Closed xmoiduts closed 5 months ago

xmoiduts commented 5 months ago

[Do not directly merge, it contains my diverged modification]

The [tokens left] part shown at the left bottom of the chat session is not correct. Simply changing OpenAiModelStats.maxTokens will and will not work.

Will work: it indeed corrects the [Tokens left] number;
Will not work: it breaks the settings -> advanced -> maxTokens upper limit and makes it really hard to locate precise settings < 4096 (models reply tokens) with mice dragging.

The issue comes from multiplexing of the concept "max tokens that a model can reply (often 4096 now)" and "the max context tokens applicable by a model (16k, 128k, ~~and even 200k (claude)~~)".

This commit aims at splitting them.

vercel[bot] commented 5 months ago

@xmoiduts is attempting to deploy a commit to the ShipBit Team on Vercel.

A member of the Team first needs to authorize it.

Shackless commented 5 months ago

The SlickGPT implementation refers to an older and outdated way how OpenAI specified and calculated model windows. It was once correct but isn't anymore. Thanks for fixing this, it has been on my list for a while but I didn't get to it.

If you remove your mods, I can merge your PR so that you're listed as a contributor. Or I can take your changes and create another PR - as you wish.

vercel[bot] commented 5 months ago

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
slickgpt	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Apr 9, 2024 10:11pm

Shackless commented 5 months ago

closed in favor of your other PR that was already merged. Thank you!