julian-shalaby
commented
3 days ago Once upon a time this actually was a feature. As the sole person developing this, I came to the conclusion that the development management/overhead ended up being too heavy for the benefits the feature brought, so I removed it. With models trending towards having larger context windows and becoming cheaper, and messages typically not using many tokens in the first place (besides some edge cases, or poorly managed context windows), I found that strict token monitoring is become increasingly less relevant over time.
Unless this becomes a commonly requested feature, I will be leaving this out
It would be nice if ChatWindow displayed a token count for the currently selected context. For example, as you select and deselect files, the token count would change accordingly. This would provide more visibility into context window usage, compared with the current trial & error approach.
Also, it would be nice if you could enter a limit for the context window size and ChatWindow would warn you once you exceed this limit. When working with APIs that support setting the context window size (Ollama), this value could also be used for that purpose. Thus, it would both set the context window size on the server and warn you when the selected files exceed the specified limit.