Chat Context Window Token Count / Overall Higher Context Visibility

[jacob-dietle]

Is your feature request related to a problem? Please describe.

Engineering the context inside and across context windows is a huge factor in getting an LLM to produce high quality output (especially code). First, I want to say Cursor has the best context management UX of any LLM product I've seen - so big props. The ability to just throw in a docs url and call it via @ is saving me so much time I previously spent cleaning up docs myself and copy/pasting into the context window. The one downside of that abstraction I have started to come across is that, as a trade off for convenience, you sacrifice visibility which can lead higher hallucination rate, especially near the end of windows. I am finding myself thinking about what parts of the docs are being used, how they're being selected and so on and so forth. I understand this is a hard balance to strike, I created a whole context-based LLM productization framework to try to fix it for myself, but I think an immediate value-add would be better visibility into how much space you've got in a context window and how cursor handles managing context across windows.

Describe the solution you'd like

I do not know how Cursor is handling context across windows/if there is an infinite scrolling window for the chat. But I would like a token count for the immediate window if that makes sense depending on how the backend is handling context. A token count and maybe more visibility/explanation (literally could just be blog post on how exactly the reranker works). I wish I could provide a more in-depth request, but without knowing how cursor is handling passing context across windows/how many windows/what context, etc. I don't think I could suggest anything useful beyond a token count request.

Happy to make a more specific request if given some more insight. Thanks!

[AndrzejPec]

+1 anyone?

[gaboesquivel]

+1

[silverberrydesert]

This is really important for real work guys. Context mechanics shouldn't be obscured away in the name of secret product sauce.

I'm finding myself using Aider more and more (insider Cursor, ironically) because for real repos, the ~ 10k context in Cursor doesn't cut it and I don't have any way to peak or tweak that precious (and small) context window.

While with aider I get full logs for what is sent to the model.

I'm kind of ending up with a workflow where I am:

1) Using Composer to build POCs 2) Using Aider + Cursor for actual repos