It changes the numbers beside each conversation to reflect the current conversation history token count. The previous behavior was that it was reflecting the entire historical token usage (even including when someone might have used 'summarize' to reduce it, or deleted messages) in the course of that conversation. I think most people looking at that number would expect/desire it to show the current effective token count of the conversation though, so that they could know when to expect to hit the 4000 token window and have the AI start to lose memory of earlier bits of a conversation.
When a user uses 'summarize' it takes that into account and reflects the new shortened token history length for the conversation immediately.
The total tokens spent is still tracked properly (the token count in the settings dialog).
This makes two changes:
The total tokens spent is still tracked properly (the token count in the settings dialog).