Using the ChatCompletion method #31, keyword tokenization #25 doesn't really work because each element of the conversation is divided into constituent parts, rendering frequency counters less useful.
Instead, we should add a forget_old_context bool config that tells gptty.context:get_context to drop the oldest elements of a conversation when approaching the max_context_length.
Honestly, we can just make this the default behavior (unless we want a config for this) be reversing the order in which we read lines in gptty.context:get_config...
Using the ChatCompletion method #31, keyword tokenization #25 doesn't really work because each element of the conversation is divided into constituent parts, rendering frequency counters less useful.
Instead, we should add a
forget_old_context
bool config that tells gptty.context:get_context to drop the oldest elements of a conversation when approaching themax_context_length
.