Mantella is a Skyrim and Fallout 4 mod which allows you to naturally speak to NPCs using Whisper (speech-to-text), LLMs (text generation), and xVASynth / XTTS (text-to-speech).
Token limit calculations have been repainted in a couple of ways:
token_limit_percent has been renamed in main.py and context.py to starting_prompt_token_limit_percent to better reflect what the percentage is calculated against. This percentage has been bumped from 45% to 50%. ie at most, the starting prompt (including the NPC bio and long term memories) can account for up to 50% of a conversation's token limit
reload_conversation() in conversation.py is no longer triggered by percentage calculations. Instead, it is triggered when the conversation so far (including the starting prompt, NPC bio, and long term memories) + max_response_tokens + 250 tokens exceeds the given LLM's token limit. max_response_tokens is added because the calculation needs to predict whether the next response will exceed the token limit, not the current response. 250 tokens are added to act as a safety buffer, accounting for the player's input as well as the slight differences in token calculations across various LLMs
Token limit calculations have been repainted in a couple of ways:
token_limit_percenthas been renamed in main.py and context.py tostarting_prompt_token_limit_percentto better reflect what the percentage is calculated against. This percentage has been bumped from 45% to 50%. ie at most, the starting prompt (including the NPC bio and long term memories) can account for up to 50% of a conversation's token limitreload_conversation()in conversation.py is no longer triggered by percentage calculations. Instead, it is triggered when the conversation so far (including the starting prompt, NPC bio, and long term memories) +max_response_tokens+ 250 tokens exceeds the given LLM's token limit.max_response_tokensis added because the calculation needs to predict whether the next response will exceed the token limit, not the current response. 250 tokens are added to act as a safety buffer, accounting for the player's input as well as the slight differences in token calculations across various LLMs