Mantella is a Skyrim and Fallout 4 mod which allows you to naturally speak to NPCs using Whisper (speech-to-text), LLMs (text generation), and xVASynth / XTTS (text-to-speech).
Mantella becomes unresponsive after few conversations in Radiant chat mode at: INFO: End of sentences.
Local LLM used served by koboldcpp, multiple models tried.
Mantella Log:
...
...
2024-06-17 20:30:50,193 INFO: Waiting 5 seconds...
2024-06-17 20:30:50,216 Level 23: Full response saved (160 tokens): Stenvar: Good evening, Sigurd. Have you been working hard at Belethor's shop today? Sigurd: Yes, Stenvar. It's been a busy day. I've been chopping and carrying firewood all day. How about you? What have you been up to? Stenvar: I've been doing some mercenary work for a merchant. It's been a tough day, but I'm used to it. Sigurd: That sounds like hard work. I'm glad I don't have to deal with that. Stenvar: Yeah, it's not for everyone. But it pays well, and I enjoy the challenge. Sigurd: I can imagine. Speaking of challenges, have you ever thought about buying a house in Whiterun?
2024-06-17 20:30:50,216 DEBUG: close.started
2024-06-17 20:30:50,216 DEBUG: close.complete
2024-06-17 20:30:55,710 INFO: End of sentences
2024-06-17 20:30:55,710 DEBUG: response_closed.started
2024-06-17 20:30:55,710 DEBUG: receive_response_body.failed exception=GeneratorExit()
2024-06-17 20:30:55,710 DEBUG: response_closed.complete
koboldcpp log:
Processing Prompt [BLAS] (1772 / 1772 tokens)
Generating (250 / 250 tokens)
CtxLimit: 2030/4096, Process:0.80s (0.5ms/T = 2217.77T/s), Generate:6.43s (25.7ms/T = 38.87T/s), Total:7.23s (34.57T/s)
Output: Stenvar: Good evening, Sigurd. Have you been working hard at Belethor's shop today?
Sigurd: Yes, Stenvar. It's been a busy day. I've been chopping and carrying firewood all day. How about you? What have you been up to?
Stenvar: I've been doing some mercenary work for a merchant. It's been a tough day, but I'm used to it.
Sigurd: That sounds like hard work. I'm glad I don't have to deal with that.
Stenvar: Yeah, it's not for everyone. But it pays well, and I enjoy the challenge.
Sigurd: I can imagine. Speaking of challenges, have you ever thought about buying a house in Whiterun?
Stenvar: I have, actually. I've been thinking about it for a while now. I've been saving up some gold, and I'm considering buying one.
Sigurd: That's great, Stenvar. I know Jorleif has a few houses for sale. He's a good guy to talk
So this seems to be an issue with the LLM's output (token limit), and perhaps a solution could be to, on such behavior try/restart again.
Mantella becomes unresponsive after few conversations in Radiant chat mode at: INFO: End of sentences.
Local LLM used served by koboldcpp, multiple models tried.
Mantella Log: ... ... 2024-06-17 20:30:50,193 INFO: Waiting 5 seconds... 2024-06-17 20:30:50,216 Level 23: Full response saved (160 tokens): Stenvar: Good evening, Sigurd. Have you been working hard at Belethor's shop today? Sigurd: Yes, Stenvar. It's been a busy day. I've been chopping and carrying firewood all day. How about you? What have you been up to? Stenvar: I've been doing some mercenary work for a merchant. It's been a tough day, but I'm used to it. Sigurd: That sounds like hard work. I'm glad I don't have to deal with that. Stenvar: Yeah, it's not for everyone. But it pays well, and I enjoy the challenge. Sigurd: I can imagine. Speaking of challenges, have you ever thought about buying a house in Whiterun? 2024-06-17 20:30:50,216 DEBUG: close.started 2024-06-17 20:30:50,216 DEBUG: close.complete 2024-06-17 20:30:55,710 INFO: End of sentences 2024-06-17 20:30:55,710 DEBUG: response_closed.started 2024-06-17 20:30:55,710 DEBUG: receive_response_body.failed exception=GeneratorExit() 2024-06-17 20:30:55,710 DEBUG: response_closed.complete
koboldcpp log: Processing Prompt [BLAS] (1772 / 1772 tokens) Generating (250 / 250 tokens) CtxLimit: 2030/4096, Process:0.80s (0.5ms/T = 2217.77T/s), Generate:6.43s (25.7ms/T = 38.87T/s), Total:7.23s (34.57T/s) Output: Stenvar: Good evening, Sigurd. Have you been working hard at Belethor's shop today? Sigurd: Yes, Stenvar. It's been a busy day. I've been chopping and carrying firewood all day. How about you? What have you been up to? Stenvar: I've been doing some mercenary work for a merchant. It's been a tough day, but I'm used to it. Sigurd: That sounds like hard work. I'm glad I don't have to deal with that. Stenvar: Yeah, it's not for everyone. But it pays well, and I enjoy the challenge. Sigurd: I can imagine. Speaking of challenges, have you ever thought about buying a house in Whiterun? Stenvar: I have, actually. I've been thinking about it for a while now. I've been saving up some gold, and I'm considering buying one. Sigurd: That's great, Stenvar. I know Jorleif has a few houses for sale. He's a good guy to talk
So this seems to be an issue with the LLM's output (token limit), and perhaps a solution could be to, on such behavior try/restart again.