search

art-from-the-machine / Mantella

Mantella is a Skyrim and Fallout 4 mod which allows you to naturally speak to NPCs using Whisper (speech-to-text), LLMs (text generation), and xVASynth / XTTS (text-to-speech).
https://art-from-the-machine.github.io/Mantella/
GNU Affero General Public License v3.0
170 stars 43 forks source link

(Feature Request) Assistant prefill for Anthropic models #216

Open maeeeeee opened 5 months ago

maeeeeee commented 5 months ago

Per Anthropic docs, Claude models support adding a prefill that allows you to determine the beginning of the model's response. Adding support for this in Mantella would go a huge way in being able to reduce rejections and filtering; in my experience, something as simple as "Certainly, here's my response: " does wonders.

art-from-the-machine commented 5 months ago

Thanks for bringing this up @maeeeeee! It looks like this might be possible to implement across all LLMs so I will aim to include this in the next release