Fallout-specific changes allowing game to handle audio, lip-sync and subtitles natively

art-from-the-machine / Mantella

Mantella is a Skyrim and Fallout 4 mod which allows you to naturally speak to NPCs using Whisper (speech-to-text), LLMs (text generation), and xVASynth / XTTS (text-to-speech).

https://art-from-the-machine.github.io/Mantella/

GNU Affero General Public License v3.0

169 stars 42 forks source link

Fallout-specific changes allowing game to handle audio, lip-sync and subtitles natively #332

Closed peteben closed 1 month ago

peteben commented 1 month ago

Fallout-specific changes allowing game to handle audio, lip-sync and subtitles natively Fixed typos -Added missing parentheses to player.equipment() in context.py -Added missing '\' to regex in utils.py Fallout specific:

Compatibility for new method of playing audio, lip-sync and subtitles
Added copying of wav and lip files to new Mantella-specific voice directories
Eliminated all code for fallout-specific playing audio, game is doing it now.
Eliminated code to create all voice directories, only 8 needed now.

Leidtier commented 1 month ago

Nice work! If the game is able to play the sounds, this will be a nice improvement over the current state.

Short note about the player.equipment thing. I tackled this here as well #330. This was meant to be a property and I think the @property tag was lost when the equipment stuff was merged in.

peteben commented 1 month ago

There is still a problem with the game engine caching wav files, so withdrawing until I can fix or supply an alternative.