art-from-the-machine / Mantella

Mantella is a Skyrim and Fallout 4 mod which allows you to naturally speak to NPCs using Whisper (speech-to-text), LLMs (text generation), and xVASynth / XTTS (text-to-speech).
https://art-from-the-machine.github.io/Mantella/
GNU General Public License v3.0
152 stars 37 forks source link

Allow player to interrupt conversation with hotkey #258

Open art-from-the-machine opened 2 months ago

art-from-the-machine commented 2 months ago

This section of code in output_manager.py allows the player to "interrupt" a radiant conversation and join in (turning it into a multi-NPC conversation):

https://github.com/art-from-the-machine/Mantella/blob/94fdfba52e5be60ded5a8e7a96d3a63bc1ae560f/src/output_manager.py#L648C1-L653C46

A new line of communication can be created which indicates whether the player is attempting to interrupt the LLM's response at any point (and not just radiant conversations), and if so, end the response as shown in the code above. For example:

end_conversation = self.game_state_manager.load_data_when_available('_mantella_end_conversation', '')
radiant_dialogue_update = self.game_state_manager.load_data_when_available('_mantella_radiant_dialogue', '')
player_interrupt = self.game_state_manager.load_data_when_available('_mantella_player_interrupt', '')
# stop processing LLM response if:
# max_response_sentences reached (and the conversation isn't radiant)
# conversation has switched from radiant to multi NPC (this allows the player to "interrupt" radiant dialogue and include themselves in the conversation)
# the conversation has ended
if ((num_sentences >= self.max_response_sentences) and (radiant_dialogue == False)) or ((radiant_dialogue == True) and (radiant_dialogue_update.lower() == 'false')) or (end_conversation.lower() == 'true') or (player_interrupt.lower == 'true'):
    break

Note that this line of communication would also need to be added to the Mantella Spell. The method of communication will change once this HTTP PR is merged: https://github.com/art-from-the-machine/Mantella/pull/205. It is better to wait for this HTTP PR to be merged before working on this task to avoid rework.

Leidtier commented 2 months ago

In the HTTP branch, the sentences are produced async and put into a queue. Every request to continue from the game simply returns the next sentence from that queue. Interrupting using HTTP should be as simple as:

The hardest part in all of this is to decide hoe to implement that on the games side, especially with VR

DaymareOn commented 2 weeks ago

As a player, and as a Dragonborn, interrupting a NPC speaking inane opinions like some are wont to do, and giving them right now a piece of my mind, is an essential bit of roleplay :P