semperai / amica

Amica is an open source interface for interactive communication with 3D characters with voice synthesis and speech recognition.
https://heyamica.com
MIT License
797 stars 125 forks source link

Refact the backend code to make it more modular and allow for easy plugin integration in the future #40

Closed snowyu closed 2 weeks ago

snowyu commented 12 months ago

Currently, there are 3 backend components: chat, text-to-speech (TTS), speech-to-text (STT). Also, a storage backend is needed and the UI could be considered as a special fifth backend component too.

IMO, all interfaces in the future are only auxiliary, and all interfaces should be able to be called by talking to the bot.

snowyu commented 11 months ago

@kasumi-1 Currently the back-end code is split in different directories, so the first step is to put them in the same directory, such as:

chatbot/
   llamacpp/
   openai/
tts/
   coqui/
   openaiTTS/
...

And the UI interface should be changed from the modal dialogue box to the modeless dialogue box: Users can use chat commands to operate the interface at the same time.

kasumi-1 commented 11 months ago

@snowyu agree! But, what do you mean by modeless dialogue box? Like, implementing slash commands or something?

snowyu commented 11 months ago

A modeless dialog box (also known as non-modal) is a type of user interface element that allows users to interact with it without interrupting their current task or session. This means that while the dialog box is open, the rest of the application remains accessible and responsive.

implementing slash commands

Yes, exactly. You can type slash command to open UI or change the config in the UI.
The next step is that the bot should analyze user intent and automatically call slash commands based on those intentions.