tts homeassistant - queuelist

[JulienNabaztag]

I use tts service from home assistant like that :

/config/automations.yaml

• alias: The rabbit wants to talk (fr-fr) trigger: platform: mqtt topic: nabaztag/sensor/box-nabaztag/talk-fr action: service: tts.cloud_say entity_id: media_player.mopidy data: message: '{{ trigger.payload }}' language: fr-CH options: gender: female

basically I do a mqtt request with text and homeassistant turn that into sound and stream it to my mopidy server file is like that : http://192.168.1.141:8123/api/tts_proxy/c22318c911181a4b7285fe9811235cd18ab366f6_fr-ch_58612170f0_cloud.audiooutput.mp3

it's great for most cases. But I'm trying to converse with chat gpt and in order to increase the delay of treatment I would like to split the response into multiple sentences and do an mqtt query for each sentence. In this way during the reading of the first sentence, the 2nd is being processed etc. I can save a few seconds this way.

If I try to do that with the mopidy integration it doesn't work, indeed as soon as the sound is ready it is played, and if the previous sound is not finished it stops it to replace it.

I would like some sort of queuelist option. I don't feel like it's feasible.

[bushvin]

Hi,

Been thinking this over, and I believe it is by design. Mind you I haven't dug into the tts integration, but what you explain makes sense.

As a possible workaround, I could create a set of 'queue mode' services which enable/disable queue mode. Problem is that it would not be persistent through a restart of ha... So you would need to enable it at the start of your ChapGPT session, and possibly disable it at the end.

[JulienNabaztag]

oh that would be perfect, I think for such use I will activate it when I need it and deactivate it right after.

I have already created an extension that notifies me when a sound playback is finished. This allows me to restart the conversation mode on a loop until no one is talking.

import logging
import json
import requests
import pykka
from mopidy import ext, core
# https://docs.mopidy.com/en/release-0.19/_modules/mopidy/core/listener/

logger = logging.getLogger(__name__)

class NabaztagListener(pykka.ThreadingActor, core.CoreListener):

    def __init__(self, config, core):
        super(NabaztagListener, self).__init__()
        self.core = core
        self.base_url = config["nabaztag"]["base_url"]

    def playback_state_changed(self, old_state, new_state): 
        event = 'playback_state_changed'  
        data = {'event': event, 'from': old_state, 'to': new_state}
        headers = {'Content-Type': 'application/json'}
        response = requests.post(self.base_url, data=json.dumps(data), headers=headers)
        if response.status_code == 200:
            logger.info(f'Request sent for event: {event}')
        else:
            logger.warning(f'Failed to send request for event: {event}')        

class NabaztagExtension(ext.Extension):

    def setup(self, registry):
        core = registry.get_core()
        listener = NabaztagListener()
        #core.tracklist.subscribe(listener)
        core.playback.subscribe(listener)

Demo: https://youtube.com/shorts/X_JdFFoGgvU

• Speech-to-Text from google (autodetection of the language RO or FR)
• Wled via api json http
• gpt-3.5-turbo (openai)
• service: tts.cloud_say from homeassistant (from mqtt request)
• mopidy for playing the answer
• mopidy extension in order to stop the movement of the ears when the sound is finished

A simple solution for me, but it would then not depend on the extension and not on mopidy either, will be to recover the precise moment of the creation of the file which is here:

/usr/share/hassio/homeassistant/tts/c22318c911181a4b7285fe9811235cd18ab366f6_fr-ch_58612170f0_cloud.audiooutput.mp3

in this case I could play the sound with anything and create my own queuelist system. Maybe by creating a "fake" tts_media which would make a mqtt request/or http request... Hum... that could be an idea for a project ^^

[bushvin]

@JulienNabaztag I don't have time to test it now, but if you want, I have pushed updated code to the feature/queue_mode branch. Check it out if you want to test.

[bushvin]

Hi @JulienNabaztag

I am working on a v2 release of my integartion and nearing completion, whcih leverages the play_media service's queue functionality.

Feel free to try it out, the updated code can be found in the feature/compat branch.

[bushvin]

Fixed with #37