[Android] No TTS with error "The session timed out because the TTS component did not respond [...]"

[Pravez]

Hi ! First of all thank you for your work on this assistant, it is really nice ! :)

What's the issue? When asked a TTS request on android by Dialogue, after approximately 15 seconds without sound the following error appears and the TTS request is cancelled : [Dialogue] session with id 'c33b814b-6b8a-4bce-90f1-6f22faa40912' was ended on site default. The session timed out because the TTS component did not respond in a timely manner. Please ensure that the tts is started and running correctly.

Use Cases Impacted Every single use case is impacted, as I am trying to create a conversational app.

How to reproduce The execution is done on a Samsung mobile with Google TTS selected. The code is the basic demo code with messages in continueSession() and endSession()

Expected behavior That some voice would show up ...

Version and Environment Samsung S8, android sdk version 28

[cpoisson]

Hello @Pravez ,

Did you check if Google TTS is installed on your phone?

If it is installed, it may be worth to have some logs to figure out the issue.

Have a good day!

[Pravez]

Google TTS is installed. I also tried to use it directly through the android API, and it perfectly works.

Here is the entire output of the snips processing service :

snipsProcessingService D/snips-platform: hermes_inprocess: Publishing Dialogue/DialogueContinueSession {
        continue_session: ContinueSessionMessage {
            session_id: "257dd58d-0a9e-4a9a-9264-fd9f30cc3c6f",
            text: "Let\'s make a chocolate cake ! I need ... ",
            intent_filter: Some(
                [
                    "WordAnswer",
                ],
            ),
            custom_data: None,
            send_intent_not_recognized: false,
            slot: None,
        },
    }
snipsProcessingService I/snips-platform: snips_dialogue_lib::coordinator::coordinator: Site [default]: reset site -> turn off ASR and turn off Hotword
snipsProcessingService D/snips-platform: hermes_inprocess: Publishing Asr/AsrStopListening {
        site: SiteMessage {
            site_id: "default",
            session_id: Some(
                "257dd58d-0a9e-4a9a-9264-fd9f30cc3c6f",
            ),
        },
    }
snipsProcessingService D/snips-platform: hermes_inprocess: Publishing Hotword/IdentifiableToggleableToggleOff {
        site: SiteMessage {
            site_id: "default",
            session_id: Some(
                "257dd58d-0a9e-4a9a-9264-fd9f30cc3c6f",
            ),
        },
        component: Hotword,
    }
snipsProcessingService D/snips-platform: hermes_inprocess: Publishing Tts/TtsSay {
        to_say: SayMessage {
            text: "Let\'s make a chocolate cake ! I need ... ",
            lang: Some(
                "en",
            ),
            id: Some(
                "6970b66b-ff06-4d68-bd15-2653b06a42fe",
            ),
            site_id: "default",
            session_id: Some(
                "257dd58d-0a9e-4a9a-9264-fd9f30cc3c6f",
            ),
        },
    }
snipsProcessingService D/snips-platform: snips_megazord: calling self.on_listening_state_changed callback
snipsProcessingService I/snips-platform: snips_megazord: calling tts_handler callback
snipsProcessingService D/snips-platform: snips_megazord: call of self.on_listening_state_changed callback done
snipsProcessingService I/snips-platform: snips_hotword_lib::audio: Audio thread for site default stopped.
snipsProcessingService I/snips-platform: snips_megazord: tts_handler done
snipsProcessingService I/snips-platform: snips_dialogue_lib::coordinator::coordinator: Session [257dd58d-0a9e-4a9a-9264-fd9f30cc3c6f]: closing session which was active: Timeout { component: Some(Tts) }
snipsProcessingService D/snips-platform: hermes_inprocess: Publishing Dialogue/DialogueSessionEnded {
        status: SessionEndedMessage {
            session_id: "257dd58d-0a9e-4a9a-9264-fd9f30cc3c6f",
            custom_data: None,
            termination: Timeout {
                component: Some(
                    Tts,
                ),
            },
            site_id: "default",
        },
    }
snipsProcessingService I/snips-platform: snips_dialogue_lib::coordinator::coordinator: Site [default]: reset site -> turn off ASR and turn on Hotword
snipsProcessingService D/snips-platform: hermes_inprocess: Publishing Asr/AsrStopListening {
        site: SiteMessage {
            site_id: "default",
            session_id: None,
        },
    }
snipsProcessingService D/snips-platform: hermes_inprocess: Publishing Hotword/IdentifiableToggleableToggleOn {
        site: SiteMessage {
            site_id: "default",
            session_id: None,
        },
        component: Hotword,
    }
snipsProcessingService D/snips-platform: snips_megazord: calling self.on_session_ended callback
snipsProcessingService D/snips-platform: snips_megazord: calling self.on_listening_state_changed callback
snipsProcessingService I/snips-platform: snips_hotword_lib::audio: Audio thread for site default started.
snipsProcessingService D/snips-platform: snips_megazord: call of self.on_listening_state_changed callback done
snipsProcessingService D/snips-platform: snips_megazord: call of self.on_session_ended callback done

I am not sure but I cannot see any error here. On the main app process I end up with this :

D/BuddyApplication: Log : [10:36:06] [Dialogue] was ask to continue session with id 257dd58d-0a9e-4a9a-9264-fd9f30cc3c6f by saying 'Let's make a chocolate cake ! I need ... '
D/BuddyApplication: Log : [10:36:06] [Tts] was asked to say "Let's make a chocolate cake ! I need ... "
D/BuddyApplication: Log : [10:36:22] [Dialogue] session with id '257dd58d-0a9e-4a9a-9264-fd9f30cc3c6f' was ended on site default. The session timed out because the TTS component did not respond in a timely manner. Please ensure that the tts is started and running correctly.
D/BuddyApplication: Current status of session : false

Concerning the logs of Google TTS, I could not find any. It seems to correspond with the fact that independantly, TTS is working. Thank you for your greatly appreciated help :)

[fredszaq]

Hi @Pravez !

The default TTS handler for doesn't log anything if it encounter errors... (shame on me) you can however change it quite easilly using the setTtsEngine on the SnipsPlatformClient

here's a quick copy paste of the default one (the sources are available on nexus) with a few more logs

class TtsHandler(private val snipsPlatformClient: SnipsPlatformClient) : TtsEngine {

    private var tts: TextToSpeech? = null
    private val ids = HashMap<String, SayFinishedMessage>()

    override fun setup(context: Context) {
        tts = TextToSpeech(context) {}.apply {
            println("setuping TTS Handler")
            setOnUtteranceProgressListener(object : UtteranceProgressListener() {
                override fun onDone(p0: String?) {
                    println("utterance done $p0")
                    ids.remove(p0)?.let { snipsPlatformClient.notifySayFinished(it) }
                }

                override fun onError(p0: String?) {
                    println("utterance error $p0")
                }

                override fun onStart(p0: String?) {
                    println("utterance start $p0")
                }
            })
        }
    }

    override fun tearDown() {
        println("tear down tts engine")
        tts?.let {
            it.setOnUtteranceProgressListener(null)
            it.shutdown()
        }
        tts = null
        ids.clear()
    }

    override fun say(message: SayMessage) {
        println("saying $message")
        tts?.let { tts ->
            message.lang?.let { tts.language = Locale.forLanguageTag(it) }
            val id = UUID.randomUUID().toString()

            ids[id] = SayFinishedMessage(message.id, message.sessionId)

            tts.speak(message.text, TextToSpeech.QUEUE_ADD, null, id)
        }
    }

}

could you try and set something like that as a TTS handler ? should give us a better idea of where the problem really is

[Pravez]

It now works nicely. I also had a problem of dialogue not continuing, but it now continues flawlessly. There's just a detail : the callback on snipsPlatformClient, onTtsSayListener is not called anymore ... I suppose it is because we changed the default tts engine. Thank you :)

[fredszaq]

The onTtsSayListener is a shortcut to register a TtsEngine, if you register one it should call the actual TTS engine to generate the sound

[Pravez]

Oh okay, I thought it was an additional callback when the TtsEngine was supposed to be called. Misread documentation ! As my problem is solved I close this issue, thank you for your help !