search

aws-samples / amazon-sumerian-hosts

Amazon Sumerian Hosts (Hosts) is an experimental open source project that aims to make it easy to create interactive animated 3D characters for Babylon.js, three.js, and other web 3D frameworks. It leverages AWS services including Amazon Polly (text-to-speech) and Amazon Lex (chatbot).
MIT No Attribution
187 stars 82 forks source link

How to sync Lex response with gesture to Sumerian Host #33

Closed KISSMonX closed 3 years ago

KISSMonX commented 3 years ago

We are working on BabylonJS that integrates Sumerian Host. By calling Lex we received audio and message, how to sync Lex respond message with Sumerian host so that Host can play both audio and gesture. This could leave out the Polly and only need Lex. Is this possible? Thanks.

c-morten commented 3 years ago

Hi @KISSMonX. We do not currently have the capability of providing audio to the TextToSpeechFeature. This is because a speechmark stream is required as well and must be generated from the same source as the audio in order for the messages to sync up and allow the lipsync and gestures to happen in time with the audio. Unfortunately Lex is not capable of generating speechmark data so there's really no way around using Polly if you want lipsync and gestures. You could however execute TextToSpeechFeature.play using the response message you receive from Lex. Hope this helps.