aws-samples / amazon-sumerian-hosts

Amazon Sumerian Hosts (Hosts) is an experimental open source project that aims to make it easy to create interactive animated 3D characters for Babylon.js, three.js, and other web 3D frameworks. It leverages AWS services including Amazon Polly (text-to-speech) and Amazon Lex (chatbot).
MIT No Attribution
183 stars 82 forks source link

Scene rendering stops upon speech completion #30

Closed M-Anne-M closed 3 years ago

M-Anne-M commented 3 years ago

Once any host completes it's speech the entire Babylon scene just quits rendering and hold a still image on the screen. It throws this error and then points to a variety of places in the different js files for the speech components: Speech.js?2a41:128 Uncaught TypeError: Cannot set property currentTime of # which has only a getter

It does however keep playing speech when you click the start button, only the visuals are stopped from working. It started doing that around three weeks ago on my computer out of the blue. Essentially without changing a single thing in the code it just quit working overnight. Using an old version of the repo (the one where the poi feature was just released) does still work, but the newest version of it does not. I tried completely re-downloading and installing the repo several times, and get the same error each time. Any help is appreciated, thanks!

jkeys-ecg-nmsu commented 3 years ago

@c-morten I can provide the actual logs over email if they would help.

Stack trace: image

Specifically, it errors on this line in Speech.js:

image

jkeys-ecg-nmsu commented 3 years ago

Also, this bug was replicated on 1.3.2 of this repo, and babylonjs@4.2.0-beta.21.

When I downgraded to babylonjs@4.2.0-beta.9, the error went away.

c-morten commented 3 years ago

^ Thanks @jkeys-ecg-nmsu, this is really helpful info. I'll take a look at Babylon's latest preview build later this evening and see what has changed in the audio system.

KISSMonX commented 3 years ago

Also, this bug was replicated on 1.3.2 of this repo, and babylonjs@4.2.0-beta.21.

When I downgraded to babylonjs@4.2.0-beta.9, the error went away.

This works! thank you very much. Look forward to further updates.

KISSMonX commented 3 years ago

^ Thanks @jkeys-ecg-nmsu, this is really helpful info. I'll take a look at Babylon's latest preview build later this evening and see what has changed in the audio system.

Looks like BabylonJS preview version has compatibility issues with Sumerian. Hope the new preview version of BabylonJS will fix this.

DC2009 commented 3 years ago

I temporary solved the problem by bypassing the library Speech stop. I think it's enough to update the babylon function by removing the line highlighted in https://github.com/aws-samples/amazon-sumerian-hosts/issues/30#issuecomment-720593374

import HOST from "amazon-sumerian-hosts/dist/host.babylon";

/**
 * @extends babylon/Speech
 */
class Speech extends HOST.aws.Speech {
  /**
   * @constructor
   *
   * @param {Babylon.js/TextToSpeechFeature} speaker - The owner of the Speech
   * that will emit speechmark messages.
   * @param {string} text - The text of the speech.
   * @param {Array.<Object>} [speechmarks=[]] - An array of speechmark objects representing
   * the text and timing of the speech.
   * @param {Object} audioConfig - Object containing audio and url.
   * @param {external:Audio} audioConfig.audio - Playable audio object.
   */

  stop() {
    this._audio.stop();
    this._pauseAudio();

    if (this._promise) {
      this._promise.resolve();
      this._promise = null;
    }

    this._playing = false;
  }
}

export default Speech;
c-morten commented 3 years ago

Fixed in v1.3.3