Sending .wav files to user at frontend

Hi guys!

I am trying to use the webchat widget as an Ionic (Angular) component such that when the user sends a text, the chatbot replies with a text message and an audio narrating the message (TTS) back to the user.

Right now, the App is able to send and receive text messages from the RASA server. I have implemented a TTS model at the backend to generate .wav files based on the text response from RASA, and would like to send it back to the frontend for the user to listen.

Is it possible to send a .wav file over to the frontend? It will be better if it auto-plays too.

As a workaround attempt, I plan to store the .wav file on a server directory served using a python HTTP server, and make use of onSocketEvent to capture the message uttered by the bot and retrieve the respective .wav file from the server, and then play on the app itself.

However, I am unsure how do I retrieve the response content from the frontend.

Here is my current code on the frontend:

chatbot.ts

export class ChatbotPage {

  constructor(public navCtrl: NavController, public navParams: NavParams) {
  }

  ionViewDidLoad() {

    WebChat.default.init({
      selector: "#webchat",
      initPayload: "hello",
      socketUrl: "http://192.168.1.5:5005",
      socketPath: "/socket.io/",
      inputTextFieldHint: "Type a message...",
      connectingText: "Waiting for server...",
      hideWhenNotConnected: false,
      embedded: true,
      fullScreenMode: true,
      profileAvatar: "http://to.avat.ar",
      openLauncherImage: 'myCustomOpenImage.png',
      closeLauncherImage: 'myCustomCloseImage.png',
      displayUnreadCount: true, // --> [view](./assets/unread_count_pastille.png)
      showMessageDate: true,
      customMessageDelay: (message) => {
        if (message.length > 100) return 2000;
        return 1000;
      },
      onSocketEvent: {
        'bot_uttered': () => console.log("Bot has uttered"),
      },
      params: {
        images: {
          dims: {
            width: 300,
            height: 200,
          }
        },
        storage: "local"
      },
    }

    )

    console.log('ionViewDidLoad ChatbotPage');

  }

}

I am using a custom socketio connector to handle the messages.

I have tried sending the .wav file as a video, using the HTTP link as the src. However, the video auto-plays every time the chatbot is refreshed. I would prefer either an audio clip similar to WhatsApp's voice recording messages, or it auto-plays upon receiving a new response from the bot.

Thanks in advance!

botfront / rasa-webchat

Sending .wav files to user at frontend #158