jovotech / jovo-framework

🔈 The React for Voice and Chat: Build Apps for Alexa, Messenger, Instagram, the Web, and more
https://www.jovo.tech
Apache License 2.0
1.67k stars 310 forks source link

Repeat doesn't work with APLA #912

Closed acerbisgianluca closed 3 years ago

acerbisgianluca commented 3 years ago

I'm submitting a...

Expected Behavior

At the end of the RepeatIntent handler, I return this.repeat() and the voice assistant should play the last output.

Current Behavior

It works on Google Assistant, but doesn't work on Alexa. Probably this is related to the fact that I use APLA for outputSpeech on Alexa, so the field in the response is empty and Jovo doesn't take into account the APLA directive.

Error log

{
    "errorType": "TypeError",
    "errorMessage": "Cannot read property 'replace' of null",
    "stack": [
        "TypeError: Cannot read property 'replace' of null",
        "    at Function.toSSML (/var/task/node_modules/jovo-core/dist/src/util/SpeechBuilder.js:24:21)",
        "    at AlexaCore.output (/var/task/node_modules/jovo-platform-alexa/dist/src/modules/AlexaCore.js:88:63)",
        "    at Middleware.run (/var/task/node_modules/jovo-core/dist/src/core/Middleware.js:76:30)",
        "    at Alexa.output (/var/task/node_modules/jovo-platform-alexa/dist/src/Alexa.js:121:42)",
        "    at Middleware.run (/var/task/node_modules/jovo-core/dist/src/core/Middleware.js:76:30)",
        "    at error (/var/task/node_modules/jovo-core/dist/src/plugins/Handler.js:205:67)",
        "    at processTicksAndRejections (internal/process/task_queues.js:97:5)",
        "    at async Middleware.run (/var/task/node_modules/jovo-core/dist/src/core/Middleware.js:76:21)",
        "    at async App.handle (/var/task/node_modules/jovo-core/dist/src/core/BaseApp.js:233:13)",
        "    at async App.handle (/var/task/node_modules/jovo-framework/dist/src/App.js:265:9)"
    ]
}

Your Environment

aswetlow commented 3 years ago

Hey @acerbisgianluca Could you provide the response JSON? How do you make APL-A response? I have a guess.

acerbisgianluca commented 3 years ago

Hey @aswetlow! This is the correct response. If I say "repeat" after this, I get the error reported in the first comment.

"response": {
        "shouldEndSession": false,
        "outputSpeech": {
            "type": "SSML",
            "ssml": "<speak></speak>"
        },
        "reprompt": {
            "outputSpeech": {
                "type": "SSML",
                "ssml": "<speak><audio src=\"URL\"></audio> <audio src=\"URL\"></audio></speak>"
            }
        },
        "directives": [
            {
                "type": "Alexa.Presentation.APLA.RenderDocument",
                "token": "8",
                "document": {
                    "version": "0.8",
                    "type": "APLA",
                    "mainTemplate": {
                        "parameters": [
                            "payload"
                        ],
                        "item": {
                            "type": "Sequencer",
                            "items": [
                                {
                                    "type": "Audio",
                                    "source": "URL"
                                },
                                {
                                    "type": "Audio",
                                    "source": "URL"
                                }
                            ]
                        }
                    }
                }
            },
            {
                "type": "Display.RenderTemplate",
                "template": {
                    "backButton": "HIDDEN",
                    "type": "BodyTemplate7",
                    "image": {
                        "contentDescription": "Launch",
                        "sources": [
                            {
                                "url": "URL",
                                "size": "X_SMALL"
                            },
                            {
                                "url": "URL",
                                "size": "SMALL"
                            },
                            {
                                "url": "URL",
                                "size": "MEDIUM"
                            },
                            {
                                "url": "URL",
                                "size": "LARGE"
                            },
                            {
                                "url": "URL",
                                "size": "X_LARGE"
                            }
                        ]
                    },
                    "token": "Launch",
                    "backgroundImage": {
                        "contentDescription": "background",
                        "sources": [
                            {
                                "url": "URL"
                            }
                        ]
                    }
                }
            }
        ]
    }

I add the directive using this.$alexaSkill.addDirective(aplaDirective.build()); where aplaDirective.build() returns the directive object as you can see it in the response.

aswetlow commented 3 years ago

I tried it with the following code and it worked:

LAUNCH() {
        this.$alexaSkill.addAplDirective({
            type: 'Alexa.Presentation.APLA.RenderDocument',
            token: '8',
            document: {
                version: '0.8',
                type: 'APLA',
                mainTemplate: {
                    parameters: ['payload'],
                    item: {
                        type: 'Sequencer',
                        items: [
                            {
                                type: 'Audio',
                                source:
                                    '<AUDIO>',
                            },
                        ],
                    },
                },
            },
        });
        return this.ask(' ', ' ');
    },

RepeatIntent in my handler:

    RepeatIntent() {
        this.repeat();
    },

This is the db entry in my local db.json

{
        "userId": "-",
        "userData": {
            "data": {},
            "context": {
                "prev": [
                    {
                        "response": {
                            "speech": " ",
                            "reprompt": " ",
                            "output": {
                                "Alexa": {
                                    "Apl": [
                                        {
                                            "type": "Alexa.Presentation.APLA.RenderDocument",
                                            "token": "8",
                                            "document": {
                                                "version": "0.8",
                                                "type": "APLA",
                                                "mainTemplate": {
                                                    "parameters": [
                                                        "payload"
                                                    ],
                                                    "item": {
                                                        "type": "Sequencer",
                                                        "items": [
                                                            {
                                                                "type": "Audio",
                                                                "source": "<AUDIO>"
                                                            }
                                                        ]
                                                    }
                                                }
                                            }
                                        }
                                    ]
                                },
                                "ask": {
                                    "speech": " ",
                                    "reprompt": " "
                                }
                            }
                        },
                        "request": {
                            "timestamp": "2021-04-01T10:16:53Z",
                            "inputs": {},
                            "intent": "LAUNCH"
                        }
                    }
                ]
            }
        }
    }

What's in your db.json, after the first request?

acerbisgianluca commented 3 years ago

@aswetlow Thank you for your example, I think I've found the problem.

You return this.ask(' ', ' ') while I'm returning this.ask(this.$speech, this.$reprompt), but for the speech part I'm using APLA so this.$speech is empty and it's stored in dynamo as null. When I call this.repeat() it throws the error ("Cannot read property 'replace' of null").

"response": {
      "output": {
       "ask": {
        "speech": null,
        "reprompt": "<audio src=\"URL\"></audio> <audio src=\"https://URL\"></audio>"
       },
       "Alexa": {
        "Apl": [
         {
          "type": "Alexa.Presentation.APLA.RenderDocument",
          "token": "5",
          "document": {
           "type": "APLA",
           "mainTemplate": {
            "parameters": [
             "payload"
            ],
            "item": {
             "type": "Sequencer",
             "items": [
              {
               "type": "Audio",
               "source": "URL"
              },
              {
               "type": "Audio",
               "source": "URL"
              }
             ]
            }
           },
           "version": "0.8"
          }
         }
        ],
        "DisplayTemplate": {
         "type": "Display.RenderTemplate",
          "type": "BodyTemplate7",
          "token": "Launch",
          .........
         }
        }
       }
      },
      "speech": null,
      "reprompt": "<audio src=\"URL\"></audio> <audio src=\"URL\"></audio>"
     }

So I added this.$speech.addText(' ') only when the request is coming from Alexa and now it's working!

aswetlow commented 3 years ago

Great!

You could also use $speech.addBreak('10ms');. Sometimes it looks cleaner than white space. Same same, but different 😂