actions-on-google / actions-on-google-nodejs

Node.js client library for Actions on Google
https://actions-on-google.github.io/actions-on-google-nodejs
Apache License 2.0
900 stars 197 forks source link

SSML strips white space incorrectly on phone? #393

Open dbarnespaychex opened 4 years ago

dbarnespaychex commented 4 years ago

Hello, I hope I'm just doing something wrong, but this response renders correctly on a smart device but incorrectly on a phone:

{
  "payload": {
    "google": {
      "expectUserResponse": true,
      "richResponse": {
        "items": [
          {
            "simpleResponse": {
              "textToSpeech": "<speak><p> You were paid <say-as interpret-as=\"unit\">$609.20</say-as> on <say-as interpret-as=\"date\" format=\"md\">Jan 10</say-as>.</p></speak>"
            }
          },
          {
            "tableCard": {
              "title": "$609.20",
              "subtitle": "Dec 30, 2019 - Jan 5, 2020",
              "image": {
                "url": "https://img.icons8.com/cotton/344/money-transfer.png",
                "accessibilityText": "alt text required"
              },
              "rows": [
                {
                  "cells": [
                    {
                      "text": "Hours Worked"
                    },
                    {
                      "text": "40"
                    }
                  ],
                  "dividerAfter": true
                },
                {
                  "cells": [
                    {
                      "text": "Gross Earnings"
                    },
                    {
                      "text": "$844.80"
                    }
                  ],
                  "dividerAfter": true
                },
                {
                  "cells": [
                    {
                      "text": "Deductions"
                    },
                    {
                      "text": "$50.69"
                    }
                  ],
                  "dividerAfter": true
                },
                {
                  "cells": [
                    {
                      "text": "Withholdings"
                    },
                    {
                      "text": "$184.91"
                    }
                  ],
                  "dividerAfter": true
                }
              ],
              "columnProperties": [
                {
                  "horizontalAlignment": "LEADING"
                },
                {
                  "horizontalAlignment": "TRAILING"
                }
              ]
            }
          },
          {
            "simpleResponse": {
              "textToSpeech": "<speak><p> Can I help you with anything else?</p></speak>"
            }
          }
        ]
      }
    }
  },
  "outputContexts": [
    {
      "name": "projects/flexgoteamtest-wbebhn/agent/sessions/ABwppHGJ4EKYfuvjvtgBHTNe2fOoBOpvM3vgLMbw6FpmNtOnvyLecrgU2KyqXw46PXSOnzhCJCpTcDprhylgD1wKYjMr/contexts/followup-system-anythingelse",
      "lifespanCount": 1
    }
  ]
}

Here's how it renders on a phone. Note the missing space between "paid" and "$609.20": bug phone

It appears this way in both the emulator and on a real phone. Maybe it has something to do with showing a card in addition to the simple response?

I've tried re-arranging the words in the sentence, using <break/> between the 2 words, wrapping "paid" in <s>...</s>, and even inserting a Unicode non-breaking space between the 2 words, but the visual presentation still strips that white space between "paid" and "$609.20".

Can someone point me in the right direction? What am I doing wrong?

dbarnespaychex commented 4 years ago

Manually adding a displayText value to the SimpleResponse presents the text correctly on a phone. This leads me to wonder if there's a bug in whatever logic is stripping SSML to create display text.

"simpleResponse": {
  "textToSpeech": "<speak>You were paid <say-as interpret-as=\"unit\">$609.20</say-as> on <say-as interpret-as=\"date\" format=\"md\">Jan 10</say-as>.</speak>",
  "displayText": "You were paid $609.20 on Jan 10."
}

bug phone

Fleker commented 4 years ago

If you're trying to compose sentences with SSML tags inside you may want to wrap sections in <s> like <speak><s>You were paid</s><say-as...

dbarnespaychex commented 4 years ago

Hi @Fleker thanks for your response!

I did try using <s>...</s> around various parts of the sentence but had no luck. It seems as if there's a bug in whatever code strips the SSML so it can be presented as plain text. Maybe the $ is throwing off a Regular Expression somewhere?