search

googlesamples / assistant-sdk-python

Samples and bindings for the Google Assistant API
http://developer.google.com/assistant/sdk
Apache License 2.0
916 stars 321 forks source link

Missing DialogStateOut.supplemental_display_text GRPC messages #158

Open ghost opened 6 years ago

ghost commented 6 years ago

Hi,

I've written my own client for the GRPC service where I am submitting text based requests. I am receiving GRPC messages containing audio and supplemental_display_text but I only ever receive one supplemental_display_text message per response. This is generally ok and provides a transcript of the assistant response but there are some queries which result in the single supplemental_display_text only containing the last part of the response. e.g. If you ask the Google Assistant App on iOS the Question "What do you think of Alexa?" one of the available responses is split over 2 speech bubbles as below:-

Q> What to you think of Alexa? A> Alexa has such a soothing voice A> I like it

Running the same question and response through the GRPC service you receive the full audio response but only a single supplemental_display_text message just containing the second response "I like it", where as I would expect 2 messages one containing "Alexa has such a soothing voice" and a second containing "I like it", or a single message containing a concatenated response

ItsWendell commented 6 years ago

I'm following this, would love to have multiple returns too.

I also noticed that you get a different text format now when you ask for example: what is the best time to go to bed?

You'll get a format like this: -- "Source tile" ( source.domain - source.domain/full-path) Information that google got from this source. --

Would love to know of there are more formats like this!

image

Because it will make my Google Assistant Desktop app a lot more like the mobile app.

Fleker commented 6 years ago

We are aware of the limitation of only the last display text being sent.

ghost commented 6 years ago

@Fleker

Hi Nick,

by limitation do that mean that is a bug or that's a feature that isn't supported by the current SDK?

Thanks

Ross

Fleker commented 6 years ago

It's something that isn't part of the API.

dataoracle commented 6 years ago

Also following this one.

It would be great to get all the messages and not just the last one. Some important use cases that we came across include tell me a joke or tell me a riddle. These ones will always send multi message responses.

@Fleker can you share any updates on this? Is it something that is expected to be handled in the future or is it something that can't be controlled at the service level?

Thanks a lot!

Fleker commented 6 years ago

Right now I do not have an update, but we are aware of it.

abdulhye commented 5 years ago

@Fleker any update on this issue? I am using C# compiled grpc v1alpha2 library of google assistant. my user has Philips hue and thermostat. When he allows me to access his google assistant through oauth, I am using his access token to control his devices.

if I send message like "Turn Lamp 4 Off" it works but if I ask a question, "Is My Lamp 4 Off ? " am not getting any supplemental text from DialogStateOut

is this bug fixed?