search

actions-on-google / assistant-conversation-nodejs

A developer friendly way to fulfill Actions SDK handlers for the Google Assistant
https://actions-on-google.github.io/assistant-conversation-nodejs
Apache License 2.0
104 stars 38 forks source link

Some phrases of an intent doesn't work while audio player is active #6

Open frathi opened 4 years ago

frathi commented 4 years ago

I implemented a simple 'next episode' use case in my action with assistant-conversation-nodejs which works like this:

The user starts a content which consists out of multiple episodes (the audio player starts with the first episode). If the user says something like "next" or "previous" the action navigates through the list of the content.

You can see the scene in this screenshot

01

In the following screenshot you'll see all the phrases of the MORE Intent

02

As you can see there are different phrases like "weiter" ("next" in english) and "mehr" ("more" in english). The right side shows that "weiter" matches MORE.

When I'm testing the application (developer console or google home device or assistant iOS app) I am able to navigate through the content by using the phrase "mehr/more" but the phrase "weiter/next" won't work (same issue for previous with the phrase "zurück/back"). In the console "weiter/next" doesn't show any event or transition, on the device "weiter" brings the google home device into the listening mode. "mehr/more" works fine on any testing device.

This behaviour does only occur when the audio player is active, because when I say something which the action doesn't recognize like "klslkdalasksdakllasdjdlsadj" my action answers "sorry I didn't understand that, can you please repeat?" and I answer this question with "weiter/next" the MORE Intent in my scene does finally match and the next episode starts.

Are there any audio player events which have conflicts with this use case? Is there maybe a given way to implement next and previous which I didn't find in the documentation?

Thank you in advance.

Cheers Frank

actionsoneverything commented 4 years ago

This was also an issue in the previous actions-on-google-nodejs repo: https://github.com/actions-on-google/actions-on-google-nodejs/issues/310

I too have experienced this first hand, 'next song' and 'previous song' don't work, however more elaborate & less intuitive phrases work (not a webhook issue). It's pretty frustrating because it doesn't warn the user that the phrases are reserved, instead it just simply doesn't respond & doesn't reach out to the webhook.

It appears that the consensus in the community is that this functionality (and the next/previous training phrases) is reserved for Google's first party music action implementations and has been intentionally disabled for third party developers.

The buttons are always disabled, as third-party developers are unable to provide metadata that there is another or previous track. Developers building a media response in their Action should expect this behavior, as it's the current state of the API.

Conflicting info:

With regards to being unable to indicate there are previous/next songs, this wouldn't be the case if we could include more than 1 media_object in the MediaObject array, it'd then have sufficient information imo.

Further if this previous/next reservation was disabled, the functionality could be easily implemented in the webhook (or possibly in action builder condition checks) by keeping track of the playlist track index and playlist tracks in session stored data.

actionsoneverything commented 3 years ago

There's now support for this functionality in the actions builder, you can create a media playlist & there's built in intents for navigation.