search

alexa / alexa-skills-kit-sdk-for-nodejs

The Alexa Skills Kit SDK for Node.js helps you get a skill up and running quickly, letting you focus on skill logic instead of boilerplate code.
Apache License 2.0
3.13k stars 737 forks source link

AudioPlayer Interface doesn't support HTTP Authorization header (in fact any of HTTP headers) along with the audio file URL #610

Open voiceactivity opened 4 years ago

voiceactivity commented 4 years ago

Hi team, About 6 months ago we published the Sound Drive skill. This skill allows users to play their own audio files from their Google Drive accounts. The skill uses AudioPlayer Interface.

Many people (including those ones with Premium subscriptions) were enjoying this skill until now. But now we have no other choice but shut it down. The reason for that is the change Google recently made in the Google Drive API. In the past Google allowed downloading files with the access_token attached to the HTTP GET request as a parameter. So, our Play Directive looked like this: { "type": "AudioPlayer.Play", "playBehavior": "valid playBehavior value such as ENQUEUE", "audioItem": { "stream": { "url": "https://www.googleapis.com/drive/v3/files/1ke4YoCoroY3YYD9tLUvrhv_WjiUZFbJY?alt=media&access_token=ya29.ImG9B...", "token": "opaque token representing this stream", "expectedPreviousToken": "opaque token representing the previous stream", "offsetInMilliseconds": 0, ... } This way the AudioPlayer was requesting the file from Google Drive providing a valid access token.

Now, it does require that the access_token been passed inside the Authorization HTTP header. For example - Authorization: Bearer ya29.Im..... Actually, Google was the last who deprecated this feature - passing the access_token in the HTTP GET request.

Unfortunately, the AudioPlayer Interface (Alexa Skill SDK) doesn't support that: https://developer.amazon.com/en-US/docs/alexa/custom-skills/audioplayer-interface-reference.html#play

According to the CloudWatch logs, Alexa uses this: AlexaMediaPlayer/2.1.3131.0 (Linux;Android 5.1.1) ExoPlayerLib/1.5.9

In particular, the ExoPlayerLib provides extensive supports of HTTP headers in the request: https://github.com/google/ExoPlayer https://github.com/google/ExoPlayer/blob/76962d50f1d80941d6768e4e765fa4ff010705e7/library/core/src/main/java/com/google/android/exoplayer2/upstream/DataSpec.java https://github.com/google/ExoPlayer/blob/76962d50f1d80941d6768e4e765fa4ff010705e7/library/core/src/main/java/com/google/android/exoplayer2/upstream/DefaultHttpDataSource.java

Having the AudioPlayer Interface without HTTP headers support is like trying to spend $1000 000 in the $0.99 shop.

Could you please consider to implement that? Until then, we are shutting down our Skill and the service.

Thank you!

ghost commented 4 years ago

Hang on, I think you are rather over-panicking here

https://cloud.google.com/blog/products/application-development/upcoming-changes-to-the-google-drive-api-and-google-picker-api

Here’s what we recommend you do if authentication with the HTTP header is not possible:

For file downloads, redirect to the webContentLink which will instruct the browser to download the content. If the application wants to display the file to the user, they can simply redirect to the alternateLink in v2 or webViewLink in v3.

For file exports, redirect to the export link in exportLinks with the desired mime type which will instruct the browser to download the content.

Also, can you just not make a pre-request to get a temporary download link, like you have to do with pre-signed downloads via S3?

There's a definitely a solution here....

voiceactivity commented 4 years ago

Hi, Thank you for your reply.

We tested those options of course. They both return the HTML pages with the button. When you click the button, the download starts. But we need a binary stream instead for the player.

Thank you Paul

On Mon, 10 Feb 2020 23:15 digitaltoast, notifications@github.com wrote:

Hang on, I think you are rather over-panicking here

https://cloud.google.com/blog/products/application-development/upcoming-changes-to-the-google-drive-api-and-google-picker-api

Here’s what we recommend you do if authentication with the HTTP header is not possible:

For file downloads, redirect to the webContentLink which will instruct the browser to download the content. If the application wants to display the file to the user, they can simply redirect to the alternateLink in v2 or webViewLink in v3.

For file exports, redirect to the export link in exportLinks with the desired mime type which will instruct the browser to download the content.

Also, can you just not make a pre-request to get a temporary download link, link you have to do with pre-signed downloads via S3?

There's a definitely a solution here....

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/alexa/alexa-skills-kit-sdk-for-nodejs/issues/610?email_source=notifications&email_token=AAPC26KRSPK44MISWRARQZDRCESMNA5CNFSM4KSHDMJ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOELH6KUA#issuecomment-584050000, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAPC26N6T36BPLUHNZ2Z5ITRCESMNANCNFSM4KSHDMJQ .

voiceactivity commented 4 years ago

In terms of file exports you mentioned ( https://developers.google.com/drive/api/v3/reference/files/export) This is the result of request:

{ "error": { "errors": [ { "domain": "global", "reason": "fileNotExportable", "message": "Export only supports Google Docs." } ], "code": 403, "message": "Export only supports Google Docs." } }

Export only supports Google Docs, so we cannot rely on anything else Google have to offer. We tested many approaches, and the files.get (with ?alt=media) is the only option to download (stream) files from Google Drive.

Maybe you are aware of any request parameters that could be passed over to the AudioPlayer Interface url field in the Play Directive? I saw somewhere on the internet that folks were trying to pass parameters as a part of player's url (HTTP GET request) for basic authentication and somehow the Player implementation (or a wrapper its wrapper) sets up some headers.

Not sure how it is possible with Authorization: Bearer xyz... header.

Ideally would be great to expose HTTP headers to the AudioPlayer Interface. It would be a tremendous benefit. In that way developers could integrate the Alexa AudioPlayer with any backend (API) that requires a specific headers.

And as I it looks not so big and difficult task (I looked at the Player's src code on Github). Maybe it would be considered worth to implement.

At the moment, as I mentioned, we had no choice but to shut this service and skill down. If it is very uncertain that this feature could be (will be) implemented at all we could to look at another option: porting the skill to the Google Assistants (Nest).

But what we really want is that the folks like us, Amazon Echo devices owners, could use it on their Alexa devices rather than Google devices :-)

My family personally own 7 Alexa devices! We were heavy users of the Sound Drive skill! And we feel deprived because we cannot use it anymore. And there are quite a few of those ones who feels this way.

Thank you Paul

On Mon, Feb 10, 2020 at 11:15 PM digitaltoast notifications@github.com wrote:

Hang on, I think you are rather over-panicking here

https://cloud.google.com/blog/products/application-development/upcoming-changes-to-the-google-drive-api-and-google-picker-api

Here’s what we recommend you do if authentication with the HTTP header is not possible:

For file downloads, redirect to the webContentLink which will instruct the browser to download the content. If the application wants to display the file to the user, they can simply redirect to the alternateLink in v2 or webViewLink in v3.

For file exports, redirect to the export link in exportLinks with the desired mime type which will instruct the browser to download the content.

Also, can you just not make a pre-request to get a temporary download link, link you have to do with pre-signed downloads via S3?

There's a definitely a solution here....

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/alexa/alexa-skills-kit-sdk-for-nodejs/issues/610?email_source=notifications&email_token=AAPC26KRSPK44MISWRARQZDRCESMNA5CNFSM4KSHDMJ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOELH6KUA#issuecomment-584050000, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAPC26N6T36BPLUHNZ2Z5ITRCESMNANCNFSM4KSHDMJQ .

--

Pavel Filippov,

Voice Activity Ltd, Founder and Director https://voiceactivity.com https://voiceactivity.com

ShenChen93 commented 4 years ago

Hi Paul @voiceactivity

Thanks for posting this issue. Cause this feature request need APIs support and then we could release new SDK models, I reached out to internal service team and delivered your demand. Right now all relevant team start looking into it.

I will keep this issue open for now, and please keep tuned :)

Thanks, Shen

voiceactivity commented 4 years ago

Thank you Shen!

If this issue were addressed that would be awesome!

We have customers with subscriptions who are eager to pay for this service. We're receiving emails from people asking when this skill will be available in their countries. Potential is great!

In case the issue addressed we could update the skill in a matter of few hours to pass the HTTP Authorization header along with the audio's URL (AudioPlayer Interface).

Just in case, if your team decided to fix HTTP headers in AudioPlayer Interface, could you please take a look at the VideoApp interface as well. It does have the same problem. Maybe in your team's next sprint :-)

Btw, Google Assistant API doesn't provide any way of integration videos apart from YouTube into Google Actions (aka Alexa skills).

Having those 2 interfaces done properly gives the Alexa SDK platform a huge advantage over the Google Assistant platform!

It is a great opportunity for Amazon and for us as a dev community.

Thank you

Paul, Voice Activity CEO

On Tue, Feb 18, 2020 at 10:40 AM ShenChen-Amazon notifications@github.com wrote:

Hi Paul @voiceactivity https://github.com/voiceactivity

Thanks for posting this issue. Cause this feature request need APIs support and then we could release new SDK models, I reached out to internal service team and delivered your demand. Right now all relevant team start looking into it.

I will keep this issue open for now, and please keep tuned :)

Thanks, Shen

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/alexa/alexa-skills-kit-sdk-for-nodejs/issues/610?email_source=notifications&email_token=AAPC26KKHGGOB46KLTVA64TRDL73DA5CNFSM4KSHDMJ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEL7XNGA#issuecomment-587167384, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAPC26IB7VWOJ6LFFRP4VV3RDL73DANCNFSM4KSHDMJQ .

--

Pavel Filippov,

Voice Activity Ltd, Founder and Director https://voiceactivity.com https://voiceactivity.com

voiceactivity commented 4 years ago

Have you given up on this request?

voiceactivity commented 4 years ago

Hi @ShenChen-Amazon I hope you're doing well. Any updates for this request? Thank you

ShenChen93 commented 4 years ago

@voiceactivity ,

Thanks for being patient. Unfortunately, at this moment, we haven't heard any updates from services team about this feature release. I will post updates here once this feature released to public.

Thanks, Shen

voiceactivity commented 4 years ago

Hi @ShenChen-Amazon ,

Any chance to get a direct contact of anybody in charge in the service team? It's really shame that we are loosing such a great opportunity right now by postponing the fix.

Cheers, Paul

ShenChen93 commented 4 years ago

@voiceactivity ,

I am very sorry for the delay. You could log the issue on forums: https://alexa.uservoice.com/forums/906892-alexa-skills-developer-voice-and-vote if you want to contact with the service team directly.

Thanks, Shen

voiceactivity commented 4 years ago

We have done this already, a couple of months ago. Dead quiet. Posted there another comment. But , to be honest, I've started giving up in my hope that feature will ever be implemented and our skill could be resurrected. I cannot see that Amazon is interested in it.

Thank you,

voiceactivity commented 3 years ago

Looks like the latest AVS AudioPlayer v1.5 added this feature: https://developer.amazon.com/en-US/docs/alexa/alexa-voice-service/audioplayer.html#version-changes

Support for passing HTTP header information in an audio stream, enabling DRM and other authenticated resources :

{ "header": { "namespace": "AudioPlayer", "name": "Play", "messageId": "{{STRING}}", "dialogRequestId": "{{STRING}}" }, "payload": { ... ], "audioItem": { "audioItemId": "{{STRING}}", ... "caption": { "content": "{{STRING}}", "type": "{{STRING}}" }, "httpHeaders": { "manifest": [ { "name:": "{{STRING}}", "value:": "{{STRING}}" } ],...

Is it what we're waiting for from the Service team?
That would be awesome.

Thank you

ShenChen93 commented 3 years ago

Hi @voiceactivity .

Yep, this feature is supported in AudioPlayer v1.5. However, the interface haven't been updated yet: https://developer.amazon.com/en-US/docs/alexa/custom-skills/audioplayer-interface-reference.html, pleaes keep tunned.

Thanks, Shen

abhjroy commented 2 years ago

Hi @voiceactivity is this ticket is still relevant?

pfilippov commented 2 years ago

Hi, yes, of course. However there is still no change been made in the ASK to support that update. Thank you

RobinBeismann commented 1 year ago

Any idea when this feature comes?

warren-bank commented 1 year ago

Hi. I'm brand new to Alexa skills. It blows my mind that this issue even exists.. I mean, the AI interaction models are amazing.. but to not implement something so basic as providing a way to secure the audio streams that a skill will ultimately access on behalf of the user.. it's just odd.

Anyway.. I just took a quick look at the code in this repo.. where support for this feature should be included and exposed through the JS API:

https://github.com/alexa/alexa-skills-kit-sdk-for-nodejs/blob/3849332e395b11ddeb1fb68a98aed1d2df026a04/ask-sdk-core/lib/response/ResponseFactory.ts#L215 https://github.com/alexa/alexa-skills-kit-sdk-for-nodejs/blob/3849332e395b11ddeb1fb68a98aed1d2df026a04/ask-sdk-core/lib/response/ResponseFactory.ts#L222

..and it seems to me that in a Lambda.. as a janky workaround.. wouldn't this work?

{
  const response = handlerInput.responseBuilder
    // ...
    .getResponse()

  if (response.directives) {
    response.directives.forEach(directive => {
      if (directive.type === 'AudioPlayer.Play') {
        directive.audioItem.stream.httpHeaders = {
          "all": [{
            "name":  "",
            "value": ""
          }]
        }
      }
    })
  }

  return response
}

..I haven't tested it yet, but will soon

pfilippov commented 1 year ago

Hi, the future you're referring to implemented in AVS (Alexa Voice Services) Audio Interface v.1.5 in about 1 year ago. The next step should be done be Alexa Skills team to make it a part of Alexa Skills SDK. So, your code just won't work, because AVS and Alexa Skills SDK are different things. Looks like Alexa Skills team is very reluctant to implement this feature. Don't know why. It's not very difficult, considering the fact that AVS v.1.5 already supports it. Anyway, give it a go and post an update 🙂

warren-bank commented 1 year ago

I just tested my proposed workaround.. and it does work.

I pushed a demo Alexa skill to this repo.. and (at least for right now) this skill is in the default branch.. so you can test it yourself by creating a new custom skill (in the Alexa Developer Console).. and cloning from: https://github.com/warren-bank/Alexa-skill-demos.git

In order to perform the test, this points the URL for an audio stream at a public URL from requestbin.com.. which allows a log of the request to be easily examined.

I'm not sure how long these logs remain available, but you can see my test result here.

The test header was received by the server, which would (typically) be hosting the audio file:

x-alexa-issue: https://github.com/alexa/alexa-skills-kit-sdk-for-nodejs/issues/610#issuecomment-1483729767
pfilippov commented 1 year ago

Hi Warren, thanks, looking at the log it seems that it works 👍 I will try that too. Thank you

warren-bank commented 1 year ago

For my own needs.. and anybody else who may be interested in using it.. I'm in the process right now of packaging this up as a tiny npm module with a clean API. It'll be on npm in less than an hour.

warren-bank commented 1 year ago

ok.. it's ready to go