DRM: Check key status before adapting to a different representation (API level 23+)

okycelt commented 6 years ago

Issue description

We use Widevine DRM together with ExoPlayer. When we limit allowed_track_types on our Widevine proxy eg. to SD_ONLY and we try playing DASH containing some HD tracks, the playback fails with the following error when the player tries to adapt to an HD track.

E/WVCdm: PolicyEngine::CanDecrypt Key '5955A05307225C50B4B4A556155C12A1' not in license.
    Decrypt error result in session sid32 during encrypted block: 5
D/VodPlayerFragment$MediaSourceListener: onDownstreamFormatChanged(format: Format(5, video/mp4, video/avc, 4500000, null, [1920, 1080, 23.98], [-1, -1]))
E/ExoPlayerImplInternal: Playback error.
    com.google.android.exoplayer2.ExoPlaybackException
        at com.google.android.exoplayer2.mediacodec.MediaCodecRenderer.feedInputBuffer(MediaCodecRenderer.java:771)
        at com.google.android.exoplayer2.mediacodec.MediaCodecRenderer.render(MediaCodecRenderer.java:574)
        at com.google.android.exoplayer2.ExoPlayerImplInternal.doSomeWork(ExoPlayerImplInternal.java:518)
        at com.google.android.exoplayer2.ExoPlayerImplInternal.handleMessage(ExoPlayerImplInternal.java:301)
        at android.os.Handler.dispatchMessage(Handler.java:101)
        at android.os.Looper.loop(Looper.java:164)
        at android.os.HandlerThread.run(HandlerThread.java:65)
     Caused by: android.media.MediaCodec$CryptoException: Crypto key not available
        at android.media.MediaCodec.native_queueSecureInputBuffer(Native Method)
        at android.media.MediaCodec.queueSecureInputBuffer(MediaCodec.java:2532)
        at com.google.android.exoplayer2.mediacodec.MediaCodecRenderer.feedInputBuffer(MediaCodecRenderer.java:762)
        at com.google.android.exoplayer2.mediacodec.MediaCodecRenderer.render(MediaCodecRenderer.java:574) 
        at com.google.android.exoplayer2.ExoPlayerImplInternal.doSomeWork(ExoPlayerImplInternal.java:518) 
        at com.google.android.exoplayer2.ExoPlayerImplInternal.handleMessage(ExoPlayerImplInternal.java:301) 
        at android.os.Handler.dispatchMessage(Handler.java:101) 
        at android.os.Looper.loop(Looper.java:164) 
        at android.os.HandlerThread.run(HandlerThread.java:65)

The expected behaviour would be that player wouldn't switch to a video track it doesn't have a license for.

Is ExoPlayer able to handle this situation itself or we have to make a workaround, eg. by making an initial track selection excluding HD tracks?

Shaka player is handling this situation correctly, ie. it's not adapting to a not allowed track.

This is more of a question, so I'm not enclosing any test content or adb bugreport. If you believe this could be an issue related to ExoPlayer, I can provide that afterwards.

Reproduction steps

Set allowed_track_types in Widevine proxy to SD_ONLY. Play content containing HD track.

Version of ExoPlayer being used

2.8.2

Device(s) and version(s) of Android being used

Amlogic STB with Android 6.0.1 Amlogic STB with Android 8.0

ojw28 commented 6 years ago

Is ExoPlayer able to handle this situation itself or we have to make a workaround, eg. by making an initial track selection excluding HD tracks?

For the time-being you need to ensure that you don't try and play a stream that's not going to be licensed. This is probably best done on the serving side, by serving a DASH manifest that does not include the tracks that aren't going to be playable.

More generally, I agree the client should not attempt to switch to a stream for which it does not have the necessary key, however:

As far as I can see, the platform API we need to do this was only added in API level 23. Hence the workaround will be needed for quite a long period of time anyway for older devices.
At the start of playback a player will typically make the license request in parallel with buffering the media it's going to play. Hence it needs to pick an initial representation to buffer without knowing which keys are going to be usable. If it picks a representation and then doesn't get a usable key for it, it'll need to discard what it's buffered and start again. So implementing the workaround may give you an efficiency gain even when it's no longer necessary for playback to be successful.

@jefftinker - Regarding the first point, is there a way to do this on earlier API levels that I've missed?

okycelt commented 6 years ago

This is probably best done on the serving side, by serving a DASH manifest that does not include the tracks that aren't going to be playable.

We're not really in favour of this as it would quite complicate our server side solution.

Regarding making an initial track selection, would it ensure the behaviour we're trying to achieve, ie. avoid switching to streams with resolution higher than some specified threshold? We only need to limit the tracks by a not higher than threshold so the player should always receive keys for the lowest resolutions.

ojw28 commented 6 years ago

If you can specify the selection you need in the form of a constraint, then a client side solution would work fine. For example something like:

trackSelector.setParameters(
  trackSelection.buildUponParameters().setMaxVideoSize(width, height));

If you want to disable HD streams, you'd probably use width=1279 and height=719 (although for that specific case you could just use setMaxVideoSizeSd, which fills in those values for you). There's also setMaxVideoBitrate if you want to constrain based on known format bitrates used in your manifests.

okycelt commented 6 years ago

I'll give it a try, thank you!

ojw28 commented 5 years ago

@paf-wv - Could you comment on the question in my initial response to this issue (i.e. the second post in the thread)? Thanks.

ojw28 commented 5 years ago

Ping @paf-wv / @jefftinker - Thanks.

paf-wv commented 5 years ago

Sorry for the delayed response. ACK the need to answer the workaround question. Have that discussion going internally now.

paf-wv commented 5 years ago

Happy New Year @ojw28,
A reply for the question posted on 9/17.

" You are correct, prior to API 23 this would need to be coordinated between client and server independently from MediaDrm API."

paf-wv commented 5 years ago

No further feedback closing please reopen if needed.

ojw28 commented 5 years ago

Thanks for confirming. Re-opening to track checking key status on API level 23 and above.

BucherTomas commented 2 years ago

The ticket is still open, I assume that you plan to automate filtering of usable tracks in ExoPlayer itself for API 23 and newer, so is there any progress please since the last activity in this thread?

We are in similar situation, our case is an L3 device attempting to play adaptive stream where HD tracks are L1 only and the license provider detecting the requesting device as L3 does not even return the key for those HD tracks, only for SD and audio. ExoPlayer then throws the same exception CanDecryptContent Key 'xxxxx' not in license. as mentioned at the start of this thread.

We target API 23+ and ExoPlayer ideally should play the remaining usable tracks here automatically without any supporting logic from application level if the API can be leveraged for this.

If you have this in the pipeline when it could be roughly completed, please? We need to decide whether it might be more beneficial to wait for some native support or if we rather should implement the filter as suggested in the meantime in our application.

Thank you.

icbaker commented 2 years ago

There's been no progress since the last update, and we haven't currently scheduled any time to work on this.

Any commits associated with this issue will show up in this thread, so when we start committing code you will be able to see it here.

chladto1 commented 2 years ago

Our content provider introduces new DRM content policies. Some streams require the L1 DRM security level, some require the L3. We obtain a URL for content that contains all streams and a DRM license for each one separately. The user should only see the resolutions that can be played, and we would like to prevent the adaptive stream from selecting a stream that cannot be played on his device. Can you advise us on how we can achieve this?

Unfortunately, we don't have any L3 devices at this time, but when we try to play this content on the emulator, we get the following error.

Content URL https://1798253129.ssl.cdn.cra.cz/POC/fullkey-cenc-dref.ism/.mpd?python_pipeline_config=multikey

License acquirer https://drm-widevine-licensing.axtest.net/AcquireLicense

DRM Axinom token eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJjb21fa2V5X2lkIjoiNjVGM0JCNjItQUUwMS00MThGLUFCNkUtRTlBNTgwOUU3MEIxIiwibWVzc2FnZSI6eyJjb250ZW50X2tleV91c2FnZV9wb2xpY2llcyI6W3sibmFtZSI6ImhpZ2hzZWMiLCJwbGF5cmVhZHkiOnsibWluX2RldmljZV9zZWN1cml0eV9sZXZlbCI6MzAwMCwiZGlnaXRhbF92aWRlb19vdXRwdXRfcHJvdGVjdGlvbnMiOlt7ImNvbmZpZ19kYXRhIjoiQUFBQUFRPT0iLCJpZCI6IkFCQjJDNkYxLUU2NjMtNDYyNS1BOTQ1LTk3MkQxN0IyMzFFNyJ9XSwiYW5hbG9nX3ZpZGVvX291dHB1dF9wcm90ZWN0aW9ucyI6W3siY29uZmlnX2RhdGEiOiJBQUFBQVE9PSIsImlkIjoiNzYwQUU3NTUtNjgyQS00MUUwLUIxQjMtRENERjgzNkE3MzA2In1dLCJhbmFsb2dfdmlkZW9fb3BsIjoyMDEsImNvbXByZXNzZWRfZGlnaXRhbF92aWRlb19vcGwiOjUwMCwidW5jb21wcmVzc2VkX2RpZ2l0YWxfdmlkZW9fb3BsIjozMDB9LCJ3aWRldmluZSI6eyJkaXNhYmxlX2FuYWxvZ19vdXRwdXQiOmZhbHNlLCJoZGNwIjoiMi4wIiwiY2dtcy1hIjoibmV2ZXIiLCJkZXZpY2Vfc2VjdXJpdHlfbGV2ZWwiOiJIV19TRUNVUkVfQUxMIn19LHsibmFtZSI6Imxvd3NlYyIsInBsYXlyZWFkeSI6eyJtaW5fZGV2aWNlX3NlY3VyaXR5X2xldmVsIjoyMDAwfSwid2lkZXZpbmUiOnsiY2dtcy1hIjoibmV2ZXIiLCJkZXZpY2Vfc2VjdXJpdHlfbGV2ZWwiOiJTV19TRUNVUkVfREVDT0RFIn19LHsicGxheXJlYWR5Ijp7Im1pbl9kZXZpY2Vfc2VjdXJpdHlfbGV2ZWwiOjIwMDB9LCJuYW1lIjoiYXVkaW8iLCJ3aWRldmluZSI6eyJkZXZpY2Vfc2VjdXJpdHlfbGV2ZWwiOiJTV19TRUNVUkVfQ1JZUFRPIn19XSwidmVyc2lvbiI6MiwidHlwZSI6ImVudGl0bGVtZW50X21lc3NhZ2UiLCJjb250ZW50X2tleXNfc291cmNlIjp7ImlubGluZSI6W3sidXNhZ2VfcG9saWN5IjoibG93c2VjIiwiaWQiOiJmNWJmMzNhYy0wODQ0LTBjODEtZDYyMy0wMTljODdmZTEzNjAiLCJpdiI6IlQzSGVMOHpiMkxMZ29nZFZiOWtYd1E9PSJ9LHsidXNhZ2VfcG9saWN5IjoiaGlnaHNlYyIsImlkIjoiZjRlYThkYjgtNzZhZS05YzhjLTJhMjEtOTg4MmQ3NjNiYzdkIiwiaXYiOiJpRXVLTEFRMElvYVpzZ3Voc0s1SENRPT0ifSx7InVzYWdlX3BvbGljeSI6ImF1ZGlvIiwiaWQiOiJmZjE0ZmM5Yy1iOGM3LTYwNjctY2Q0Yy1hY2YyOTQzNjU4NWEiLCJpdiI6Im1rUTA2bENFdG45YkVLYStzSUpMUGc9PSJ9XX0sImxpY2Vuc2UiOnsiZHVyYXRpb24iOjYwNDgwMCwid2lkZXZpbmUiOnsiaW5jbHVkZV9hbGxfZW50aXRsZWRfa2V5cyI6dHJ1ZX19fSwidmVyc2lvbiI6MX0.yCnrwiTz3Fep8WHH0TCK-aER33ezPt07jL5FvEtUmEE

Error 2022-02-08 22:20:20.760 4331-6064/com.example.player E/ExoPlayerImplInternal: Playback error com.google.android.exoplayer2.ExoPlaybackException: MediaCodecVideoRenderer error, index=0, format=Format(video=1830000, null, null, video/avc, avc1.4D401F, 1830000, null, [1280, 720, -1.0], [-1, -1]), format_supported=YES at com.google.android.exoplayer2.ExoPlayerImplInternal.handleMessage(ExoPlayerImplInternal.java:558) at android.os.Handler.dispatchMessage(Handler.java:103) at android.os.Looper.loop(Looper.java:214) at android.os.HandlerThread.run(HandlerThread.java:67) Caused by: android.media.MediaCodec$CryptoException: Crypto key not available at android.media.MediaCodec.native_queueSecureInputBuffer(Native Method) at android.media.MediaCodec.queueSecureInputBuffer(MediaCodec.java:2699) at com.google.android.exoplayer2.mediacodec.SynchronousMediaCodecAdapter.queueSecureInputBuffer(SynchronousMediaCodecAdapter.java:148) at com.google.android.exoplayer2.mediacodec.MediaCodecRenderer.feedInputBuffer(MediaCodecRenderer.java:1367) at com.google.android.exoplayer2.mediacodec.MediaCodecRenderer.render(MediaCodecRenderer.java:825) at com.google.android.exoplayer2.ExoPlayerImplInternal.doSomeWork(ExoPlayerImplInternal.java:978) at com.google.android.exoplayer2.ExoPlayerImplInternal.handleMessage(ExoPlayerImplInternal.java:482) at android.os.Handler.dispatchMessage(Handler.java:103) at android.os.Looper.loop(Looper.java:214) at android.os.HandlerThread.run(HandlerThread.java:67)

chladto1 commented 2 years ago

@icbaker Could you please help us with the issue described above? It would help us a lot. Thanks!

icbaker commented 2 years ago

@chladto1 If you require the enhancement tracked by this issue then I'm afraid there's no update since last month: https://github.com/google/ExoPlayer/issues/4825#issuecomment-1023250924

The recommendation above to restrict the streams yourself (either server-side or client-side) stands: https://github.com/google/ExoPlayer/issues/4825#issuecomment-422116376

If you believe what you're seeing is unrelated to this enhancement then please file a new issue and we can take a look, in order to keep the discussion separate. It's unclear from the information provided so far whether you see the error only when adapting to the L1 streams (in which case it sounds related to this issue, and we will just dupe back this) or whether you always see the error when playing L3 content (which may be an emulator issue, and the first thing we will ask you to try is reproing on a real device.

icbaker commented 2 days ago

Closing this as a duplicate of the an equivalent issue in the newer media3 tracker: https://github.com/androidx/media/issues/1776

google / ExoPlayer