search

xbmc / inputstream.adaptive

kodi inputstream addon for several manifest types
Other
452 stars 241 forks source link

Image artefacts (pixel) when streaming video DRM #393

Closed Besur closed 3 years ago

Besur commented 4 years ago

I have some video playback problem, a lot of miss pixel are falling apart like big artefacts...

i test all the settings available but nothing change, and it's always on the same parts on same video, for exemple in video 1 : at 1m9s i get artefacts, i change params and restart kodi and i get same artefacts at 1m9s ...

some example : https://i.imgur.com/av3NDNV.png https://i.imgur.com/8fuO9Px.png

i have 1 Gbps down and up btw. Thanks in advance.

Besur commented 4 years ago

Seriously ? no one ? a lot of ppl have this issue...

glennguy commented 4 years ago

A debug log would be a good start to finding the issue.

Besur commented 4 years ago

if you click on dagwieers mention you can see a debug log.

2020-03-14 13:52:46.719 T:22668   DEBUG: AddOnLog: InputStream Adaptive: DecryptSampleData: Decrypt failed with error: 2 and key: EB676ABBCB345E96BBCF616630F1A3DA
2020-03-14 13:52:46.719 T:22668   ERROR: AddOnLog: InputStream Adaptive: Decrypt Sample returns failure!
2020-03-14 13:52:46.722 T:22668   DEBUG: AddOnLog: InputStream Adaptive: DecryptSampleData: Decrypt failed with error: 2 and key: EB676ABBCB345E96BBCF616630F1A3DA
2020-03-14 13:52:46.722 T:22668   ERROR: AddOnLog: InputStream Adaptive: Decrypt Sample returns failure!
2020-03-14 13:52:46.735 T:22668   DEBUG: AddOnLog: InputStream Adaptive: DecryptSampleData: Decrypt failed with error: 2 and key: EB676ABBCB345E96BBCF616630F1A3DA
2020-03-14 13:52:46.735 T:22668   ERROR: AddOnLog: InputStream Adaptive: Decrypt Sample returns failure!
2020-03-14 13:52:46.735 T:22668   DEBUG: AddOnLog: InputStream Adaptive: DecryptSampleData: Decrypt failed with error: 2 and key: EB676ABBCB345E96BBCF616630F1A3DA
2020-03-14 13:52:46.735 T:22668   ERROR: AddOnLog: InputStream Adaptive: Decrypt Sample returns failure!
Besur commented 4 years ago

and for my case it's :

https://i.imgur.com/Ro3HYCA.jpg

> 2020-03-16 18:48:27.143 T:11112   DEBUG: CPtsTracker: pattern lost on diff 83417.000000, number of losses 1
> 2020-03-16 18:48:27.598 T:7084   DEBUG: CurlFile::ParseAndCorrectUrl() adding custom header option 'connection: keep-alive'
> 2020-03-16 18:48:27.598 T:7084   DEBUG: CurlFile::Open(0x2632865eb30) https://wmvideocdn-wakatest.streaming.mediaservices.windows.net/e04424e5-a90a-466c-8342-5d921d39a2f4/LOGO_FIRST.ism/QualityLevels(4027301)/Fragments(video=821242916,format=mpd-time-cmaf,encryption=cenc)
> 2020-03-16 18:48:27.690 T:7084   DEBUG: AddOnLog: InputStream Adaptive: Download https://wmvideocdn-wakatest.streaming.mediaservices.windows.net/e04424e5-a90a-466c-8342-5d921d39a2f4/LOGO_FIRST.ism/QualityLevels(4027301)/Fragments(video=821242916,format=mpd-time-cmaf,encryption=cenc) finished, avg speed: 4763136.61byte/s, current speed: 3273053.00byte/s
> 2020-03-16 18:48:27.731 T:5760   DEBUG: CurlFile::ParseAndCorrectUrl() adding custom header option 'connection: keep-alive'
> 2020-03-16 18:48:27.731 T:5760   DEBUG: CurlFile::Open(0x2632865daf0) https://wmvideocdn-wakatest.streaming.mediaservices.windows.net/e04424e5-a90a-466c-8342-5d921d39a2f4/LOGO_FIRST.ism/QualityLevels(127999)/Fragments(aac_UND_2_127=822193333,format=mpd-time-cmaf,encryption=cenc)
> 2020-03-16 18:48:27.801 T:5760   DEBUG: AddOnLog: InputStream Adaptive: Download https://wmvideocdn-wakatest.streaming.mediaservices.windows.net/e04424e5-a90a-466c-8342-5d921d39a2f4/LOGO_FIRST.ism/QualityLevels(127999)/Fragments(aac_UND_2_127=822193333,format=mpd-time-cmaf,encryption=cenc) finished, avg speed: 4627397.81byte/s, current speed: 544129.00byte/s
> 2020-03-16 18:48:29.604 T:7084   DEBUG: CurlFile::ParseAndCorrectUrl() adding custom header option 'connection: keep-alive'
> 2020-03-16 18:48:29.604 T:7084   DEBUG: CurlFile::Open(0x2632865eb30) https://wmvideocdn-wakatest.streaming.mediaservices.windows.net/e04424e5-a90a-466c-8342-5d921d39a2f4/LOGO_FIRST.ism/QualityLevels(4027301)/Fragments(video=841262916,format=mpd-time-cmaf,encryption=cenc)
> 2020-03-16 18:48:29.721 T:7084   DEBUG: AddOnLog: InputStream Adaptive: Download https://wmvideocdn-wakatest.streaming.mediaservices.windows.net/e04424e5-a90a-466c-8342-5d921d39a2f4/LOGO_FIRST.ism/QualityLevels(4027301)/Fragments(video=841262916,format=mpd-time-cmaf,encryption=cenc) finished, avg speed: 9489053.23byte/s, current speed: 9533266.00byte/s
> 2020-03-16 18:48:29.731 T:5760   DEBUG: CurlFile::ParseAndCorrectUrl() adding custom header option 'connection: keep-alive'
> 2020-03-16 18:48:29.731 T:5760   DEBUG: CurlFile::Open(0x2632865eb30) https://wmvideocdn-wakatest.streaming.mediaservices.windows.net/e04424e5-a90a-466c-8342-5d921d39a2f4/LOGO_FIRST.ism/QualityLevels(127999)/Fragments(aac_UND_2_127=842246666,format=mpd-time-cmaf,encryption=cenc)
> 2020-03-16 18:48:29.790 T:5760   DEBUG: AddOnLog: InputStream Adaptive: Download https://wmvideocdn-wakatest.streaming.mediaservices.windows.net/e04424e5-a90a-466c-8342-5d921d39a2f4/LOGO_FIRST.ism/QualityLevels(127999)/Fragments(aac_UND_2_127=842246666,format=mpd-time-cmaf,encryption=cenc) finished, avg speed: 9200978.36byte/s, current speed: 544709.00byte/s
> 2020-03-16 18:48:31.599 T:7084   DEBUG: CurlFile::ParseAndCorrectUrl() adding custom header option 'connection: keep-alive'
> 2020-03-16 18:48:31.599 T:7084   DEBUG: CurlFile::Open(0x2632865eb30) https://wmvideocdn-wakatest.streaming.mediaservices.windows.net/e04424e5-a90a-466c-8342-5d921d39a2f4/LOGO_FIRST.ism/QualityLevels(4027301)/Fragments(video=861282916,format=mpd-time-cmaf,encryption=cenc)
> 2020-03-16 18:48:31.718 T:7084   DEBUG: AddOnLog: InputStream Adaptive: Download https://wmvideocdn-wakatest.streaming.mediaservices.windows.net/e04424e5-a90a-466c-8342-5d921d39a2f4/LOGO_FIRST.ism/QualityLevels(4027301)/Fragments(video=861282916,format=mpd-time-cmaf,encryption=cenc) finished, avg speed: 11831576.00byte/s, current speed: 11831576.00byte/s
> 2020-03-16 18:48:31.734 T:5760   DEBUG: CurlFile::ParseAndCorrectUrl() adding custom header option 'connection: keep-alive'
> 2020-03-16 18:48:31.734 T:5760   DEBUG: CurlFile::Open(0x2632865eb30) https://wmvideocdn-wakatest.streaming.mediaservices.windows.net/e04424e5-a90a-466c-8342-5d921d39a2f4/LOGO_FIRST.ism/QualityLevels(127999)/Fragments(aac_UND_2_127=862300000,format=mpd-time-cmaf,encryption=cenc)
> 2020-03-16 18:48:31.812 T:5760   DEBUG: AddOnLog: InputStream Adaptive: Download https://wmvideocdn-wakatest.streaming.mediaservices.windows.net/e04424e5-a90a-466c-8342-5d921d39a2f4/LOGO_FIRST.ism/QualityLevels(127999)/Fragments(aac_UND_2_127=862300000,format=mpd-time-cmaf,encryption=cenc) finished, avg speed: 11465501.54byte/s, current speed: 431692.00byte/s
> 2020-03-16 18:48:32.691 T:10904   DEBUG: Keyboard: scancode: 0x39, sym: 0x0020, unicode: 0x0020, modifier: 0x0
> 2020-03-16 18:48:32.691 T:10904   DEBUG: CInputManager::HandleKey: space (0xf020) pressed, action is Pause
> 2020-03-16 18:48:32.691 T:10904   DEBUG: ------ Window Init (DialogSeekBar.xml) ------
> 2020-03-16 18:48:32.692 T:10904   DEBUG: ------ Window Init (Custom_1109_TopBarOverlay.xml) ------
> 2020-03-16 18:48:32.705 T:2948   DEBUG: CDVDAudio::Pause - pausing audio stream
> 2020-03-16 18:48:32.823 T:10904   DEBUG: Keyboard: scancode: 0x39, sym: 0x0020, unicode: 0x0000, modifier: 0x0
> 2020-03-16 18:48:33.066 T:10904   DEBUG: Keyboard: scancode: 0x38, sym: 0x0134, unicode: 0x0000, modifier: 0x100
> 2020-03-16 18:48:33.066 T:10904   DEBUG: CInputManager::HandleKey: alt-leftalt (0x4f0d4) pressed, action is
> 2020-03-16 18:48:33.178 T:11112 WARNING: CRenderManager::WaitForBuffer - timeout waiting for buffer
glennguy commented 4 years ago

Ok thanks for that.

I've done some testing on:

Same issue although I don't get the corrupt pixels, just a frozen frame. That maybe to do with how the hardware decoding handles it.

I'm testing using the same stream as in #394 : 

#KODIPROP:inputstreamaddon=inputstream.adaptive
#KODIPROP:inputstream.adaptive.manifest_type=mpd
#KODIPROP:inputstream.adaptive.license_type=com.widevine.alpha
#KODIPROP:inputstream.adaptive.license_key=https://widevine-proxy.appspot.com/proxy||R{SSM}|
https://bitmovin-a.akamaihd.net/content/art-of-motion_drm/mpds/11331.mpd
Besur commented 4 years ago

Thanks man, we are not crazy so :p

And no matter what you do, in the same parts of video, you have this issue.

i think it's related on how drm's works, it's lot of video and audio parts coming through as you playback.

so it's possible that this issue is happening when you reach a new "parts"

glennguy commented 4 years ago

@peak3d I can't yet figure this one out but I have done some investigating

In Windows 10 x64 firefox/chrome there is an initial POST request to the license url with hex bytes 0804, with a response of 739 bytes, then the proper license request of size 5848 bytes, response 782 bytes.

In comparison in win 10 x64 Kodi the request is 1778 bytes, response 759 bytes.

We are using cdm host_10, not sure what the browsers are using.

Here is a log from chrome EME Logger: Bitmovin EME log.txt

I've done some searching through the Shaka Player source for some ideas but have not yet found what the difference could be.

peak3d commented 4 years ago

Reason is that browsers are vmp certified. Vmp requires server certificate, because of that libwidevine makes the 0804 server certificate request automatically. Kodi don't have any vmp implementation.

peak3d commented 4 years ago

Are video and audio both drm encrypted? If so, pls. Try playing only the video track.

glennguy commented 4 years ago

Thanks for the explanation, that must explain all the extra data being sent in the license request. I guess VMP isn't active for this video as we get keys?

Audio is encrypted, playing video only has the same result, some frames don't decrypt.

Watching the video on the lowest resolution, mostly when the whole scene changes (I-frame?) we only see 1 or 2 frames play then the picture freezes - it's a slide show for most of the video. Log file if it helps: kodi.log

Is there anything I can do to help debug further?

Besur commented 4 years ago

I'm glad that this issue can be fix

Besur commented 4 years ago

Nothing ?

vhvvx777 commented 4 years ago

I am having the same problem. Are there any ways to solve it, or is it already forever.

ghost commented 3 years ago

Still have this issue, "corrupt pixels" (with the same logs as https://github.com/xbmc/inputstream.adaptive/issues/393#issuecomment-600931142 )

Same issue although I don't get the corrupt pixels, just a frozen frame. That maybe to do with how the hardware decoding handles it. I'm testing using the same stream as in #394

@glennguy I'm not sure this is related to "hardware decoding", I tried to play the stream just after the CDM decryption (so before kodi decoding, as https://github.com/xbmc/inputstream.adaptive/issues/393#issuecomment-599854430 stream has sw_secure_crypto robustness only), and I still have these corrupt pixels... So I think something is messing up during decryption (not decoding), I guess ? but I don't know where... I also have a lot of error logs like this when watching the video :

DEBUG: AddOnLog: InputStream Adaptive: DecryptSampleData: Decrypt failed with error: 2 and key: EB676ABBCB345E96BBCF616630F1A3DA
ERROR: AddOnLog: InputStream Adaptive: Decrypt Sample returns failure!
DEBUG: AddOnLog: InputStream Adaptive: DecryptSampleData: Decrypt failed with error: 2 and key: EB676ABBCB345E96BBCF616630F1A3DA
ERROR: AddOnLog: InputStream Adaptive: Decrypt Sample returns failure!
DEBUG: AddOnLog: InputStream Adaptive: DecryptSampleData: Decrypt failed with error: 2 and key: EB676ABBCB345E96BBCF616630F1A3DA
ERROR: AddOnLog: InputStream Adaptive: Decrypt Sample returns failure!
DEBUG: AddOnLog: InputStream Adaptive: DecryptSampleData: Decrypt failed with error: 2 and key: EB676ABBCB345E96BBCF616630F1A3DA
ERROR: AddOnLog: InputStream Adaptive: Decrypt Sample returns failure!
DEBUG: AddOnLog: InputStream Adaptive: DecryptSampleData: Decrypt failed with error: 2 and key: EB676ABBCB345E96BBCF616630F1A3DA
ERROR: AddOnLog: InputStream Adaptive: Decrypt Sample returns failure!
DEBUG: AddOnLog: InputStream Adaptive: DecryptSampleData: Decrypt failed with error: 2 and key: EB676ABBCB345E96BBCF616630F1A3DA
ERROR: AddOnLog: InputStream Adaptive: Decrypt Sample returns failure!
glennguy commented 3 years ago

As it turns out I'm trying to solve this as we speak. You're correct in your findings, it's the decryption which is failing. @matthuisman discovered that forcing Annexb/secure path in the decrypter caps will fix the decryption issue but it reveals another - when transitioning between periods the decoder is reinitialised and the remaining video for the period is lost. On my setup the picture freezes for several seconds at the end of period (and the start of next too I think).

I'm hoping to discover how to do this soon without resorting to some sort of hack

ghost commented 3 years ago

Thank you for your answer! If I want to try the decryption fix from @matthuisman, where do I have to force the annexb/secure_path caps ?

matthuisman commented 3 years ago

Its not a fix. It was just an interesting discovery. Doing it makes everything done in SW

You uncomment the two lines here: https://github.com/xbmc/inputstream.adaptive/blob/master/wvdecrypter/wvdecrypter.cpp#L511

ghost commented 3 years ago

Thank you @matthuisman

As I like to understand everything, I have some questions : Do you know what these two capabilities are made for, what does they mean ?

Because for example for a SW_SECURE_CRYPTO L3 stream, before setting these caps, the decrypted sample had the same size as the encrypted one https://github.com/xbmc/inputstream.adaptive/blob/461202194c01d6089e2ccfb25643b47d331dc12c/src/main.cpp#L1193 after this line I checked that m_encrypted.GetDataSize() == m_sampleData.GetDataSize() and it's true

But after uncommented the two lines (forcing the caps), I always have more data (42) in m_sampleData than in m_encrypted (m_sampleData .GetDataSize() > m_encrypted.GetDataSize()) , so is the decrypter adding some sort of header or metadata related to these two capabilities ?

Also, I found that with these two caps, we enter in this condition https://github.com/xbmc/inputstream.adaptive/blob/461202194c01d6089e2ccfb25643b47d331dc12c/wvdecrypter/wvdecrypter.cpp#L995 the comment say //we can not decrypt only but it's SW_SECURE_CRYPTO so we should be able to (and without these caps, we do not enter in this condition). But is not that a problem ? because setting these caps (at least SECURE_PATH here) will fails to continue after this line https://github.com/xbmc/inputstream.adaptive/blob/Matrix/wvdecrypter/wvdecrypter.cpp#L1103 and it will not simply decrypt the sample as it should for SW_SECURE_CRYPTO, at line https://github.com/xbmc/inputstream.adaptive/blob/Matrix/wvdecrypter/wvdecrypter.cpp#L1205

matthuisman commented 3 years ago

i actually have no idea - it was just a random find. That comment is also from the original author.

was your issue fixed by uncommenting them?

ghost commented 3 years ago

was your issue fixed by uncommenting them?

Yes, it does fix the "corrupt pixels" issue, but it seems like a bit hacky, idk (but as you said it's not a fix) EDIT: only video is playing (not audio)

Maybe @peak3d could explain what happens here ?

ghost commented 3 years ago

Well, I made some more tests, and it appear that with these two caps, only video is playing (not audio) for stream https://github.com/xbmc/inputstream.adaptive/issues/393#issuecomment-599854430 Without these caps, audio is playing but we have corrupt pixels.

I tried to follow the flow of the code (with debug logs), and it appears that for this SW_SECURE_CRYPTO stream, it calls this cdm function :

But the second one should only be called for SW_SECURE_DECODE streams, not SW_SECURE_CRYPTO, imho

glennguy commented 3 years ago

@HawkLiking thanks for joining the effort to get this sorted.

The above analysis is correct.

Kodi's crypto sessions (used when SSD_SECURE_PATH is set) only support a video decoder atm so this would be why audio fails.

If you use 

if (media == SSD_DECRYPTER::SSD_CAPS::SSD_MEDIA_VIDEO)
  caps.flags |= (SSD_DECRYPTER::SSD_CAPS::SSD_SECURE_PATH | SSD_DECRYPTER::SSD_CAPS::SSD_ANNEXB_REQUIRED);

then audio should be fine.

The detection algorithm at https://github.com/xbmc/inputstream.adaptive/blob/461202194c01d6089e2ccfb25643b47d331dc12c/wvdecrypter/wvdecrypter.cpp#L540 is now maybe not fit for our purpose? The only thing we pass in here that is unique to the stream being played is the keyid, it just uses dummy data to see if decryption will be successful without annexb format.

If we change the behaviour to always set annexb/secure path for video streams then this issue is solved, but the problem mentioned above still remains.

Please give your opinion on this next part, I'm ~80% sure it's right... Content in both examples is decrypted and given to Kodi when DemuxRead() is called, then presumably put into Kodi's internal buffer. Without secure path, when it comes time for Kodi to decode it just hands it over to ffmpeg, job done. With secure path, the data is sent back to IA into CVideoCodecAdaptive for decoding, then given back to Kodi. The decoding happens some time after decrypting/demuxread (assuming this, not 100%). When starting a new period, the segments are downloaded, decrypted, and put into Kodi's buffer some seconds before the actual period change happens on screen, and because in a new period VideoCodec is reinitialised we end up losing the last few seconds of video in the period.

You can see the effect with this stream

#KODIPROP:inputstream=inputstream.adaptive
#KODIPROP:inputstream.adaptive.manifest_type=hls
#KODIPROP:inputstream.adaptive.license_type=com.widevine.alpha
#KODIPROP:inputstream.adaptive.license_key=https://proxy.uat.widevine.com/proxy||R{SSM}|
https://raw.githubusercontent.com/glennguy/wvtest/multi-period/h264_master.m3u8

Second period starts only 12 seconds in, but with the above change video freezes between roughly :05 and :12

Hacking the VideoCodec to never reinitialize (return false) after opened once makes for nice playback (at least in this stream), but will undoubtedly break when we actually need to reinitialise due to different video codec, resolution etc.

Android handles this fine but afaik it doesn't use IA's VideoCodec

glennguy commented 3 years ago

This change might also be exposing an issue with https://github.com/xbmc/inputstream.adaptive/pull/663 as I am seeing crashing now for VTM GO stream with secure_path

ghost commented 3 years ago

If we change the behaviour to always set annexb/secure path for video streams then this issue is solved

But we will no longer be able to take advantage of hardware decoding (ffmpeg) for sw_secure_crypto, am I wrong ?

With secure path, the data is sent back to IA into CVideoCodecAdaptive for decoding, then given back to Kodi.

for decrypting and decoding. When CVideoCodecAdaptive::AddData() is called, it calls m_session->GetDecrypter()->DecodeVideo() which decrypt and decode (drm_.DecryptAndDecodeFrame()) the frame in the cdm

Anyway, sorry I can't really help, I was looking for the real reason why we get a kNoKey for some samples only on 720p and not in 1080p (because I'm not sure setting secure_path, which means using DecryptAndDecodeFrames(), is the proper way for sw_secure_crypto), 720p and 1080p have exact same parameters (both sw_securecrypto, key is kUsable, which means the CDM is certain the key is currently usable for decryption, they both have the same OutputProtectionMethods and OutputLinkTypes, same license response, even the same key, ...). So I tried some funny things (I managed to get Kodi to crash several times 👍), I also tried to run `drm.GetCdmAdapter()->Decrypt()` many times until it succeed (desperate move here^^) but it always fails on the same samples...

From Mozilla source code https://searchfox.org/mozilla-central/source/dom/media/gmp/ChromiumCDMChild.cpp#737

case cdm::kNoKey:
      GMP_LOG_DEBUG("NoKey for sample at time=%" PRId64 "!", input.timestamp);

// Somehow our key became unusable. Typically this would happen when
// a stream requires output protection, and the configuration changed
// such that output protection is no longer available. For example, a
// non-compliant monitor was attached. The JS player should notice the
// key status changing to "output-restricted", and is supposed to switch
// to a stream that doesn't require OP. In order to keep the playback
// pipeline rolling, just output a black frame. See bug 1343140.

But if our stream had such an "ouput protection" enabled, we won't be able to play the stream at all, isn't it ? (for both 720p and 1080p)

glennguy commented 3 years ago

I was thinking that (regardless of the name DecryptAndDecodeFrame) decrypting still happens first because of this block here https://github.com/xbmc/inputstream.adaptive/blob/461202194c01d6089e2ccfb25643b47d331dc12c/src/main.cpp#L1207-L1212 but please correct me if that's not right.

You should also set IA settings to manual stream selection and try the lowest res - my experience shows the lower the bitrate the more decryption errors. It makes me think there might be an issue to do with padding out very small samples. I've also noticed in 1080p streams that when viewing credits or other easily compressible sequences the issue happens as well.

ghost commented 3 years ago

I was thinking that (regardless of the name DecryptAndDecodeFrame) decrypting still happens first because of this block here but please correct me if that's not right.

In fact, there is no decryption happening here in m_singleSampleDecryptor->DecryptSampleData() when the secure_path caps is set. That's what I was trying to explain in https://github.com/xbmc/inputstream.adaptive/issues/393#issuecomment-851078584

If we have secure_path caps, in the DecryptSampleData()function it will go into this if condition to prepare the sample for future work https://github.com/xbmc/inputstream.adaptive/blob/461202194c01d6089e2ccfb25643b47d331dc12c/wvdecrypter/wvdecrypter.cpp#L995-L996 and at the end of this condition, it will return with AP4_SUCCESS, without decrypting anything (the Decrypt() function call is after the condition, but we will never reach it if we have secure_path (I double checked by putting debug logs everywhere)) https://github.com/xbmc/inputstream.adaptive/blob/461202194c01d6089e2ccfb25643b47d331dc12c/wvdecrypter/wvdecrypter.cpp#L1103-L1104

I think it prepares the sample (annexb stuff and NALU parsing, and can we talk about 'sample(s) to frame' conversion here), then it is sent back to kodi (i dont know why but it seems to be the flow), and then sent back to IA through ADDON_AddData()-> CVideoCodecAdaptive::AddData() where it will finally calls drm_.DecryptAndDecodeFrame() to decrypt and decode the frame into the cdm (again, all this happen only if we set secure_path).

You should also set IA settings to manual stream selection and try the lowest res - my experience shows the lower the bitrate the more decryption errors. It makes me think there might be an issue to do with padding out very small samples. I've also noticed in 1080p streams that when viewing credits or other easily compressible sequences the issue happens as well.

Indeed... I'm also trying to understand this... I think (imho) that this would be the proper way to fix this issue: to find the problem which certainly occurs before the decryption (as we should be able to decrypt anyway (kUsable)), I'm not 100% sure, but...

glennguy commented 3 years ago

there is no decryption happening here...

Thanks for the correction, I didn't follow the code path properly

I think (imho) that this would be the proper way to fix this issue: to find the problem which certainly occurs before the decryption

Totally agreed. The more I think about this the more I believe that this is the real solution. I need to know more about the low level on how mp4 samples are structured, vps, pps, sps etc. To me when I view 180p stream of something it looks like just the key (i) frames make it through some times.

glennguy commented 3 years ago

@HawkLiking I have found the issue - https://github.com/xbmc/inputstream.adaptive/blob/461202194c01d6089e2ccfb25643b47d331dc12c/wvdecrypter/wvdecrypter.cpp#L1163

Set this to 2 and you'll get only the failed frames that now decode. When secure path is being used there's a lot of work around setting subsample_count which doesn't happen for our case, we need to figure out how to apply the same logic to determine subsample count dynamically.

ghost commented 3 years ago

you'll get only the failed frames that now decode

(did you mean "decrypt" ?)

I tried to set this to 2, but nothing has changed... I still have corrupt pixels (decryption error) like usual, maybe I am doing something wrong, do I have to set a specific flag ?

glennguy commented 3 years ago

Sorry yes I meant decrypt

I'm using this stream

#KODIPROP:inputstreamaddon=inputstream.adaptive
#KODIPROP:inputstream.adaptive.manifest_type=mpd
#KODIPROP:inputstream.adaptive.license_type=com.widevine.alpha
#KODIPROP:inputstream.adaptive.license_key=https://widevine-proxy.appspot.com/proxy||R{SSM}|
https://bitmovin-a.akamaihd.net/content/art-of-motion_drm/mpds/11331.mpd

And changing to 2 I get to see none of the frames that decrypted successfully with 1, and all of the frames that didn't.

glennguy commented 3 years ago

Ideally we end up with the line I posted above being the same as this one https://github.com/xbmc/inputstream.adaptive/blob/461202194c01d6089e2ccfb25643b47d331dc12c/wvdecrypter/wvdecrypter.cpp#L1181 we just need to do the work to correctly set subsample_count properly during each call to DecryptSampleData

matthuisman commented 3 years ago

@glennguy I tried setting to 2 and all that I see different is IA / Kodi skips the first 5 seconds. Once you get to the lady sitting down - i still get the artifacts etc.

Still only thing that fixes for me is turning off DXVA2. @glennguy @HawkLiking Have we seen the issue on any non-Windows platforms?

glennguy commented 3 years ago

Forgot to mention with the above stream - set stream selection to manual and use the lowest quality

@matthuisman The first 5 seconds decode correctly when set to 1, then the lady sitting down is full of artifacts (on my setup the image is just frozen). When set to 2 it skips the first 5 seconds and then when the lady is sitting down I get a mostly fluid picture.

In other words - setting to 1 some content/samples are decrypted properly, setting to 2 other content/samples are decrypted properly. For the low bitrate content there seems to be more subsamples than higher bitrate?

edit yes I've seen this on LibreELEC/Rpi

kszaq commented 3 years ago

@glennguy I did some testing on my side and after I removed subsample_count > 1 from this line: https://github.com/xbmc/inputstream.adaptive/blob/461202194c01d6089e2ccfb25643b47d331dc12c/wvdecrypter/wvdecrypter.cpp#L1144 I am no longer getting Decrypt Sample returns failure! on the bitmovin stream (lowest bitrate), the stream plays smoothly. With the lowest bitrate stream every sample consists of 1 subsample and it looks like even with 1 subsample we need to remove clear bytes before passing it to the CDM.

Patch for testing: https://github.com/kszaq/inputstream.adaptive/commit/097c6fff31d9e67ca5c826c12097be756755c949

ghost commented 3 years ago

Forgot to mention with the above stream - set stream selection to manual and use the lowest quality

indeed i can see the difference when tuning this parameter, great finding @glennguy 👍

I am no longer getting Decrypt Sample returns failure! on the bitmovin stream (lowest bitrate), the stream plays smoothly.

nice ! congratulations @kszaq , even in 720p i no longer have corrupt pixels with your patch 👍 Still i'm not sure to understand the real reason behind, but... i think we are close to a good patch here

matthuisman commented 3 years ago

i can confirm that also fixes more me using DXVA on windows.

i also tried cdm_in.num_subsamples = subsample_count; with the above patch but that causes missing sections etc. So, that needs to remain as 1.

Makes a bit of sense that patch. If sample count was 1, the we would never set useSingleDecrypt. It would be swapping between useSingleDecrypt with different number of samples

I guess we could set it to >0 or >= 1. I assume it was there for a reason and maybe a typo? Is it possible to have 0 subsample_count?

kszaq commented 3 years ago

@matthuisman SINGLE_DECRYPTER cap means that per every sample CDM should get 1 block of encrypted data, that we should collect encrypted data from all subsamples and merge them into one block. Every sample consists of at least 1 subsample, every subsample can have cleartext bytes and/or encrypted bytes. The subsample_count > 1 in my opinion is wrong because even if we get 1 subsample, we should remove cleartext bytes before passing the data to CDM.

I don't think we should modify this part, with any value other than 1 one of my plugins fails to play: https://github.com/xbmc/inputstream.adaptive/blob/461202194c01d6089e2ccfb25643b47d331dc12c/wvdecrypter/wvdecrypter.cpp#L1163

subsample_count will be always >=1 because of: https://github.com/xbmc/inputstream.adaptive/blob/461202194c01d6089e2ccfb25643b47d331dc12c/wvdecrypter/wvdecrypter.cpp#L1127-L1132

matthuisman commented 3 years ago

yup, thats what I found. cdm_in.num_subsamples needs to be 1.

I looked back to see when that subsample_count > 1 was commited. Its been in since the 1st commit lol. 1st commit was quite large though - so maybe it was just bringing in private repo work.

kszaq commented 3 years ago

PR created so that people can get binaries from Jenkins for testing.

glennguy commented 3 years ago

@kszaq Awesome work, and thank you heaps for the analysis! It's good for all of us to increase our knowledge of how it works.

AlecJY commented 3 years ago

I found a video still has the decrypt failed error after update inputstream.adaptive to 2.6.17

screenshot

2021-06-29 17:37:38.958 T:10128   DEBUG <general>: AddOnLog: inputstream.adaptive: DecryptSampleData: Decrypt failed with error: 2 and key: CB1B9833F8134A09A086301B6AE4D75A
2021-06-29 17:37:38.958 T:10128   ERROR <general>: AddOnLog: inputstream.adaptive: Decrypt Sample returns failure!

It seems not a serious problem because it only happened once and seems to happened rarely that I only found one video with the error.

If use the old version of inputstream.adaptive to play the video, it got many decrypt failed errors. However, if use the latest version it still got one decrypt failed error. If I set annexb and secure path flags in decrypter caps, it looks normally and without errors in the log.

I found that modify some bytes the encrypted data or fill in some dummy bytes to the clear data like https://github.com/AlecJY/inputstream.adaptive/commit/9229ca1b8d1dc472e2e2d4a0cac3c6df21e3d97f will solve the error. I guess that there are some mechanisms inside the cdm to check that some data the cdm doesn't want to decrypt. Therefore, add or modify the input data may bypass the checks. The patch solve my problem in the video. Then I test the patch with various videos. It seems not cause additional problems. But I'm not sure why it works and don't know if it's a proper fix of the issue.

matthuisman commented 3 years ago

@AlecJY Thank you for your investigation work! Is this a public stream that you could provide to allow testing?

@kszaq could you please have a look at above comment / patch and see what you think?

Possibly it's not the right solution but gives you a clue to what may be causing it.

Its not a regression, just another troublesome stream that would be nice to fix as well

AlecJY commented 3 years ago

@matthuisman The stream is on a VOD platform in Taiwan. I'm not sure if it has geo-blocking and it needs pay 79TWD for renting 48 hr.

https://video.friday.tw/movie/detail/65783 It happens at around 00:23:53

kszaq commented 3 years ago

@matthuisman I'll have a look - it seems like in some cases we need to not remove the clear bytes...

kszaq commented 3 years ago

@AlecJY I am unable to play the video from the link so I was looking at your patch - it is in some way "reverting" https://github.com/xbmc/inputstream.adaptive/pull/709 - you are always adding clear bytes and #709 prevents it. I am a bit surprised that without #709 you have more artifacts and now only some places are not working...

Can you try changing this line: https://github.com/xbmc/inputstream.adaptive/blob/Matrix/wvdecrypter/wvdecrypter.cpp#L1190 to if (true) and see if artifacts are gone - without your + 20 patch? Also, would you be able to log subsamples, bytes_of_cleartext_data[0] and bytes_of_cleartext_data[0] for the sample that fails to decrypt?

matthuisman commented 3 years ago

@AlecJY Are you using android by any chance?

If so, try a IA build from here: https://jenkins.kodi.tv/blue/organizations/jenkins/xbmc%2Finputstream.adaptive/detail/PR-734/1/artifacts

kszaq commented 3 years ago

@matthuisman From the screenshot it looks like @AlecJY is using Windows.

AlecJY commented 3 years ago

@matthuisman I mostly use Kodi on Windows. The video works well on my Android phone with inputstream.adaptive from the official repo or you provided.

@kszaq I changed the line to if (true) and the issue still exists.

I try to print subsample_count, bytes_of_cleartext_data[0], bytes_of_encrypted_data[0] to the log when the decryption is failed.

2021-07-15 14:50:03.282 T:18632   DEBUG <general>: AddOnLog: inputstream.adaptive: DecryptSampleData: inputparams invalid
2021-07-15 14:50:03.282 T:18632   DEBUG <general>: AddOnLog: inputstream.adaptive: DecryptSampleData: Decrypt failed with error: 2 and key: CB1B9833F8134A09A086301B6AE4D75A
2021-07-15 14:50:03.282 T:18632   DEBUG <general>: AddOnLog: inputstream.adaptive: subsample_count: 1
2021-07-15 14:50:03.282 T:18632   DEBUG <general>: AddOnLog: inputstream.adaptive: bytes_of_cleartext_data[0]: 22
2021-07-15 14:50:03.282 T:18632   DEBUG <general>: AddOnLog: inputstream.adaptive: bytes_of_encrypted_data[0]: 18320
2021-07-15 14:50:03.282 T:18632   ERROR <general>: AddOnLog: inputstream.adaptive: Decrypt Sample returns failure!

And the value of data_in in hex string data_in.log

kszaq commented 3 years ago

@AlecJY Are both video and audio encrypted in your video? If so, can you try using approach similar to https://github.com/xbmc/inputstream.adaptive/pull/734/files, that is always return false: https://github.com/xbmc/inputstream.adaptive/blob/Matrix/wvdecrypter/wvdecrypter.cpp#L1469-L1474

I suspect there are 2 encrypted streams because with one stream you should have Decrypt Sample returns failure! right after DecryptSampleData: inputparams invalid but I can see there is another DecryptSampleData call in between, which is a bit confusing... https://github.com/xbmc/inputstream.adaptive/blob/Matrix/src/main.cpp#L1192-L1195 https://github.com/xbmc/inputstream.adaptive/blob/Matrix/wvdecrypter/wvdecrypter.cpp#L1150-L1155

AlecJY commented 3 years ago

@kszaq I'm terribly sorry. I found that I accidently copy Log(SSD_HOST::LL_DEBUG, "DecryptSampleData: inputparams invalid"); to the line https://github.com/xbmc/inputstream.adaptive/blob/931f274b47bdafc7532b174dc6f519b264d81f79/wvdecrypter/wvdecrypter.cpp#L1255 while debugging.

I regenerate the log with the right code data_in.log

And here is my code to print the debug messages https://github.com/AlecJY/inputstream.adaptive/commit/2c53d0e1bcfa2607f1d3310474d65cc0273088fe