search

axiomatic-systems / Bento4

Full-featured MP4 format, MPEG DASH, HLS, CMAF SDK and tools
http://www.bento4.com
1.97k stars 479 forks source link

Cannot play encoded video in dash.js #134

Open boris-petrov opened 7 years ago

boris-petrov commented 7 years ago

Using:

ffmpeg - version 3.2.2
mp4fragment - version 1.6.0 (Bento4 Version 1.5.0.0)
mp4-dash - version 1.7.0 r614
dash.js - version 2.4.1
chromium - version 57.0.2987.133

I'm using the following commands:

ffmpeg -i lego.mp4 -f mp4 -ar 44100 -ab 96k -pix_fmt yuv420p -filter:a aformat=s16:44100:stereo -vcodec libx264 -crf 25 -g 250 -r 25 -coder 1 -flags +loop -filter:v scale=min(640\,iw):trunc(ow/a/2)*2 -strict -2 lego-transcoded.mp4
mp4fragment lego-transcoded.mp4 lego-fragmented.mp4
mp4-dash --force --profiles=on-demand --output-dir=output --encryption-key=aa9555190e5e41cfa08b8b688fb6e5ea:D241E2386B5A91E567389AC2E29A6D09 lego-fragmented.mp4

The first one to transcode the video in a web-friendly way (in this case it is not needed but we have to work with any kind of video). The second and third should be self-explanatory.

Result MPD is of the form:

<?xml version="1.0"?>
<MPD xmlns="urn:mpeg:dash:schema:mpd:2011" xmlns:cenc="urn:mpeg:cenc:2013" mediaPresentationDuration="PT5.600S" minBufferTime="PT5.60S" profiles="urn:mpeg:dash:profile:isoff-on-demand:2011" type="static">
  <!-- Created with Bento4 mp4-dash.py, VERSION=1.7.0-614 -->
  <Period>
    <!-- Video -->
    <AdaptationSet maxHeight="320" maxWidth="560" mimeType="video/mp4" minHeight="320" minWidth="560" segmentAlignment="true" startWithSAP="1">
      <!-- MPEG Common Encryption -->
      <ContentProtection cenc:default_KID="aa955519-0e5e-41cf-a08b-8b688fb6e5ea" schemeIdUri="urn:mpeg:dash:mp4protection:2011" value="cenc" />
      <Representation bandwidth="159215" codecs="avc1.640015" frameRate="25" height="320" id="video-avc1" scanType="progressive" width="560">
        <BaseURL>/api/objects/lego-drmVideoStream.mp4</BaseURL>
        <SegmentBase indexRange="811-854">
          <Initialization range="0-810" />
        </SegmentBase>
      </Representation>
    </AdaptationSet>
    <!-- Audio -->
    <AdaptationSet lang="en" mimeType="audio/mp4" segmentAlignment="true" startWithSAP="1">
      <!-- MPEG Common Encryption -->
      <ContentProtection cenc:default_KID="aa955519-0e5e-41cf-a08b-8b688fb6e5ea" schemeIdUri="urn:mpeg:dash:mp4protection:2011" value="cenc" />
      <Representation audioSamplingRate="44100" bandwidth="108106" codecs="mp4a.40.2" id="audio-en-mp4a">
        <AudioChannelConfiguration schemeIdUri="urn:mpeg:dash:23003:3:audio_channel_configuration:2011" value="2" />
        <BaseURL>/api/objects/lego-drmAudioStream.mp4</BaseURL>
        <SegmentBase indexRange="751-818">
          <Initialization range="0-750" />
        </SegmentBase>
      </Representation>
    </AdaptationSet>
  </Period>
</MPD>

But then when I try playing this in dash.js, only a black screen is shown and the video doesn't play. Logs from dash.js:

[10] [dash.js 2.4.1] MediaPlayer has been initialized
[13] EME detected on this user agent! (ProtectionModel_21Jan2015)
[20] Playback Initialized
[534] Parsing complete: ( xml2json: 5.55ms, objectiron: 2.19ms, total: 0.00774s)
[547] MediaSource attached to element.  Waiting on open...
[548] Manifest has been refreshed at Sun Apr 02 2017 09:51:30 GMT+0300 (EEST)[1491115890.189]
[989] MediaSource is open!
[990] Duration successfully set to: 5.6
[991] Added 0 inline events
[992] video codec: video/mp4;codecs="avc1.640015"
[1005] Schedule controller stopping for video
[1009] Perform SIDX load: http://192.168.88.111:4200/api/objects/lego-drmVideoStream.mp4
[1009] audio codec: audio/mp4;codecs="mp4a.40.2"
[1015] Schedule controller stopping for audio
[1016] Perform SIDX load: http://192.168.88.111:4200/api/objects/lego-drmAudioStream.mp4
[1017] No text data.
[1018] No fragmentedText data.
[1018] No embeddedText data.
[1018] No muxed data.
[1021] Start Event Controller
[1189] Parsing segments from SIDX.
[1199] Parsing segments from SIDX.
[1202] Schedule controller starting for video
[1204] Schedule controller starting for audio
[1259] Init fragment finished loading saving to audio's init cache
[1263] Init fragment finished loading saving to video's init cache
[1270] Native video element event: loadedmetadata
[1279] Seeking to: 0
[1283] Top qualityaudio index has changed from undefined to 0
[1286] AbrController (audio) stay on 0/0 (buffer: 0)
[1287] Getting the request for audio time : 0
[1288] SegmentBase: 0 / 5.6
[1292] Top qualityvideo index has changed from undefined to 0
[1293] AbrController (video) stay on 0/0 (buffer: 0)
[1293] Getting the request for video time : 0
[1294] SegmentBase: 0 / 5.6
[1330] Buffered Range for type: audio : 0.001882  -  1.996915
[1333] Got enough buffer to start.
[1334] Requesting seek to time: 0.001882
[1346] Seeking to: 0.001882
[1354] ThroughputRule requesting switch to index:  0 type:  audio Average throughput 5815 kbps
[1355] AbrController (audio) stay on 0/0 (buffer: 1.995)
[1356] Prior to making a request for time, NextFragmentRequestRule is aligning index handler's currentTime with bufferedRange.end. 0.001882  was changed to  1.996915
[1356] Getting the request for audio time : 1.996915
[1357] SegmentBase: 0 / 5.6
[1358] Getting the next request at index: 1
[1359] SegmentBase: 1.9969160997732427 / 5.6
[1364] Got enough buffer to start.
[1373] Buffered Range for type: video : 0  -  5.6
[1380] ThroughputRule requesting switch to index:  0 type:  video Average throughput 221192 kbps
[1380] AbrController (video) stay on 0/0 (buffer: 5.598)
[1381] Prior to making a request for time, NextFragmentRequestRule is aligning index handler's currentTime with bufferedRange.end. 0.001882  was changed to  5.6
[1381] Getting the request for video time : 5.6
[1381] SegmentBase: 0 / 5.6
[1381] Getting the next request at index: 1
[1382] Signal complete.
[1382] Schedule controller stopping for video
[1383] Stream is complete
[1423] Buffered Range for type: audio : 0.001882  -  3.993831
[1426] ThroughputRule requesting switch to index:  0 type:  audio Average throughput 3439 kbps
[1427] AbrController (audio) stay on 0/0 (buffer: 3.992)
[1427] Getting the request for audio time : 3.9938321995464854
[1428] Index for audio time 3.9938321995464854 is 1
[1428] SegmentBase: 1.9969160997732427 / 5.6
[1428] Getting the next request at index: 2
[1428] SegmentBase: 3.9938321995464854 / 5.6
[1460] Buffered Range for type: audio : 0.001882  -  5.569886
[1462] ThroughputRule requesting switch to index:  0 type:  audio Average throughput 5707 kbps
[1463] AbrController (audio) stay on 0/0 (buffer: 5.568)
[1463] Getting the request for audio time : 5.569886621315193
[1463] Index for audio time 5.569886621315193 is 2
[1464] SegmentBase: 3.9938321995464854 / 5.6
[1464] Getting the next request at index: 3
[1465] Signal complete.
[1465] Schedule controller stopping for audio
[1466] Stream is complete
[2752] Native video element event: play
[6720] Native video element event: pause

When using the same file with MP4Box it plays fine (so I think the configuration of dash.js is correct - however we have some video files for which MP4Box generates a broken MPD file and that's why we switched to Bento4). The commands we use there:

... extract video and audio streams with ffmpeg from lego-transcoded.mp4 in lego-video-stream.mp4 and lego-audio-stream.mp4
MP4Box -crypt drm.key lego-video-stream.mp4 -out lego-encrypted.mp4
MP4Box -dash 10000 -profile onDemand -out lego.mpd lego-encrypted.mp4 lego-audio-stream.mp4

We use it slightly differently here - we encode only the video stream and include an unencrypted audio stream but otherwise I think it should be the same.

And the MPD generated:

<?xml version="1.0"?>
<!-- MPD file Generated with GPAC version 0.6.1-revrelease  at 2017-04-02T07:10:07.321Z-->
<MPD xmlns="urn:mpeg:dash:schema:mpd:2011" xmlns:cenc="urn:mpeg:cenc:2013" minBufferTime="PT1.500S" type="static" mediaPresentationDuration="PT0H0M5.601S" maxSegmentDuration="PT0H0M5.600S" profiles="urn:mpeg:dash:profile:isoff-on-demand:2011,http://dashif.org/guidelines/dash264">
 <ProgramInformation moreInformationURL="http://gpac.sourceforge.net">
  <Title>lego.mpd generated by GPAC</Title>
 </ProgramInformation>

 <Period duration="PT0H0M5.601S">
  <AdaptationSet segmentAlignment="true" maxWidth="560" maxHeight="320" maxFrameRate="25" par="560:320" lang="und" subsegmentAlignment="true" subsegmentStartsWithSAP="1">
   <ContentProtection schemeIdUri="urn:mpeg:dash:mp4protection:2011" value="cenc" cenc:default_KID="70f13f28-611b-4484-9d9b-679b1b68e4e6" />
   <Representation id="1" mimeType="video/mp4" codecs="avc1.640015" width="560" height="320" frameRate="25" sar="1:1" startWithSAP="1" bandwidth="157277">
    <BaseURL>/api/objects/lego-drmVideoStream.mp4</BaseURL>
    <SegmentBase indexRangeExact="true" indexRange="1091-1134">
      <Initialization range="0-1090" />
    </SegmentBase>
   </Representation>
  </AdaptationSet>
  <AdaptationSet segmentAlignment="true" lang="eng" subsegmentAlignment="true" subsegmentStartsWithSAP="1">
   <Representation id="2" mimeType="audio/mp4" codecs="mp4a.40.2" audioSamplingRate="44100" startWithSAP="1" bandwidth="99536">
    <AudioChannelConfiguration schemeIdUri="urn:mpeg:dash:23003:3:audio_channel_configuration:2011" value="2" />
    <BaseURL>/api/objects/lego-audio-stream.mp4</BaseURL>
    <SegmentBase indexRangeExact="true" indexRange="835-902">
      <Initialization range="0-834" />
    </SegmentBase>
   </Representation>
  </AdaptationSet>
 </Period>
</MPD>

I could upload the test video and any other code/configuration/commands that we use if needed. What are we doing wrong?

P.S. Forgot to mention - we're using clearkey encryption with video.js, dash.js and videojs-contrib-dash - all are the latest versions. Sample code:

clearkeys = {}
drmKeys.forEach (drmKey) =>
  kid = @_toBase64(drmKey.kid.replace(/-/g, ''))
  key = @_toBase64(drmKey.k)
  clearkeys[kid] = key

mplayer.src
  src: '/api/objects/lego-dashDrmContent.mpd'
  type: 'application/dash+xml'
  keySystemOptions: [
    name: 'org.w3.clearkey', options: { clearkeys }
  ]
barbibulle commented 7 years ago

I'll try to reproduce this and get back to you shortly.

boris-petrov commented 7 years ago

@barbibulle - any progress on this?

barbibulle commented 7 years ago

Hi. Sorry, I forgot I needed to look at this. Now that I did read through your post, I realize that you're asking a question for a different software package. The 'MP4Box' tool you're using here isn't part of the Bento4 software package, it is part of GPAC, which is something different.

boris-petrov commented 7 years ago

Hi, of course I know MP4Box is from another package - I was giving it as a comparison. In my post I first give an example with Bento4 which doesn't work and then I give a working example with gpac. I hope I'm clear enough now, please reread my initial post. If you can't understand something from there, I could try explaining it again. :) I could also upload the sample video I test with if that's needed.