Low latency Dash+DRM playback issue.

codeindream commented 6 years ago

Issue description

Hello!

We'd like to use Exoplayer for DASH+DRM live streaming with latency ~8 sec and timeShiftBuffer = 5M. Priority is to keep stream latency ~8 sec from realtime (skip frames if required).

Our manifest working well for Shaka player, but with Exo we observs next issues:

After first chunk received, player show seek bar for 5M with position at the end. Content not playing, only init and first chunk downloaded.
than each ~5 sec seek position moved back to ~4:52 and content start/continue playing
if buffering state occured during playback, player waiting for new chunks and continue to play from last place. But we expecting to keep ~8sec latency from realtime, by skipping frames, if required.
As latency very low, 404 is often and expected state for chunk requests. But segments get to blacklist and after some time, lowest quality selected.

Reproduction steps

Update Exoplayer Demo with links to manifest and widevine license (will provide by email).

Configure LoadControl:

new DefaultLoadControl(
          new DefaultAllocator(true, C.DEFAULT_BUFFER_SEGMENT_SIZE),
          4000, 6000, 2000, 3000, 
          C.LENGTH_UNSET, true
  )

DASH manifest example:

<?xml version="1.0"?>
<MPD xmlns="urn:mpeg:dash:schema:mpd:2011" xmlns:cenc="urn:mpeg:cenc:2013" minBufferTime="PT2S" type="dynamic" publishTime="2018-01-22T03:09:49.244Z" availabilityStartTime="2018-01-22T01:43:31.878Z" timeShiftBufferDepth="PT5M" suggestedPresentationDelay="P0D" minimumUpdatePeriod="PT96H" maxSegmentDuration="PT3S" profiles="urn:mpeg:dash:profile:isoff-live:2011">
    <Period id="p0" start="PT0H0M0.000S">
        <AdaptationSet segmentAlignment="true" bitstreamSwitching="true" subsegmentStartsWithSAP="1">
            <Representation id="fhd" mimeType="video/mp4" codecs="avc1.640029" startWithSAP="1" height="1080" bandwidth="3000000" >
                <SegmentTemplate timescale="1000" media="fhd-$Number%05d$.m4s" startNumber="0" duration="2000" initialization="fhd-init.mp4"/>
            </Representation>
            <Representation id="720" mimeType="video/mp4" codecs="avc1.42c01f" startWithSAP="1" height="720" bandwidth="1600000" >
                <SegmentTemplate timescale="1000" media="720-$Number%05d$.m4s" startNumber="0" duration="2000" initialization="720-init.mp4"/>
            </Representation>
            <Representation id="480" mimeType="video/mp4" codecs="avc1.42c01f" startWithSAP="1" height="480" bandwidth="830000" >
                <SegmentTemplate timescale="1000" media="480-$Number%05d$.m4s" startNumber="0" duration="2000" initialization="480-init.mp4"/>
            </Representation>
            <Representation id="360" mimeType="video/mp4" codecs="avc1.42c01f" startWithSAP="1" height="360" bandwidth="520000" >
                <SegmentTemplate timescale="1000" media="360-$Number%05d$.m4s" startNumber="0" duration="2000" initialization="360-init.mp4"/>
            </Representation>
            <Representation id="240" mimeType="video/mp4" codecs="avc1.42c01f" startWithSAP="1" height="240" bandwidth="350000" >
                <SegmentTemplate timescale="1000" media="240-$Number%05d$.m4s" startNumber="0" duration="2000" initialization="240-init.mp4"/>
            </Representation>
            <Representation id="144" mimeType="video/mp4" codecs="avc1.42c01e" startWithSAP="1" height="144" bandwidth="80000" >
                <SegmentTemplate timescale="1000" media="144-$Number%05d$.m4s" startNumber="0" duration="2000" initialization="144-init.mp4"/>
            </Representation>
        </AdaptationSet>
        <AdaptationSet segmentAlignment="true" bitstreamSwitching="true" subsegmentStartsWithSAP="1">
            <Representation id="alo" mimeType="audio/mp4" codecs="mp4a.40.2" startWithSAP="1" bandwidth="64000" audioSamplingRate="48000">
                <AudioChannelConfiguration schemeIdUri="urn:mpeg:dash:23003:3:audio_channel_configuration:2011" value="2"/>
                <SegmentTemplate timescale="1000" media="alo-$Number%05d$.m4s" startNumber="0" duration="2000" initialization="alo-init.mp4"/>
            </Representation>
        </AdaptationSet>
    </Period>
    <UTCTiming schemeIdUri="urn:mpeg:dash:utc:direct:2014" value="2018-01-22T03:09:49.244Z" />
</MPD>

Version of ExoPlayer being used

Exoplayer v2.6.1

Device(s) and version(s) of Android being used

Samsung: s5 (5.0), Note 5 (7.0) Emulator x86 (7.1.1) Xiaome Redmi 4 (7.0)

codeindream commented 6 years ago

email sent

linhai326 commented 6 years ago

my understanding of "availabilityStartTime" is the time since Epoch time. Should it be something like "availabilityStartTime="1970-01-01T00:00:09Z" ? in that case, it will give a 9 seconds latency from current live point.

codeindream commented 6 years ago

Should it be something like "availabilityStartTime="1970-01-01T00:00:09Z" ?

My understanding, it's time when stream started. For example, in DashMediaSource#processManifest(): long liveStreamDurationUs = getNowUnixTimeUs() - C.msToUs(manifest.availabilityStartTimeMs);

Regarding issue _#1, I think Exoplayer expecting that livestream couldn't be played with suggestedPresentationDelay = 0, as it mean that we at livestream edge, so chunks phisicaly can't be ready. But in that case, why it starting to play after 5 sec and continue?! In DashMediaSource I found logic: handler.postDelayed(simulateManifestRefreshRunnable, NOTIFY_MANIFEST_INTERVAL_MS); where NOTIFY_MANIFEST_INTERVAL_MS = 5000. This is the reason of issue _#2. But why there's difference for first and next iterations?

codeindream commented 6 years ago

Ok, so I found some answers on my questions:

After first chunk received, player show seek bar for 5M with position at the end. Content not playing, only init and first chunk downloaded.

When Exo get live stream, initial playback duration calculated as suggestedPresentationDelay + timeShiftBufferDepth with start position at playback end - suggestedPresentationDelay. So, when we load manifest from question, playback duration = 5m with start at the end -> nothing to play.

than each ~5 sec seek position moved back to ~4:52 and content start/continue playing

Exo has internal manifest reload, as I understand, to reload current playback position. As previous position was at the end of stream, and 5 sec pass, we start to play from 4:55~4:52.

if buffering state occur during playback, player waiting for new chunks and continue to play from last place. But we expecting to keep ~8sec latency from realtime, by skipping frames, if required.

This is fixed default behavior. Looks like for another behavior we need to customize Exo.

As latency very low, 404 is often and expected state for chunk requests. But segments get to blacklist and after some time, lowest quality selected.

Blacklisting can't be disabled. Could be achieved by overloading DefaultDashChunkSource.onChunkLoadError() and check if ChunkedTrackBlacklistUtil.shouldBlacklist(e) = true return false.

google / ExoPlayer