Freeze/Buffering on dash live multi-period stream for ad insertion

jgbayser commented 7 months ago

Have you read the FAQ and checked for duplicate open issues? YES

If the problem is related to FairPlay, have you read the tutorial? Not related to fairplay

What version of Shaka Player are you using? 4.7.11

Can you reproduce the issue with our latest release version? YES

Can you reproduce the issue with the latest code from main? Did not test that

Are you using the demo app or your own custom app? We are using our own custom application

If custom app, can you reproduce the issue using our demo app? Did not test that

What browser and OS are you using? Chromium 123.0.6312

For embedded devices (smart TVs, etc.), what model and firmware version are you using? Reproduced on all Tizen TV 2020, 2021, 2022

What are the manifest and license server URIs? URI sent MP

What configuration are you using? What is the output of player.getConfiguration()?

{
    "drm": {
        "keepMediaKeys": true,
        "retryParameters": {
            "maxAttempts": 5,
            "baseDelay": 1000,
            "backoffFactor": 2,
            "fuzzFactor": 0.5,
            "timeout": 10000,
            "stallTimeout": 5000,
            "connectionTimeout": 10000,
            "autoReset": false
        },
        "servers": {
            "com.widevine.alpha": "*****",
            "com.apple.fps.1_0": "*****"
        },
        "clearKeys": {},
        "advanced": {
            "com.widevine.alpha": {
                "distinctiveIdentifierRequired": false,
                "persistentStateRequired": false,
                "videoRobustness": "",
                "audioRobustness": "",
                "sessionType": "",
                "serverCertificate": {
                    *****
                },
                "serverCertificateUri": "",
                "individualizationServer": ""
            }
        },
        "delayLicenseRequestUntilPlayed": false,
        "logLicenseExchange": false,
        "updateExpirationTime": 1,
        "preferredKeySystems": [],
        "keySystemsMapping": {},
        "parseInbandPsshEnabled": false
    },
    "manifest": {
        "retryParameters": {
            "maxAttempts": 5,
            "baseDelay": 1000,
            "backoffFactor": 2,
            "fuzzFactor": 0.5,
            "timeout": 10000,
            "stallTimeout": 5000,
            "connectionTimeout": 10000,
            "autoReset": false
        },
        "availabilityWindowOverride": null,
        "disableAudio": false,
        "disableVideo": false,
        "disableText": false,
        "disableThumbnails": false,
        "defaultPresentationDelay": 12,
        "segmentRelativeVttTiming": false,
        "dash": {
            "clockSyncUri": "",
            "ignoreDrmInfo": false,
            "disableXlinkProcessing": false,
            "xlinkFailGracefully": false,
            "ignoreMinBufferTime": true,
            "autoCorrectDrift": true,
            "initialSegmentLimit": 1000,
            "ignoreSuggestedPresentationDelay": true,
            "ignoreEmptyAdaptationSet": false,
            "ignoreMaxSegmentDuration": false,
            "keySystemsByURI": {
                "urn:uuid:1077efec-c0b2-4d02-ace3-3c1e52e2fb4b": "org.w3.clearkey",
                "urn:uuid:e2719d58-a985-b3c9-781a-b030af78d30e": "org.w3.clearkey",
                "urn:uuid:edef8ba9-79d6-4ace-a3c8-27dcd51d21ed": "com.widevine.alpha",
                "urn:uuid:9a04f079-9840-4286-ab92-e65be0885f95": "com.microsoft.playready",
                "urn:uuid:79f0049a-4098-8642-ab92-e65be0885f95": "com.microsoft.playready",
                "urn:uuid:f239e769-efa3-4850-9c16-a903c6932efb": "com.adobe.primetime"
            }
        },
        "hls": {
            "ignoreTextStreamFailures": false,
            "ignoreImageStreamFailures": false,
            "defaultAudioCodec": "mp4a.40.2",
            "defaultVideoCodec": "avc1.42E01E",
            "ignoreManifestProgramDateTime": false,
            "mediaPlaylistFullMimeType": "video/mp2t; codecs=\"avc1.42E01E, mp4a.40.2\""
        }
    },
    "streaming": {
        "retryParameters": {
            "maxAttempts": 5,
            "baseDelay": 1000,
            "backoffFactor": 2,
            "fuzzFactor": 0.5,
            "timeout": 10000,
            "stallTimeout": 5000,
            "connectionTimeout": 10000,
            "autoReset": false
        },
        "rebufferingGoal": 0.01,
        "bufferingGoal": 4,
        "bufferBehind": 1,
        "ignoreTextStreamFailures": true,
        "alwaysStreamText": false,
        "startAtSegmentBoundary": false,
        "gapDetectionThreshold": 0.5,
        "durationBackoff": 1,
        "forceTransmuxTS": false,
        "safeSeekOffset": 5,
        "stallEnabled": true,
        "stallThreshold": 1,
        "stallSkip": 0,
        "useNativeHlsOnSafari": true,
        "forceNativeHlsOnSafari": false,
        "inaccurateManifestTolerance": 1,
        "lowLatencyMode": false,
        "autoLowLatencyMode": false,
        "forceHTTPS": false,
        "preferNativeHls": false,
        "updateIntervalSeconds": 1,
        "dispatchAllEmsgBoxes": false,
        "observeQualityChanges": false,
        "maxDisabledTime": 10,
        "smarttv": true
    },
    "offline": {
        "usePersistentLicense": true,
        "numberOfParallelDownloads": 5
    },
    "abr": {
        "enabled": true,
        "useNetworkInformation": true,
        "defaultBandwidthEstimate": 2000000,
        "defaultBandwidthCount": 8,
        "fastSwitchCount": 0,
        "downCountRestriction": 3,
        "downDurationRestriction": 60,
        "switchInterval": 4,
        "bandwidthUpgradeTarget": 0.85,
        "bandwidthDowngradeTarget": 0.95,
        "restrictions": {
            "minWidth": 0,
            "maxWidth": null,
            "minHeight": 0,
            "maxHeight": null,
            "minPixels": 0,
            "maxPixels": null,
            "minFrameRate": 0,
            "maxFrameRate": null,
            "minBandwidth": 0,
            "maxBandwidth": null
        },
        "advanced": {
            "minTotalBytes": 128000,
            "minBytes": 16000,
            "fastHalfLife": 2,
            "slowHalfLife": 5
        },
        "restrictToElementSize": false,
        "ignoreDevicePixelRatio": false
    },
    "preferredAudioLanguage": "fr",
    "preferredTextLanguage": "fr",
    "preferredVariantRole": "",
    "preferredTextRole": "",
    "preferredAudioChannelCount": 2,
    "preferredVideoCodecs": [],
    "preferredAudioCodecs": [],
    "preferForcedSubs": false,
    "preferredDecodingAttributes": [],
    "restrictions": {
        "minWidth": 0,
        "maxWidth": null,
        "minHeight": 0,
        "maxHeight": null,
        "minPixels": 0,
        "maxPixels": null,
        "minFrameRate": 0,
        "maxFrameRate": null,
        "minBandwidth": 0,
        "maxBandwidth": null
    },
    "playRangeStart": 0,
    "playRangeEnd": null,
    "cmcd": {
        "enabled": false,
        "sessionId": "",
        "contentId": "",
        "useHeaders": false
    }
}

What did you do? We are analysing endurance (12h) playback tests for Dash live multi-period streams on Tizen TVs

What did you expect to happen? We expect the performance to be equivalent to mono-period production streams

What actually happened? Stats and logs are showing multiple buffering and seeks after approximatively 4h of playback.

During our analysis we have found two specificities in the multi-period stream related to the slicing :

The audio and video tracks do have occasional gaps and overlaps
The audio and video tracks do not have the same duration for a given period and therefore the audio track ends after the period with a variable delta (cf schema)

Please find attached and example of our MPD

151223_0047_1.mpd.zip

jgbayser commented 6 months ago

A small precision about the context : the stream is split in multiple period to prepare for ad replacement. So we have about 10-30s periods grouped in 5min frames

jgbayser commented 5 months ago

We are trying to make the user experience as smooth as possible when playing a multi-period stream for ads replacement. While periods in manifests are expected to be adjacents one another and timeline to be continuous, gaps or overlap may happen in some real life cases. A major use case for multiperiod is for substitutions, such as ads replacements. One can understand it is up to the provisioning platform to avoid those. However, resilience of client side players to imperfect cases is what has enabled intermet streaming in the first place and opened new use cases. On large deployments, the ability to act on all components of the system may be limited, too.

Proposal #1 - Skip on gaps inside or between periods We have experienced player pauses when a manifest would have large gaps in it (between 3 and 6s). Cf following examples : image (27) We seek the following option to be available in Shaka Player: skip through time in case of gap between periods.

Proposal #2 - Automatic merge on multi period In our deployment use cases, we have found several cases with gaps and overlaps between periods coming from some equipments., while technically segments across periods follow each others. We seek the following option to be available in Shaka Player, disabled by default: merge adjacent periods when representation are identical (codec, origin, framerate...etc).

jgbayser commented 4 months ago

The behavior is unchanged with v4.9.2

avelad commented 4 months ago

Can you test if the problem still occurs with version 4.10.6? Thanks! We have added support to https://github.com/shaka-project/shaka-player/pull/6896 recently

shaka-bot commented 3 months ago

Closing due to inactivity. If this is still an issue for you or if you have further questions, the OP can ask shaka-bot to reopen it by including @shaka-bot reopen in a comment.

jgbayser commented 3 months ago

We have run some test with 4.10.7 and the issue is still reproduced

shaka-project / shaka-player

Freeze/Buffering on dash live multi-period stream for ad insertion #6409