shaka-project / shaka-player

JavaScript player library / DASH & HLS client / MSE-EME player
Apache License 2.0
7.11k stars 1.33k forks source link

HLS Playlist with mixed codecs causes CHUNK_DEMUXER_ERROR_APPEND_FAILED when switching from AV1 to other codec #5946

Closed Lubjan closed 10 months ago

Lubjan commented 10 months ago

Have you read the FAQ and checked for duplicate open issues? Yes

What version of Shaka Player are you using? 4.6.1 as of testing

Can you reproduce the issue with our latest release version? Yes

Can you reproduce the issue with the latest code from main? Yes

Are you using the demo app or your own custom app? Demo App

What browser and OS are you using? Chrome 119.0.6045.160 (other Browsers apply as well, Edge doesn't list AV1 Tracks) Windows 11 22H2

What are the manifest and license server URIs? https://01.cdn.vod.farm/testing/b2ac102ac3f1434504bf5e43d957bc8b/master.m3u8

What configuration are you using? What is the output of player.getConfiguration()?

Demo configuration (didn't change) ```json { "drm": { "retryParameters": { "maxAttempts": 2, "baseDelay": 1000, "backoffFactor": 2, "fuzzFactor": 0.5, "timeout": 30000, "stallTimeout": 5000, "connectionTimeout": 10000 }, "servers": {}, "clearKeys": {}, "advanced": {}, "delayLicenseRequestUntilPlayed": false, "persistentSessionOnlinePlayback": false, "persistentSessionsMetadata": [], "logLicenseExchange": false, "updateExpirationTime": 1, "preferredKeySystems": [], "keySystemsMapping": {}, "parseInbandPsshEnabled": false, "minHdcpVersion": "", "ignoreDuplicateInitData": true }, "manifest": { "retryParameters": { "maxAttempts": 2, "baseDelay": 1000, "backoffFactor": 2, "fuzzFactor": 0.5, "timeout": 30000, "stallTimeout": 5000, "connectionTimeout": 10000 }, "availabilityWindowOverride": null, "disableAudio": false, "disableVideo": false, "disableText": false, "disableThumbnails": false, "defaultPresentationDelay": 0, "segmentRelativeVttTiming": false, "raiseFatalErrorOnManifestUpdateRequestFailure": false, "dash": { "clockSyncUri": "", "ignoreDrmInfo": false, "disableXlinkProcessing": false, "xlinkFailGracefully": false, "ignoreMinBufferTime": false, "autoCorrectDrift": true, "initialSegmentLimit": 1000, "ignoreSuggestedPresentationDelay": false, "ignoreEmptyAdaptationSet": false, "ignoreMaxSegmentDuration": false, "keySystemsByURI": { "urn:uuid:1077efec-c0b2-4d02-ace3-3c1e52e2fb4b": "org.w3.clearkey", "urn:uuid:e2719d58-a985-b3c9-781a-b030af78d30e": "org.w3.clearkey", "urn:uuid:edef8ba9-79d6-4ace-a3c8-27dcd51d21ed": "com.widevine.alpha", "urn:uuid:9a04f079-9840-4286-ab92-e65be0885f95": "com.microsoft.playready", "urn:uuid:79f0049a-4098-8642-ab92-e65be0885f95": "com.microsoft.playready", "urn:uuid:f239e769-efa3-4850-9c16-a903c6932efb": "com.adobe.primetime" }, "sequenceMode": false, "enableAudioGroups": false }, "hls": { "ignoreTextStreamFailures": false, "ignoreImageStreamFailures": false, "defaultAudioCodec": "mp4a.40.2", "defaultVideoCodec": "avc1.42E01E", "ignoreManifestProgramDateTime": false, "mediaPlaylistFullMimeType": "video/mp2t; codecs=\"avc1.42E01E, mp4a.40.2\"", "useSafariBehaviorForLive": true, "liveSegmentsDelay": 3, "sequenceMode": true, "ignoreManifestTimestampsInSegmentsMode": false, "disableCodecGuessing": false, "allowLowLatencyByteRangeOptimization": true }, "mss": { "sequenceMode": false, "keySystemsBySystemId": { "9a04f079-9840-4286-ab92-e65be0885f95": "com.microsoft.playready", "79f0049a-4098-8642-ab92-e65be0885f95": "com.microsoft.playready" } } }, "streaming": { "retryParameters": { "maxAttempts": 2, "baseDelay": 1000, "backoffFactor": 2, "fuzzFactor": 0.5, "timeout": 30000, "stallTimeout": 5000, "connectionTimeout": 10000 }, "rebufferingGoal": 2, "bufferingGoal": 10, "bufferBehind": 30, "ignoreTextStreamFailures": false, "alwaysStreamText": false, "startAtSegmentBoundary": false, "gapDetectionThreshold": 0.5, "gapJumpTimerTime": 0.25, "durationBackoff": 1, "safeSeekOffset": 5, "stallEnabled": true, "stallThreshold": 1, "stallSkip": 0.1, "useNativeHlsOnSafari": true, "inaccurateManifestTolerance": 2, "lowLatencyMode": false, "autoLowLatencyMode": false, "forceHTTPS": false, "preferNativeHls": false, "updateIntervalSeconds": 1, "dispatchAllEmsgBoxes": false, "observeQualityChanges": false, "maxDisabledTime": 30, "parsePrftBox": false, "segmentPrefetchLimit": 0, "liveSync": false, "liveSyncMaxLatency": 1, "liveSyncPlaybackRate": 1.1, "liveSyncMinLatency": 0, "liveSyncMinPlaybackRate": 1 }, "mediaSource": { "codecSwitchingStrategy": "smooth", "sourceBufferExtraFeatures": "", "forceTransmux": false, "insertFakeEncryptionInInit": true }, "offline": { "usePersistentLicense": true, "numberOfParallelDownloads": 5 }, "abr": { "enabled": true, "useNetworkInformation": true, "defaultBandwidthEstimate": 1000000, "switchInterval": 8, "bandwidthUpgradeTarget": 0.85, "bandwidthDowngradeTarget": 0.95, "restrictions": { "minWidth": 0, "maxWidth": null, "minHeight": 0, "maxHeight": null, "minPixels": 0, "maxPixels": null, "minFrameRate": 0, "maxFrameRate": null, "minBandwidth": 0, "maxBandwidth": null }, "advanced": { "minTotalBytes": 128000, "minBytes": 16000, "fastHalfLife": 2, "slowHalfLife": 5 }, "restrictToElementSize": false, "restrictToScreenSize": false, "ignoreDevicePixelRatio": false, "clearBufferSwitch": false, "safeMarginSwitch": 0 }, "autoShowText": 3, "preferredAudioLanguage": "de-DE", "preferredAudioLabel": "", "preferredTextLanguage": "de-DE", "preferredVariantRole": "", "preferredTextRole": "", "preferredAudioChannelCount": 2, "preferredVideoHdrLevel": "AUTO", "preferredVideoLayout": "", "preferredVideoCodecs": [], "preferredAudioCodecs": [], "preferForcedSubs": false, "preferredDecodingAttributes": [], "restrictions": { "minWidth": 0, "maxWidth": null, "minHeight": 0, "maxHeight": null, "minPixels": 0, "maxPixels": null, "minFrameRate": 0, "maxFrameRate": null, "minBandwidth": 0, "maxBandwidth": null }, "playRangeStart": 0, "playRangeEnd": null, "cmcd": { "enabled": false, "sessionId": "", "contentId": "", "useHeaders": false }, "lcevc": { "enabled": false, "dynamicPerformanceScaling": true, "logLevel": 0, "drawLogo": false }, "ads": { "customPlayheadTracker": false } } ```

What did you do?

What did you expect to happen? The player should load the non AV1 track normally.

What actually happened? Player doesn't load the track and fails with Error CHUNK_DEMUXER_ERROR_APPEND_FAILED: Initialization segment misses expected aac track.

I am aware that this is an 3016 error, but the tracks themself work fine, an other reproduction scenario:

At least to me, this would mean that there is no problem with the tracks themself, thus I'm reporting this error.

shaka.util.Error console output ```json { "severity": 2, "category": 3, "code": 3016, "data": [ 3, null, "CHUNK_DEMUXER_ERROR_APPEND_FAILED: Initialization segment misses expected aac track." ], "handled": false, "message": "Shaka Error MEDIA.VIDEO_ERROR (3,,CHUNK_DEMUXER_ERROR_APPEND_FAILED: Initialization segment misses expected aac track.)", "stack": "Error: Shaka Error MEDIA.VIDEO_ERROR (3,,CHUNK_DEMUXER_ERROR_APPEND_FAILED: Initialization segment misses expected aac track.) at new shaka.util.Error (https://shaka-player-demo.appspot.com/lib/util/error.js:101:15)\n at shaka.Player.videoErrorToShakaError_ (https://shaka-player-demo.appspot.com/lib/player.js:6304:12) at shaka.Player.onVideoError_ (https://shaka-player-demo.appspot.com/lib/player.js:6316:24) at HTMLVideoElement.onError (https://shaka-player-demo.appspot.com/lib/player.js:1012:39)" } ```

Context: For bandwidth reasons, we want to deliver AV1 as our highest quality, while still having the lower qualities using H.264/AVC on older VODs.

avelad commented 10 months ago

I have verified that the problem does not occur with the main branch because we reset Media Source when there are some errors. It seems like your content is muxed. Can you try not using muxed content with the same codec and see if it works? I would say that it is a limitation of the browser that with the main branch we are able to support. (HW vs SW codecs)

avelad commented 10 months ago

Internally, it is also possible that there is a problem when we change the codec and container that transmuxer uses. the highest quality uses mp4 and the lowest qualities use ts. I'm going to look into this.

avelad commented 10 months ago

I have found a bug in our logic that decides if we have to reset Media Source or call changeType.

Lubjan commented 10 months ago

I have verified that the problem does not occur with the main branch because we reset Media Source when there are some errors. It seems like your content is muxed. Can you try not using muxed content with the same codec and see if it works? I would say that it is a limitation of the browser that with the main branch we are able to support. (HW vs SW codecs)

I'll ask if we can get a non muxed one, but can't say how quickly we'll be able to test it, I will get back to you as soon as we tried it :D

Lubjan commented 10 months ago

@avelad Just to clarify, am I correct in assuming that your fix only solves this issue when we use non-muxed content and that muxed content is not supported in this case? Because I tried it with the nightly-demo (assuming that the commit is already in there) and am now getting a different Error, but I suppose that is the wanted behavior?

shaka.util.Error ```json { "severity": 2, "category": 3, "code": 3014, "data": [ 0 ], "handled": false, "message": "Shaka Error MEDIA.MEDIA_SOURCE_OPERATION_FAILED (0)", "stack": "Error: Shaka Error MEDIA.MEDIA_SOURCE_OPERATION_FAILED (0) at new shaka.util.Error (https://nightly-dot-shaka-player-demo.appspot.com/lib/util/error.js:101:15) at shaka.media.MediaSourceEngine.onError_ (https://nightly-dot-shaka-player-demo.appspot.com/lib/media/media_source_engine.js:1396:24) at SourceBuffer. (https://nightly-dot-shaka-player-demo.appspot.com/lib/media/media_source_engine.js:532:22)" } ```
avelad commented 10 months ago

@Lubjan I just launched the job to update the nightly demo, in 15 minutes you will be able to test the fix in the nightly demo.

Note: the nightly demo only updates once a day by default, and therefore does not include this change yet.

Lubjan commented 10 months ago

Works fabulously, thank you very much!

avelad commented 10 months ago

@Lubjan I will create version 4.6.3 with this fix this week.