Cannot skip to unbuffered video parts in ffmpeg > 4.2.2

[cars10]

Environment

• [x] The MPD passes the DASH-IF Conformance Tool on https://conformance.dashif.org/
• [x] The stream has correct Access-Control-Allow-Origin headers (CORS)
• [x] There are no network errors such as 404s in the browser console when trying to play the stream
• [x] The issue observed is not mentioned on https://github.com/Dash-Industry-Forum/dash.js/wiki/FAQ
• [x] The issue occurs in the latest reference client on http://reference.dashif.org/dash.js/ and not just on my page
• Link to playable MPD file:
  • working manifest, generated with ffmpeg 4.2.2: https://static.cars10k.de/dash-error/ffmpeg-4.2.2/manifest.mpd
  • not working manifest, generated with ffmpeg n5.0: https://static.cars10k.de/dash-error/ffmpeg-n5.0/manifest.mpd
• Dash.js version: 4.3.0
• Browser name/version: Google Chrome Version 99.0.4844.51 (Official Build) (64-bit)
• OS name/version: Manjaro Linux

Steps to reproduce

1. Get a test video, for example this 1080p test video of a timer
2. Create 480p and 720p version of the video

bash -c "ffmpeg -y -i input.mp4 -map 0:v:0 -vsync passthrough -c:v libx264 -keyint_min 24 -g 24 -sc_threshold 0 -crf 23 -maxrate 2500k -bufsize 5M -profile:v main -preset slow -an -dn -sn -write_tmcd 0 -map_chapters -1 -vf scale='trunc(oh*a/2)'*2:480 -movflags +faststart -aspect 16:9 output480.mp4"

bash -c "ffmpeg -y -i input.mp4 -map 0:v:0 -vsync passthrough -c:v libx264 -keyint_min 24 -g 24 -sc_threshold 0 -crf 20 -maxrate 5M -bufsize 10M -profile:v main -preset slow -an -dn -sn -write_tmcd 0 -map_chapters -1 -vf scale='trunc(oh*a/2)'*2:720 -movflags +faststart -aspect 16:9 output720.mp4"

3. Generate dash manifest

bash -c 'ffmpeg -y -i output480.mp4 -i output720.mp4 -c copy \
        -map 0:v:0 -map 0:v:0 -f dash -single_file 1 -single_file_name stream-\$RepresentationID\$.m4s \
        -adaptation_sets "id=0,streams=0,1" manifest.mpd'

4. Load the manifest in a dash capable player, for example mediaelementjs or the official dash reference player
5. Immediately skip to an unbuffered part of the video, e.g.:

Observed behavior

FFMPEG <= 4.2.2 works as expected. You can skip to an unbuffered part, dashjs loads the video, and starts playing.

FFMPEG > 4.2.2 does not work. It does not load the needed parts and play/pause do not work. The videos is broken until you reload the player. You can only skip to buffered parts of the video.

Console output

[536458][MediaPlayer] Streaming Initialized 
Debug.js:169 [536469][MediaPlayer] Playback Initialized 
Debug.js:169 [536576][DashParser] Parsing complete: ( xml2json: 12.0ms, objectiron: 0.300ms, total: 0.0123s) 
Debug.js:169 [536579][StreamController] Manifest updated... updating data system wide. 
Debug.js:169 [536583][ManifestUpdater] Manifest has been refreshed at Mon Mar 14 2022 16:05:54 GMT+0100 (Central European Standard Time)[1647270354.684]  
Debug.js:169 [536590][StreamController] Switch to stream 0. Seektime is 0, current playback time is null. Seamless period switch is set to false 
Debug.js:169 [536594][MediaSourceController] Set MediaSource duration:181.2 
Debug.js:169 [536602][Stream] No audio data. 
Debug.js:169 [536602][Stream] No text data. 
Debug.js:169 [536602][Stream] No muxed data. 
Debug.js:169 [536602][Stream] No image data. 
Debug.js:169 [536603][AbrController] Stream ID: 0 [video] switch from 0 to 1/1 (buffer: 0) . 
Debug.js:169 [536638][StreamProcessor][video] OnFragmentLoadingCompleted for stream id 0 and media type video - Url: https://static.cars10k.de/dash-error/ffmpeg-n5.0/stream-1.m4s , Range:0-832 
Debug.js:169 [536640][ScheduleController][video] [video] lastInitializedRepresentationInfo changed to 1 
Debug.js:169 [536642][PlaybackController] Native video element event: play 
Debug.js:169 [536642][PlaybackController] Native video element event: waiting 
Debug.js:169 [536643][PlaybackController] Native video element event: loadedmetadata 
Debug.js:169 [536644][ScheduleController][video] Top quality video index has changed from NaN to 1 
Debug.js:169 [536688][StreamProcessor][video] OnFragmentLoadingCompleted for stream id 0 and media type video - Url: https://static.cars10k.de/dash-error/ffmpeg-n5.0/stream-1.m4s , Range:833-55081 
Debug.js:169 [536709][PlaybackController] Native video element event: loadeddata 
Debug.js:169 [536709][PlaybackController] Native video element event: playing 
Debug.js:169 [536729][StreamProcessor][video] OnFragmentLoadingCompleted for stream id 0 and media type video - Url: https://static.cars10k.de/dash-error/ffmpeg-n5.0/stream-1.m4s , Range:55082-109331 
Debug.js:169 [536779][StreamProcessor][video] OnFragmentLoadingCompleted for stream id 0 and media type video - Url: https://static.cars10k.de/dash-error/ffmpeg-n5.0/stream-1.m4s , Range:109332-160402 
Debug.js:169 [536782][AbrController] [video] switching from throughput to buffer occupancy ABR rule (buffer: 17.926). 
Debug.js:169 [536819][StreamProcessor][video] OnFragmentLoadingCompleted for stream id 0 and media type video - Url: https://static.cars10k.de/dash-error/ffmpeg-n5.0/stream-1.m4s , Range:160403-213155 
Debug.js:169 [536854][StreamProcessor][video] OnFragmentLoadingCompleted for stream id 0 and media type video - Url: https://static.cars10k.de/dash-error/ffmpeg-n5.0/stream-1.m4s , Range:213156-267548 
Debug.js:169 [536886][StreamProcessor][video] OnFragmentLoadingCompleted for stream id 0 and media type video - Url: https://static.cars10k.de/dash-error/ffmpeg-n5.0/stream-1.m4s , Range:267549-320005 

// now we skip to a later, unbuffered part in the video

Debug.js:169 [538318][PlaybackController] Requesting seek to time: 145 
Debug.js:169 [538332][PlaybackController] Seeking to: 145 
Debug.js:169 [538334][PlaybackController] Native video element event: waiting 
Debug.js:169 [538335][AbrController] [video] switching from buffer occupancy to throughput ABR rule (buffer: 0.000). 
Debug.js:169 [538368][StreamProcessor][video] OnFragmentLoadingCompleted for stream id 0 and media type video - Url: https://static.cars10k.de/dash-error/ffmpeg-n5.0/stream-1.m4s , Range:0-832 
Debug.js:169 [538369][ScheduleController][video] [video] lastInitializedRepresentationInfo changed to 1 
Debug.js:169 [538430][StreamProcessor][video] OnFragmentLoadingCompleted for stream id 0 and media type video - Url: https://static.cars10k.de/dash-error/ffmpeg-n5.0/stream-1.m4s , Range:1511638-1565315 
Debug.js:169 [538460][StreamProcessor][video] OnFragmentLoadingCompleted for stream id 0 and media type video - Url: https://static.cars10k.de/dash-error/ffmpeg-n5.0/stream-1.m4s , Range:1565316-1581120 
Debug.js:169 [538463][AbrController] Stream ID: 0 [video] switch from 1 to 0/1 (buffer: 0) "InsufficientBufferRule: Buffer is empty" 
Debug.js:169 [538498][StreamProcessor][video] OnFragmentLoadingCompleted for stream id 0 and media type video - Url: https://static.cars10k.de/dash-error/ffmpeg-n5.0/stream-0.m4s , Range:0-832 
Debug.js:169 [538499][ScheduleController][video] [video] lastInitializedRepresentationInfo changed to 0 
Debug.js:169 [538541][StreamProcessor][video] OnFragmentLoadingCompleted for stream id 0 and media type video - Url: https://static.cars10k.de/dash-error/ffmpeg-n5.0/stream-0.m4s , Range:1511638-1565315 
Debug.js:169 [538570][StreamProcessor][video] OnFragmentLoadingCompleted for stream id 0 and media type video - Url: https://static.cars10k.de/dash-error/ffmpeg-n5.0/stream-0.m4s , Range:1565316-1581120 
Debug.js:169 [538573][StreamController] [onStreamBufferingCompleted] calls signalEndOfStream of mediaSourceController. 
Debug.js:169 [538573][MediaSourceController] call to mediaSource endOfStream 

I think the specific issue is:

[562604][AbrController] Stream ID: 0 [video] switch from 1 to 0/1 (buffer: 0) "InsufficientBufferRule: Buffer is empty" 

Expected behavior

The player should load the video correctly, as it does when the file is generated with ffmpeg <= 4.2.2

I have tested multiple versions of ffmpeg (4.3, 4.4, etc) and i can reproduce the issue with every version > 4.2.2. The last working version for this setup is 4.2.2. I have also tested multiple videos from different sources.

It is irrelevant which version of ffmpeg you use to create the 480p and 720p files, only the last step (dash manifest creation) matters.

[dsilhavy]

This sounds like an ffmpeg issue to me and not like a dash.js issue. Can you check the behavior in Shaka player please as well: https://shaka-player-demo.appspot.com/ . If it shows the same behavior it is very unlikely that this is a player issue.

[cars10]

Thanks, i can reproduce the issue with shaka. I will open an issue in the ffmpeg bugtracker and link it here.

[cars10]

FFMPEG issue: https://trac.ffmpeg.org/ticket/9701#ticket

[taras-clarifai]

Hey @cars10 . Did you get to workaround that issue?

[cars10]

Unfortunately no. We are still using 4.2.2 because of this issue. Do you have the same problem?

[taras-clarifai]

Yeah, I do. The original manifest files that you linked in the ffmpeg bug tracker have minimal differences (e.g. few attributes and a different segment duration). I was wondering if you ever got to try to find if removing some of the differences will fix the problem (or maybe the problem is in reencoded files itself)

[tot-ra]

Looks like SegmentList.duration that ffmpeg produces in the manifest is 4.3 is no longer valid and is just some 5000000

upd. nevermind. the problem was that I used -use_template 1 while using -single_file_name