How to ensure track alignment on ABR

[irg1008]

Hi! (You must think I am crazy at this point)

Well I am trying and succeding (sometimes) to create multiple qualities for a stream. It works with stream proccesses both with ffmpeg and AV. The problem is that consuming the streaming with HLS (hls.js) sometimes receives all the video tracks sometimes just the original one:

index.m3u8

#EXTM3U
#EXT-X-INDEPENDENT-SEGMENTS
#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="aud",LANGUAGE="und-1",NAME="AAC-1",URI="1/index.m3u8?mTrack=2&iMsn=67&tkn=668057077"
#EXT-X-STREAM-INF:AUDIO="aud",CODECS="avc1.64002a,mp4a.40.2",RESOLUTION=1920x1080,FRAME-RATE=60,BANDWIDTH=10618598,AVERAGE-BANDWIDTH=8984967
2/index.m3u8?mTrack=2&iMsn=67&tkn=668057077
## NOTE: Track 3 is available, but ignored because it is not aligned with track 2.
## NOTE: Track 4 is available, but ignored because it is not aligned with track 2.

Sometimes the tracks are able to align and I get all tracks and so I am able to swap between them.

My Hls config in the frontend is this one:

const hlsConfig: Partial<HlsConfig> = {
  debug: false,
  enableWorker: true,
  lowLatencyMode: true,
  backBufferLength: 90,
  maxBufferLength: 300,
}

A stream proccess for 720p with ffmpeg looks like this:

Sometimes it just works, then you load the stream event and it doesn't anymore.

Is there any way to ensure track alignment even if it takes a bit for the player to load the stream??

Thank you and I know I am a pain in the ass!! Have a nice day

[Rokamun]

Hmm, are there multiple encodes on the same stream? MistServer will by default take the "last" added live track and use that as a source. So you might need to mask every encode from processes. You can do that by setting the Source track mask to not be usable for processes. That way you can never re-encode from an already encoded stream. Chances are that the times it worked sometimes are when all processes managed to start encoding from the original source track.

Normally you should be able to force keyframe alignment with the -force_key_frames source which you set by the keyframes Match input keyframes which does make it a tad weird that it is not aligning though...

[irg1008]

It seems opting out on proccessing is not available

I opted out from the output track for proccessing but that didn't work, still getting misaligned tracks, maybe I have too many stream proccesses?

[irg1008]

I think I am able to "fix" this for now by encoding the original stream in the same quality. Example OBS streams 1080p, I encode into another ouput with same quality. Those then are always (or at least seems to be) algined. The problem is you have one more proccess for each stream

[irg1008]

The only problem facing know it's the bitrate it's absolutely ignored. For example I set 6000k minimun bitrate and output has 1200k, but I guess that's a problem for another day, and tomorow it's friday, so I guess another week haah

[Rokamun]

Heya,

I see I said it a bit confusing. So let me make it a bit more clear, as I definitely could do better there:

You need the Source track to only be available for processing Then all the output tracks available for viewers (and optionally processing)

Even easier honestly is to just make the whole bitrate tree a single command through MKV-exec. It would be more reliable considering you can then at least force all those qualities to have the same keyframe interval. The multiple processes clearly isn't doing well for you:

ffmpeg -loglevel quiet -hide_banner -fflags nobuffer -i -  -map v:0 -map v:0 -map v:0 -map a:0  -c:v:0 h264 -b:v:0 5000k -s:v:0 1920x1080 -c:v:1 h264 -b:v:1 2000k -s:v:1  1280x720  -c:v:2 h264 -b:v:2 1000k -s:v:2 854x480 -c:a copy -force_key_frames source -cluster_time_limit 0 -f matroska -

Chances are your CPU will not be happy with this, you can try to alleviate that by setting -preset veryfast right behind -c:a copy or using a hardware encoder like h264_nvenc at every place it says h264 assuming you have an Nvidia card available.

That should generate a video with the following qualities:

• 1080p at ~5000kbps
• 720p at ~2000kbps
• 4380p at ~1000kbps

The reason you see the bitrate "ignored" is because FFmpeg will by see the bitrate as a target, if it's capable of doing the target quality with lesser bitrate it will. You can counter this with setting a constant bitrate.

That would be adding a -minrate X , -maxrate X and -bufsize x to every -b:v:0/1/2

-minrate X this is the target minimum bitrate, -maxrate X this is the target maximum bitrate, -bufsize X This is the size of the buffer it can use to cut these up. Usually target 1 to 5 times the target bitrate in order to make sure the target bitrate can be met.

But honestly I would first try it without the minrate/maxrate/bufsize and see if you're happy with that quality.