search

mpv-player / mpv

🎥 Command line video player
https://mpv.io
Other
28.21k stars 2.89k forks source link

Improve the behavior when switching tracks between HLS master playlist #9361

Open Romantic-LiXuefeng opened 2 years ago

Romantic-LiXuefeng commented 2 years ago

Before requesting a new feature make sure it hasn't been requested yet. https://github.com/mpv-player/mpv/labels/meta%3Afeature-request

Expected behavior of the wanted feature

HLS master playlist As the link in HLS draft description , if the HLS master playlist without Alternative Audio or Video, we could assume the different variant streams in master playlist contains the same audio codec attributes(eg. same codec, same channel count, same sample rate..). So to the player track interface, there should be only one audio track and some different video tracks, the user could only switch between video tracks.

Now in current implementation, the mpv player parsers all audio tracks from different variant streams in master playlist and thinks the audio tracks are different, and exposés the audio tracks to switch.

 #EXTM3U
 #EXT-X-STREAM-INF:BANDWIDTH=1280000,AVERAGE-BANDWIDTH=1000000
 http://example.com/low.m3u8
 #EXT-X-STREAM-INF:BANDWIDTH=2560000,AVERAGE-BANDWIDTH=2000000
 http://example.com/mid.m3u8
 #EXT-X-STREAM-INF:BANDWIDTH=7680000,AVERAGE-BANDWIDTH=6000000
 http://example.com/hi.m3u8

For example, if the mpv player switch video track from low[low.m3u8] to high[hi.m3u8], but the audio track still consume from low[low.m3u8] stream, At the same time , there are two parallel streams(http connections) to request, it's the waste of
network. More serious happened if we switch between the audios, there is obviously a/v not in sync. and if we first switch the audio track and then to switch the video track, the switch(seek) point become discontinue, the switch(seek) result is the player start playback at different position.

Log file

Reproduction steps:

  1. Start playback normally. http://hls.cntv.myhwcdn.cn/asp/hls/main//0303000a/3/default/00b621218d9841169bdf1f7dcf287e64/main.m3u8?maxbr=2048
  2. Switch the audio Audio --aid=3 (aac 2ch 48000Hz) (1229 kbps) -->no audio --> Audio --aid=1 (aac 2ch 48000Hz) (461 kbps), now the a/v is not in sync. From: 
    [  37.364][i][cplayer]      Video --vid=1 (h264 480x270 25.000fps) (461 kbps)
[  37.364][i][cplayer]      Video --vid=2 (h264 640x360 25.000fps) (870 kbps)
[  37.364][i][cplayer]  (+) Video --vid=3 (h264 1280x720 25.000fps) (1229 kbps)
[  37.364][i][cplayer]      Audio --aid=1 (aac 2ch 48000Hz) (461 kbps)
[  37.364][i][cplayer]      Audio --aid=2 (aac 2ch 48000Hz) (870 kbps)
[  37.364][i][cplayer]  (+) Audio --aid=3 (aac 2ch 48000Hz) (1229 kbps)

    to：

[  54.766][i][cplayer]      Video --vid=1 (h264 480x270 25.000fps) (461 kbps)
[  54.766][i][cplayer]      Video --vid=2 (h264 640x360 25.000fps) (870 kbps)
[  54.766][i][cplayer]  (+) Video --vid=3 (h264 1280x720 25.000fps) (1229 kbps)
[  54.766][i][cplayer]      Audio --aid=1 (aac 2ch 48000Hz) (461 kbps)
[  54.766][i][cplayer]      Audio --aid=2 (aac 2ch 48000Hz) (870 kbps)
[  54.766][i][cplayer]      Audio --aid=3 (aac 2ch 48000Hz) (1229 kbps)

then 

[  56.695][i][cplayer]      Video --vid=1 (h264 480x270 25.000fps) (461 kbps)
[  56.695][i][cplayer]      Video --vid=2 (h264 640x360 25.000fps) (870 kbps)
[  56.695][i][cplayer]  (+) Video --vid=3 (h264 1280x720 25.000fps) (1229 kbps)
[  56.695][i][cplayer]  (+) Audio --aid=1 (aac 2ch 48000Hz) (461 kbps)
[  56.695][i][cplayer]      Audio --aid=2 (aac 2ch 48000Hz) (870 kbps)
[  56.695][i][cplayer]      Audio --aid=3 (aac 2ch 48000Hz) (1229 kbps)
  1. Then switch the video, Video --vid=3 (h264 1280x720 25.000fps) (1229 kbps) -> no video stream -> Video --vid=1 (h264 480x270 25.000fps) (461 kbps), now the switch result is the playback start from the discontinue position. From: 
    [  56.695][i][cplayer]      Video --vid=1 (h264 480x270 25.000fps) (461 kbps)
[  56.695][i][cplayer]      Video --vid=2 (h264 640x360 25.000fps) (870 kbps)
[  56.695][i][cplayer]  (+) Video --vid=3 (h264 1280x720 25.000fps) (1229 kbps)
[  56.695][i][cplayer]  (+) Audio --aid=1 (aac 2ch 48000Hz) (461 kbps)
[  56.695][i][cplayer]      Audio --aid=2 (aac 2ch 48000Hz) (870 kbps)
[  56.695][i][cplayer]      Audio --aid=3 (aac 2ch 48000Hz) (1229 kbps)

    to:

    [  79.906][i][cplayer]      Video --vid=1 (h264 480x270 25.000fps) (461 kbps)
[  79.906][i][cplayer]      Video --vid=2 (h264 640x360 25.000fps) (870 kbps)
[  79.906][i][cplayer]      Video --vid=3 (h264 1280x720 25.000fps) (1229 kbps)
[  79.906][i][cplayer]  (+) Audio --aid=1 (aac 2ch 48000Hz) (461 kbps)
[  79.906][i][cplayer]      Audio --aid=2 (aac 2ch 48000Hz) (870 kbps)
[  79.906][i][cplayer]      Audio --aid=3 (aac 2ch 48000Hz) (1229 kbps)

    then:

    [  81.233][i][cplayer]  (+) Video --vid=1 (h264 480x270 25.000fps) (461 kbps)
[  81.233][i][cplayer]      Video --vid=2 (h264 640x360 25.000fps) (870 kbps)
[  81.233][i][cplayer]      Video --vid=3 (h264 1280x720 25.000fps) (1229 kbps)
[  81.233][i][cplayer]  (+) Audio --aid=1 (aac 2ch 48000Hz) (461 kbps)
[  81.233][i][cplayer]      Audio --aid=2 (aac 2ch 48000Hz) (870 kbps)
[  81.233][i][cplayer]      Audio --aid=3 (aac 2ch 48000Hz) (1229 kbps)

The log file: mpv.log

SebiderSushi commented 1 year ago

Another important aspect when switching HLS tracks would be an option to apply the new track selection only when fetching new segments instead of discarding the current cache and switching immediately.

This is useful when track changes are meant to change the bitrate/resolution. Fetching segments that are already cached might not be desired depending on the networking conditions. Also, while watching a livestream it is usually not desired to discard segments from the cache which are expired from the remote server and can't be fetched again. Depending on the available RAM it can be very useful to allocate a lot of memory for mpv's demuxer cache to allow pausing a livestream without missing anything.

Possibly worth its own issue?