m1k1o / go-transcode

On-demand transcoding origin server for live inputs and static files in Go using ffmpeg. Also with NVIDIA GPU hardware acceleration.
Apache License 2.0
208 stars 38 forks source link

Some freezes with static files #23

Closed rudnypc closed 2 years ago

rudnypc commented 2 years ago

I'm have a problem with static files, sometimes the screen freezes for while

I believe that probleme is: "force_key_frames"

m1k1o commented 2 years ago

Did you try it with and without video-keyframes enabled? Is it giving different results for you?

rudnypc commented 2 years ago

Yes, I don't have the same problem but ffmpeg generates TS files with different duration

rudnypc commented 2 years ago

I removed -force_key_frames and -segment_times, the video works better

m1k1o commented 2 years ago

But that would evenutally disable seeking functionality. Since length of the segments is not known anymore. And we cannot determine, if segment have been transcoded or if it overlaps with existing.

rudnypc commented 2 years ago

but do you think ffprobe is generating incorrectly? do you have the same problem?

is skip_frame nokey the problem?

rudnypc commented 2 years ago

What do you think about fixed segment time? Something like ffmpeg -i <VIDEO> -sn -vf scale=-2:1080 -preset ultrafast -c:v libx264 -b:v 14400k -c:a aac -ac 2 -b:a 320k -force_key_frames 'expr:gte(t,n_forced*4)' -hls_time 4 -hls_playlist_type vod -max_muxing_queue_size 2048 -hls_segment_filename t/720p_%03d.ts t/720p.m3u

m1k1o commented 2 years ago

but do you think ffprobe is generating incorrectly?

When not using keyframes, then they are not generated by ffprobe. Just by an algorithm, so that all chunk times are known. I did not see problems with video I tested with, but there might be different encoding causing different problems. Maybe adding no-scenecut could force keyframes to by only the selected.

What do you think about fixed segment time?

I don't see how it should be different to just telling ffmpeg where to split the video explicitly.

rudnypc commented 2 years ago

ffprobe generated this:

ffprobe -v error -skip_frame nokey -show_entries frame=pkt_pts_time -show_entries format=duration -show_entries stream=duration,width,height -select_streams v -of json MYMOVIE.mkv
{
    "frames": [
        {
            "pkt_pts_time": "0.000000"
        },
        {
            "pkt_pts_time": "10.005000"
        },
        {
            "pkt_pts_time": "20.005000"
        },
        {
            "pkt_pts_time": "28.285000"
        },
        {
            "pkt_pts_time": "33.525000"
        },
        {
            "pkt_pts_time": "37.165000"
        },
        {
            "pkt_pts_time": "47.165000"
        },
        {
            "pkt_pts_time": "57.165000"
        },
        {
            "pkt_pts_time": "67.165000"
        },
        {
            "pkt_pts_time": "76.605000"
        },
        {
            "pkt_pts_time": "78.165000"
        },
        {
            "pkt_pts_time": "79.525000"
        },
        {
            "pkt_pts_time": "80.925000"
        },
        {
            "pkt_pts_time": "88.365000"
        },
        {
            "pkt_pts_time": "89.405000"
        },
        {
            "pkt_pts_time": "99.405000"
        },
        {
            "pkt_pts_time": "102.045000"
        },
        {
            "pkt_pts_time": "103.805000"
        },
        {
            "pkt_pts_time": "106.125000"
        },
        {
            "pkt_pts_time": "107.925000"
        },
        {
            "pkt_pts_time": "109.205000"
        },
        {
            "pkt_pts_time": "110.365000"
        },
        {
            "pkt_pts_time": "115.725000"
        },
        {
            "pkt_pts_time": "121.085000"
        },
        {
            "pkt_pts_time": "123.965000"
        },
        {
            "pkt_pts_time": "131.125000"
        },
        {
            "pkt_pts_time": "132.765000"
        },
        {
            "pkt_pts_time": "134.005000"
        },
        {
            "pkt_pts_time": "137.885000"
        },
        {
            "pkt_pts_time": "140.605000"
        },
        {
            "pkt_pts_time": "144.205000"
        },
        {
            "pkt_pts_time": "148.765000"
        },
        {
            "pkt_pts_time": "150.765000"
        },
        {
            "pkt_pts_time": "156.245000"
        }
        .......................

When I started to play, I found the command:

 2434 root      0:02 ffmpeg -loglevel warning -ss 52.464056 -i media/MYMOVIE.mkv -to 59.459263 -copyts -force_key_frames 55.961659,59.459263 -sn -vf scale=-2:540 -c:v libx264 -preset faster -profile:v high -level:v 4.0 -b:v 1800k -c:a aac -b:a 192k -f segment -segment_time_delta 0.2 -segment_format mpegts -segment_times 55.961659,59.459263 -segment_start_number 15 -segment_list_type flat -segment_list pipe:1 transcode/vod-540p-517768070/540p-%05d.ts
 2476 root      0:00 bash
 2483 root      0:00 ps -a
 2484 root      0:00 cat
bash-5.1# ps -a|cat -
PID   USER     TIME  COMMAND
    1 root      0:02 ./go-transcode serve
 2476 root      0:00 bash
 2527 root      0:02 ffmpeg -loglevel warning -ss 69.952074 -i media/MYMOVIE.mkv -to 76.947282 -copyts -force_key_frames 73.449678,76.947282 -sn -vf scale=-2:540 -c:v libx264 -preset faster -profile:v high -level:v 4.0 -b:v 1800k -c:a aac -b:a 192k -f segment -segment_time_delta 0.2 -segment_format mpegts -segment_times 73.449678,76.947282 -segment_start_number 20 -segment_list_type flat -segment_list pipe:1 transcode/vod-540p-517768070/540p-%05d.ts
 2569 root      0:00 ps -a
 2570 root      0:00 bash
bash-5.1# ps -a|cat -
PID   USER     TIME  COMMAND
    1 root      0:02 ./go-transcode serve
 2476 root      0:00 bash
 2739 root      0:05 ffmpeg -loglevel warning -ss 108.425715 -i media/MYMOVIE.mkv -to 115.420922 -copyts -force_key_frames 111.923319,115.420922 -sn -vf scale=-2:540 -c:v libx264 -preset faster -profile:v high -level:v 4.0 -b:v 1800k -c:a aac -b:a 192k -f segment -segment_time_delta 0.2 -segment_format mpegts -segment_times 111.923319,115.420922 -segment_start_number 31 -segment_list_type flat -segment_list pipe:1 transcode/vod-540p-517768070/540p-%05d.ts

I'm missing something? why the the force_key_frames and segment_times is a little different from ffprobe?

when a I'm looking inside from playlist, it's generated like a fixed time 3.498:

#EXTM3U
#EXT-X-VERSION:4
#EXT-X-PLAYLIST-TYPE:VOD
#EXT-X-MEDIA-SEQUENCE:0
#EXT-X-TARGETDURATION:4.75
#EXTINF:3.498, no desc
540p-00001.ts
#EXTINF:3.498, no desc
540p-00002.ts
#EXTINF:3.498, no desc
540p-00003.ts
#EXTINF:3.498, no desc
540p-00004.ts
#EXTINF:3.498, no desc
540p-00005.ts
#EXTINF:3.498, no desc
540p-00006.ts
#EXTINF:3.498, no desc
540p-00007.ts
#EXTINF:3.498, no desc
540p-00008.ts
#EXTINF:3.498, no desc
540p-00009.ts
#EXTINF:3.498, no desc

am i looking and calculating wrong?

I found this document: https://www.nginx.com/wp-content/uploads/2018/12/NGINX-Conf-2018-slides_Choi-streaming.pdf

the playlist result looks like ffprobe image

m1k1o commented 2 years ago

By default, usage of keyframes is disabled: video-keyframes: false.

That means, video is not split by keyframes, but in fixed time 3.498. Only if you enable it in config, that ffprobe will be executed, keyframe times will be generated (and cached if wanted) and used in splitting videos.

https://github.com/m1k1o/go-transcode/blob/14e1d9146183cc9c85e7e9cb28a2916b0cabd26f/hlsvod/manager.go#L122-L123

rudnypc commented 2 years ago

I try to enable the video-keyframes: false tag, but I believe it didn't work. I try to put inside and outside of vod tag:

  videokeyframes: true
  video-keyframes: true
  VideoKeyframes: true

Only ffprobe running is: ffprobe -v error -show_format -show_streams -of json MYMOVIE.mkv

m1k1o commented 2 years ago

Thanks for reporting, I missed that one to pass down from config file to the transcoding backend. Could you please try again?

rudnypc commented 2 years ago

Yes, the problem with the tag was fixed. I found this command: ffprobe -v error -skip_frame nokey -show_entries frame=pkt_pts_time -show_entries format=duration -show_entries stream=duration,width,height -select_streams v -of json media/MYMOVIE.mp4

The output is something like this:

 {
            "pkt_pts_time": "67.359000",
            "side_data_list": [
                {

                }
            ]
        },
        {
            "pkt_pts_time": "71.780000",
            "side_data_list": [
                {

                }
            ]
        },
        {
            "pkt_pts_time": "76.702000",
            "side_data_list": [
                {

                }
            ]
        },
        {
            "pkt_pts_time": "81.707000",
            "side_data_list": [
                {

                }
            ]
        },
        {
            "pkt_pts_time": "85.002000",
            "side_data_list": [
                {

                }
            ]
        },
        {
            "pkt_pts_time": "89.798000",
            "side_data_list": [
                {

                }
            ]
        }

When I play, I found this command too: ffmpeg -loglevel warning -ss 79.204500 -i media/MYVIDEO.mkv -to 85.002000 -copyts -force_key_frames 81.707000,85.002000 -sn -vf scale=-2:720 -c:v libx264 -preset faster -profile:v high -level:v 4.0 -b:v 2800k -c:a aac -b:a 192k -f segment -segment_time_delta 0.2 -segment_format mpegts -segment_times 81.707000,85.002000 -segment_start_number 27 -segment_list_type flat -segment_list pipe:1 transcode/vod-720p-4273827325/720p-%05d.ts

But I have a question, Should the -SS match with the first force_key_frames?

rudnypc commented 2 years ago

When I open the generated HLS file, I found:

EXTINF:4.630, no desc

720p-00009.ts

When I download the segment, the time is 2 second: image

m1k1o commented 2 years ago

Segments are 3.50s long, but with offset 1.25s. That means, the shortest segment length can be 2.25s, longest can be 4.75s. https://github.com/m1k1o/go-transcode/blob/b3af116d56b058a1b0284fbaf949115fea0cf54d/hlsvod/manager.go#L60-L61

An algorithm takes all key frames and tries to use most of them. That means, segment length should average 3.50s, but if there is key frames 1.25s earlier or later in the stream, it would be taken. If there are multiple key frames in this timespan, only one is taken.

https://github.com/m1k1o/go-transcode/blob/b3af116d56b058a1b0284fbaf949115fea0cf54d/hlsvod/utils.go#L11-L58

Minimum buffer length is 3 segments, and maximum is 5. That means, maximum 5 segments are transcoded at the same time and minimum 3 of them are always available. That means, initially there are 5 segments transcoded, and after 2 segments, another 5 segments are transcoded, and offset is controlled by -ss. And so on, throughout the stream. When playing head moves because of seeking, next segments are being transcoded until we reach the end of the stream.

https://github.com/m1k1o/go-transcode/blob/b3af116d56b058a1b0284fbaf949115fea0cf54d/hlsvod/manager.go#L62-L63

rudnypc commented 2 years ago

Ok, I changed some values to work better: 1º I'm using the ffprobe 2º I changed the segment value to:

                segmentLength:    29.50,
                segmentOffset:    29.50,
                segmentBufferMin: 1,
                segmentBufferMax: 50,

3º I changed the playlist generator to ignore the first result (0.00000)

         // playlist segments
        for i := 2; i < len(m.breakpoints); i++ {
                playlist = append(playlist,
                        fmt.Sprintf("#EXTINF:%.3f, no desc", m.breakpoints[i]-m.breakpoints[i-1]),
                        m.getSegmentName(i-1),
                )
        }

Now it's working much better, thanks

m1k1o commented 2 years ago

I changed the playlist generator to ignore the first result (0.00000)

I applied this fix in recent commit.

Thinking about passing other values as configs.

rudnypc commented 2 years ago

Fixed, thank you \o/

rudnypc commented 2 years ago

sorry, but your fix to segment is making the ffmpeg to start after sometime, the skip is only in the segment generator. ffmpeg -loglevel warning -ss 0.208333 -i MYVIDEO.......

m1k1o commented 2 years ago

Hmm, so that mean we miss first seconds of the stream? It should be removed only from playlist? I'll look into it today again. Thanks for letting me know.

rudnypc commented 2 years ago

yes, the zero number should be removed only from playlist segment.

m1k1o commented 2 years ago

Fixed.

rudnypc commented 2 years ago

The start value is correct, but the ts is wrong, the segment should start with 00001. The calculation time of the 720p-00001.ts is from 0 and the next keyframe.

#EXTM3U
#EXT-X-VERSION:4
#EXT-X-PLAYLIST-TYPE:VOD
#EXT-X-MEDIA-SEQUENCE:0
#EXT-X-TARGETDURATION:4.75
#EXTINF:2.502, no desc
720p-00002.ts
#EXTINF:2.586, no desc
720p-00003.ts
#EXTINF:3.003, no desc
720p-00004.ts
#EXTINF:2.502, no desc
rudnypc commented 2 years ago

because this I added "-1" in the segment name: m.getSegmentName(i-1),

I don't know if this will create a problem without keyframe.

m1k1o commented 2 years ago

I didn't notice that, sorry, my mistake!

rudnypc commented 2 years ago

The profile-0001 segment is a m3u8

rudnypc commented 2 years ago

image

m1k1o commented 2 years ago

It looks like it returns manifest only if the transcoding did not start yet. When i request second segment, then even first one works.

UPDATE: Even after re-requesting first manifest it returns it correctly.

m1k1o commented 2 years ago

Usecase of requesting segments before manifests was not considered, that is why it fails. The initialization request was expected to be manifest. That means, the first request to any segment ot playlist returns playlist. But this is unexpected behaviour.