leandromoreira / ffmpeg-libav-tutorial

FFmpeg libav tutorial - learn how media works from basic to transmuxing, transcoding and more. Translations: 🇺🇸 🇨🇳 🇰🇷 🇪🇸 🇻🇳 🇧🇷

https://github.com/leandromoreira/ffmpeg-libav-tutorial

BSD 3-Clause "New" or "Revised" License

9.94k stars 956 forks source link

Add transcoding example #54

Closed leandromoreira closed 4 years ago

leandromoreira commented 4 years ago

After I learned a little I think this is the best examples to write/show the transcoding chapter:

[x] 1 - a transmuxing example - mp4 -> mp4 dbab188a9509884ee15e778e285721aba06a2d57
[x] 2 - a transmuxing example - mp4 -> fmp4 9f316100b68fa7c6c9f280c733dd89a82c9f2e2f
[x] 3 - a transcoding example - h264 -> h264 (fixed gop, no scenecut) 6ad6a82fb37f7b8d187f88e3e3ee3aeeb240e832
[x] 4 - a transrating example - h264 -> h264 (fixed gop & CBR bit rate) 8ed2c1fcee15f62c01b980875fdddcc11194b4ab

-b:v 1M -minrate 1M -maxrate 1M -bufsize 2M
[x] 5 - a transcoding example - h264 -> h265 60608711f0d343f0f3f8110c0a6b304780d40755
[x] 6 - add the markdown chapter explanning the process
[ ] 7 - (Bonus) a complete transmux|cod|ing example - mp4 -> webm, h264 -> vp9, aac -> opus

Issues

Trying to achieve 3 & 4

high bit rate (10.6 Mb/s) fixed ✅ by b5378f38a582aff7ebf0821a561c63ad7f277cf8 (had to use force-cfr 😢 I really don't understand rate control at this level)
warning messages (forced frame type (5) at 80 was changed to frame type (3)) fixed ✅ by 0abee0803914fb03a1aeb2847a904f34bd0d4059
variable FPS (59.99 fps) fixed ✅ by 8ed2c1fcee15f62c01b980875fdddcc11194b4ab

Trying to achieve 3 & 4

warning messages specified frame type (5) at 180 is not compatible with keyframe interval ✅ by 0abee0803914fb03a1aeb2847a904f34bd0d4059

Wanna help me?

If you want to test or run it locally make sure you have docker installed and that you had ran make fetch_small_bunny_video to download the sample video.

git clone --single-branch --branch transcoding-chapter https://github.com/leandromoreira/ffmpeg-libav-tutorial.git

make run_transcoding

# this is going to create the file bunny_1s_gop.mp4

leandromoreira commented 4 years ago

Simple transmuxing and fmp4 is working!

leandromoreira commented 4 years ago

https://github.com/leandromoreira/ffmpeg-libav-tutorial/pull/52

leandromoreira commented 4 years ago

While coding the 3 38c6c782aa9530c61537e74b1e1781d59f3eddc8 I faced the following issues but it works:

the fps has changed 59.99 fps

Video: h264, yuv420p, 1920x1080, 7632 kb/s, 59.99 fps, 60 tbr, 16384 tbn, 0.03 tbc (default)

a single warning message has appeared

[libx264 @ 0x1e78200] forced frame type (5) at 597 was changed to frame type (3)

✅ Playable in VLC

leandromoreira commented 4 years ago

While dealing with 3 6ad6a82fb37f7b8d187f88e3e3ee3aeeb240e832 now it seems that the gop is fixed but other issues arrived:

bit rate is high

Bit rate                                 : 10.6 Mb/s

fps still seems to be variable

Frame rate mode                          : Variable
Frame rate                               : 60.000 FPS
Minimum frame rate                       : 56.302 FPS
Maximum frame rate                       : 60.015 FPS

a lot of warning messages have appeared:

[libx264 @ 0x1a5a200] forced frame type (5) at 80 was changed to frame type (3)
[libx264 @ 0x1a5a200] forced frame type (3) at 81 was changed to frame type (1)
[libx264 @ 0x1a5a200] forced frame type (5) at 140 was changed to frame type (3)
[libx264 @ 0x1a5a200] forced frame type (3) at 141 was changed to frame type (1)
[libx264 @ 0x1a5a200] forced frame type (5) at 200 was changed to frame type (3)
[libx264 @ 0x1a5a200] forced frame type (3) at 201 was changed to frame type (1)
[libx264 @ 0x1a5a200] forced frame type (5) at 310 was changed to frame type (1)
[libx264 @ 0x1a5a200] forced frame type (5) at 369 was changed to frame type (3)
[libx264 @ 0x1a5a200] forced frame type (5) at 370 was changed to frame type (1)
[libx264 @ 0x1a5a200] forced frame type (5) at 429 was changed to frame type (3)
[libx264 @ 0x1a5a200] forced frame type (5) at 430 was changed to frame type (1)
[libx264 @ 0x1a5a200] forced frame type (5) at 489 was changed to frame type (3)
...

✅ Playable in VLC

leandromoreira commented 4 years ago

When trying to change the bit rate. If you change only the AVCodecContext->bit_rate this is going to only change the nominal bit rate:

mediainfo bunny_1s_gop.mp4
Bit rate                                 : 39.9 Mb/s
Nominal bit rate                         : 2 000 kb/s
Width                                    : 1 920 pixels
Height                                   : 1 080 pixels

Changing more AVCodecContext rate controls parameters don't seem to affect:

AVCodecContext->bit_rate = 2000 * 1000;
AVCodecContext->rc_max_rate = 2000 * 1000;
AVCodecContext->rc_min_rate = 2000 * 1000;
AVCodecContext->rc_buffer_size = 2 * 2000 * 1000;

mediainfo
Bit rate                                 : 27.8 Mb/s
Nominal bit rate                         : 2 000 kb/s

Or even the x264opts bitrate:

  av_opt_set(sc->video_avcc->priv_data, "x264opts", "bitrate=2000:keyint=60:min-keyint=60:scenecut=-1", 0);

Or the generic options (minrate, maxrate, bufsize) via priv_data:

  av_opt_set(sc->video_avcc->priv_data, "minrate", "2M", 0);
  av_opt_set(sc->video_avcc->priv_data, "maxrate", "2M", 0);
  av_opt_set(sc->video_avcc->priv_data, "bufsize", "4M", 0);

mediainfo 
Bit rate                                 : 10.6 Mb/s

Tried codec private options, it didn't work as well:

  av_dict_set(&encoder_private_options , "b", "2.0M", 0);
  av_dict_set(&encoder_private_options , "minrate", "2.0M", 0);
  av_dict_set(&encoder_private_options , "maxrate", "2.0M", 0);
  av_dict_set(&encoder_private_options , "bufsize", "4.0M", 0);

leandromoreira commented 4 years ago

I tried to control rate by:

  int M = 100; // when I use 1000 it gets worst.
  sc->video_avcc->bit_rate = 2100 * M;
  sc->video_avcc->rc_max_rate = 2600 * M;
  sc->video_avcc->rc_min_rate = 2600 * M;
  sc->video_avcc->rc_buffer_size = 2 * 2600 * M;

The final bit rate is still no how it was suppose to be:

Bit rate                                 : 4 932 kb/s
Nominal bit rate                         : 210 kb/s

When I set `force-crf

av_opt_set(sc->video_avcc->priv_data, "x264opts", "keyint=60:min-keyint=60:scenecut=-1:bitrate=1700:force-cfr=1", 0);

It behaves better but not as expected:

Bit rate                                 : 1 388 kb/s
Nominal bit rate                         : 1 700 kb/s

Only after I set up force-cfrplus AVCodecContext->rc_*:

av_opt_set(sc->video_avcc->priv_data, "x264opts", "keyint=60:min-keyint=60:scenecut=-1:force-cfr=1", 0);

AVCodecContext->video_avcc->bit_rate = 2 * 1000 * 1000;
AVCodecContext->video_avcc->rc_buffer_size = 4 * 1000 * 1000;
AVCodecContext->video_avcc->rc_max_rate = 2 * 1000 * 1000;
AVCodecContext->video_avcc->rc_min_rate = 2.5 * 1000 * 1000;

Bit rate                                 : 2 000 kb/s

leandromoreira commented 4 years ago

While I'm trying to figure out all the issues I went deeper into ffmpeg (the command line) "transcode" path source code:

main

And I noticed that there are lots of lines of code to take care of guessing the frame rate, set up the context (timing, streaming params) to encode a video stream and many others checks and adjustments.

I think maybe it'll be too time-consuming for me and too hard for the readers to grasp 😞 therefore I think I'll reduce the chapter to: transmuxing (mp4->mp4, mp4->fmp4) and transcoding (h264->h264 fixed gop, h264->h265)

leandromoreira commented 4 years ago

I just put libvpx-vp9 as the video codec and it worked BUT again with a huge bit rate 😕 and it took too long even for a 10 second video.

Stream #0:0(und): Video: vp9, yuv420p(tv), 1920x1080, 93847 kb/s, 60 fps, 60 tbr, 15360 tbn, 15360 tbc (default)