[BUG] Quality drops significantly

[fpokryvk]

Describe the bug Here is screenshot of 4K video before and after overlay, the quality drop is more than noticeable, and the motion in video is teary and unwatchable. Also, encoding utilizes only half of the cores available, is there some configuration for that?

Screenshots Before:

After:

[progweb]

Hi

You can use these 3 options to set bitrate :

• video-bitrate = 16000000
• video-min-bitrate = 0
• video-max-bitrate = 32000000 Values are here the default values (16MB, 0 and 32MB)

All other encoder settings are the same that the input video.

At last, about cpu cores usage, gpx2video is based on ffmpeg. If you decode / encode your video in using ffmeg tool, you should get the same result that gpx2video with an empty layout file. On my i7 whole cores are used.

[fpokryvk]

Thank you, quality seems to be fixed (4K@60 needs at least double of default). Weird that if I try other video, all cores are utilized, probably some ffmpeg bug. Maybe mention the options in the readme and -h.

[progweb]

Options not yet documented because it's new and not tested :)

[fpokryvk]

Now I noticed even double bitrate is not sufficient for some videos, even though input bitrate is 60mbit, but it is H265, is there any option to preserve H265?

[progweb]

Can you give me ffprobe output for input video and output video rendered with gpx2video ?

[fpokryvk]

input:

# ffprobe ../../gopro/GX010013_1699097123740.MP4 
ffprobe version 5.1.3 Copyright (c) 2007-2022 the FFmpeg developers
  built with gcc 11 (GCC)
  configuration: --prefix=/usr --bindir=/usr/bin --datadir=/usr/share/ffmpeg --docdir=/usr/share/doc/ffmpeg --incdir=/usr/include/ffmpeg --libdir=/usr/lib64 --mandir=/usr/share/man --arch=x86_64 --optflags='-O2 -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -march=x86-64-v2 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection' --extra-ldflags='-Wl,-z,relro -Wl,--as-needed -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 ' --extra-cflags=' -I/usr/include/rav1e' --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libvo-amrwbenc --enable-version3 --enable-bzlib --enable-chromaprint --disable-crystalhd --enable-fontconfig --enable-frei0r --enable-gcrypt --enable-gnutls --enable-ladspa --enable-libaom --enable-libdav1d --enable-libass --enable-libbluray --enable-libbs2b --enable-libcdio --enable-libdrm --enable-libjack --enable-libjxl --enable-libfreetype --enable-libfribidi --enable-libgsm --enable-libilbc --enable-libmp3lame --enable-libmysofa --enable-nvenc --enable-openal --enable-opencl --enable-opengl --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librav1e --enable-librubberband --enable-libsmbclient --enable-version3 --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libsvtav1 --enable-libtesseract --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libv4l2 --enable-libvidstab --enable-libvmaf --enable-version3 --enable-vapoursynth --enable-libvpx --enable-vulkan --enable-libshaderc --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-libxml2 --enable-libzimg --enable-libzmq --enable-libzvbi --enable-lv2 --enable-avfilter --enable-libmodplug --enable-postproc --enable-pthreads --disable-static --enable-shared --enable-gpl --disable-debug --disable-stripping --shlibdir=/usr/lib64 --enable-lto --enable-libmfx --enable-runtime-cpudetect
  libavutil      57. 28.100 / 57. 28.100
  libavcodec     59. 37.100 / 59. 37.100
  libavformat    59. 27.100 / 59. 27.100
  libavdevice    59.  7.100 / 59.  7.100
  libavfilter     8. 44.100 /  8. 44.100
  libswscale      6.  7.100 /  6.  7.100
  libswresample   4.  7.100 /  4.  7.100
  libpostproc    56.  6.100 / 56.  6.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '../../gopro/GX010013_1699097123740.MP4':
  Metadata:
    major_brand     : mp41
    minor_version   : 538120216
    compatible_brands: mp41
    creation_time   : 2023-11-04T10:37:39.000000Z
    firmware        : H22.01.02.20.00
  Duration: 00:05:11.18, start: 0.000000, bitrate: 60012 kb/s
  Stream #0:0[0x1](eng): Video: hevc (Main) (hvc1 / 0x31637668), yuvj420p(pc, bt709), 3840x2160 [SAR 1:1 DAR 16:9], 59737 kb/s, 59.94 fps, 59.94 tbr, 60k tbn (default)
    Metadata:
      creation_time   : 2023-11-04T10:37:39.000000Z
      handler_name    : GoPro H.265
      vendor_id       : [0][0][0][0]
      encoder         : GoPro H.265 encoder
      timecode        : 11:36:57:44
    Side data:
      displaymatrix: rotation of -180.00 degrees
  Stream #0:1[0x2](eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 189 kb/s (default)
    Metadata:
      creation_time   : 2023-11-04T10:37:39.000000Z
      handler_name    : GoPro AAC  
      vendor_id       : [0][0][0][0]
      timecode        : 11:36:57:44
  Stream #0:2[0x3](eng): Data: none (tmcd / 0x64636D74) (default)
    Metadata:
      creation_time   : 2023-11-04T10:37:39.000000Z
      handler_name    : GoPro TCD  
      timecode        : 11:36:57:44
  Stream #0:3[0x4](eng): Data: bin_data (gpmd / 0x646D7067), 63 kb/s (default)
    Metadata:
      creation_time   : 2023-11-04T10:37:39.000000Z
      handler_name    : GoPro MET  
Unsupported codec with id 0 for input stream 2
Unsupported codec with id 98314 for input stream 3

output:

# ffprobe gx13_1.mp4 
ffprobe version 5.1.3 Copyright (c) 2007-2022 the FFmpeg developers
  built with gcc 11 (GCC)
  configuration: --prefix=/usr --bindir=/usr/bin --datadir=/usr/share/ffmpeg --docdir=/usr/share/doc/ffmpeg --incdir=/usr/include/ffmpeg --libdir=/usr/lib64 --mandir=/usr/share/man --arch=x86_64 --optflags='-O2 -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -march=x86-64-v2 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection' --extra-ldflags='-Wl,-z,relro -Wl,--as-needed -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 ' --extra-cflags=' -I/usr/include/rav1e' --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libvo-amrwbenc --enable-version3 --enable-bzlib --enable-chromaprint --disable-crystalhd --enable-fontconfig --enable-frei0r --enable-gcrypt --enable-gnutls --enable-ladspa --enable-libaom --enable-libdav1d --enable-libass --enable-libbluray --enable-libbs2b --enable-libcdio --enable-libdrm --enable-libjack --enable-libjxl --enable-libfreetype --enable-libfribidi --enable-libgsm --enable-libilbc --enable-libmp3lame --enable-libmysofa --enable-nvenc --enable-openal --enable-opencl --enable-opengl --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librav1e --enable-librubberband --enable-libsmbclient --enable-version3 --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libsvtav1 --enable-libtesseract --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libv4l2 --enable-libvidstab --enable-libvmaf --enable-version3 --enable-vapoursynth --enable-libvpx --enable-vulkan --enable-libshaderc --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-libxml2 --enable-libzimg --enable-libzmq --enable-libzvbi --enable-lv2 --enable-avfilter --enable-libmodplug --enable-postproc --enable-pthreads --disable-static --enable-shared --enable-gpl --disable-debug --disable-stripping --shlibdir=/usr/lib64 --enable-lto --enable-libmfx --enable-runtime-cpudetect
  libavutil      57. 28.100 / 57. 28.100
  libavcodec     59. 37.100 / 59. 37.100
  libavformat    59. 27.100 / 59. 27.100
  libavdevice    59.  7.100 / 59.  7.100
  libavfilter     8. 44.100 /  8. 44.100
  libswscale      6.  7.100 /  6.  7.100
  libswresample   4.  7.100 /  4.  7.100
  libpostproc    56.  6.100 / 56.  6.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'gx13_1.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf59.27.100
  Duration: 00:05:11.17, start: 0.000000, bitrate: 61287 kb/s
  Stream #0:0[0x1](und): Video: h264 (High) (avc1 / 0x31637661), yuvj420p(pc, progressive), 3840x2160 [SAR 1:1 DAR 16:9], 61148 kb/s, 59.94 fps, 59.94 tbr, 60k tbn (default)
    Metadata:
      handler_name    : VideoHandler
      vendor_id       : [0][0][0][0]
    Side data:
      displaymatrix: rotation of -180.00 degrees
  Stream #0:1[0x2](und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 128 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
      vendor_id       : [0][0][0][0]

[progweb]

OK, I understand the issue.

For 4k video, I have to use hevc / h265 instead of h264. And I have to set CRF with a bigger value.

I’ll fix it in few days, thank you for your report.

[progweb]

Hi,

From the last commit 726d92a, you can choose video codec encoder settings.

--video-codec=h264 --video-codec=hevc

By default, h264 is set.

In H264, I set crf to 27 In HEVC, I set crf to 31

Actually, you can't change this value from command line. It's hardcoded in 'src/encoder.cpp' (line: 360)

              if (codec_id == AV_CODEC_ID_H264) 
                  av_opt_set(codec_context->priv_data, "crf", "27", AV_OPT_SEARCH_CHILDREN); 
              if (codec_id == AV_CODEC_ID_HEVC) 
                  av_opt_set(codec_context->priv_data, "crf", "31", AV_OPT_SEARCH_CHILDREN); 

If all is OK, I'll add documentation and I'll close the issue.

[fpokryvk]

Great news! It seems to work. CRF value 31 produces pretty bad results, but I can change that and recompile to suit my needs (thanks for the location in cpp file, I already wanted to play with quality settings of x264, but was too lazy to study what to change). HEVC seems to work fine, thanks a lot!

Meanwhile, I found some other issue, so I will open another :) , feel free to close this.

[fpokryvk]

I have even tried CRF 1, and it the result is still not satisfactory... The video looks fine, except bottom where the numbers are - they are basically unreadable with plenty of artifacts around.

I see this in the output q=2-31 seems suspicious.. maybe CRF is ignored?

  Stream #0:0: Video: hevc, yuvj420p(pc), 3840x2160 [SAR 1:1 DAR 16:9], q=2-31, 64000 kb/s, 59.94 tbn

[progweb]

I’m going to continue to test :(

[fpokryvk]

 av_opt_set(codec_context->priv_data, "preset", "fast", AV_OPT_SEARCH_CHILDREN);

This works for me for hevc, quality is much better, probably default is ultrafast...

[fpokryvk]

But some frames are still like this... :( which breaks the motion a lot... no idea what is causing that...

It is with this:

av_opt_set(codec_context->priv_data, "preset", "medium", AV_OPT_SEARCH_CHILDREN);
av_opt_set(codec_context->priv_data, "crf", "15", AV_OPT_SEARCH_CHILDREN);

[progweb]

In using ffmpeg command line tool, can you encode your video without issue ? If yes, we have to look all the options set by default.

[progweb]

Please try

av_opt_set(codec_context->priv_data, "crf", "-1", AV_OPT_SEARCH_CHILDREN); 

[fpokryvk]

crf seems to have no effect (even with -1, the result is the same), maybe bug in my system? I have ffmpeg/libavformat from epel 9, which is probably not that new...

[fpokryvk]

I tired preset veryslow for 2s of video... it took ages and quality is still noticeably worse than original... It must be some bug in x265 or configuration..

encoded 120 frames in 557.87s (0.22 fps), 8500.74 kb/s, Avg QP:50.34

And I also see the following message (might be related):

x265 [warning]: poc:0, VBV underflow (-340864 bits)

I will try to find some parameters that work reasonably well, so far thanks for help!!!

[fpokryvk]

OK, I finally found it... I removed ffmpeg-free-devel, installed ffmpeg-devel instead and after recompile the result is much better (took me ages to realize, sorry). Default preset seems to be fast, but ultrafast makes even nicer quality in my case. So, this works for me:

av_opt_set(codec_context->priv_data, "preset", "ultrafast", AV_OPT_SEARCH_CHILDREN);
av_opt_set(codec_context->priv_data, "crf", "31", AV_OPT_SEARCH_CHILDREN);

Maybe, it would be nice to add --video-preset option to command line... or something more general like --encoder-options preset:untrafast,crf:31. Thanks for patience and help!

[progweb]

Before gpx2video, I didn't know ffmpeg api and audio / video format.

But I understand that crf has to be set to '-1' since I use min / max bitrate.

If you want to use crf, you have to comment the lines where I set bitrate values.

I'll read again documentation and I'll do other test.

At last, as you suggest I'll add preset settings.

[progweb]

Finally:

https://github.com/progweb/gpx2video#video-encoder-settings-in-progress

[fpokryvk]

Great, I have QSV change in progress... I hope to create pull request tomorrow.

[progweb]

Great here, I have started nvidia hardware acceleration support in using nvenc_h264 / nvenc_hevc.

[fpokryvk]

I guess condition on this line should be negated:

https://github.com/progweb/gpx2video/blob/8102fbbd7d405c918bc62aeab1f6063b060e0a37/src/encoder.cpp#L382

[progweb]

Don't worry, please sync on the last commit.

I have just pushed NVEnc H264 & HEVC support.

I think that it will be easier to add QSV support.

Maybe open MR to avoid to add comments here.

[fpokryvk]

OK, thanks! I will study your changes and try to add QSV (I have no nvidia card to test NVEnc)...

[progweb]

I have just pushed a QSV branch.

But I can't test on my laptop :(

[fpokryvk]

I have checked, it needs slight changes, I will add commit to your branch...

[fpokryvk]

Oh, I am unable to push... nevermind, the patch is simple enough. The problem witch crf seems to be, that if unspecified it remains -2.

diff --git a/src/encoder.cpp b/src/encoder.cpp
index 864bf8e..30b935e 100644
--- a/src/encoder.cpp
+++ b/src/encoder.cpp
@@ -356,8 +356,9 @@ bool Encoder::initializeStream(AVMediaType type, AVStream **stream_ptr, AVCodecC
                codec_context->height = settings().videoParams().height();
                codec_context->sample_aspect_ratio = settings().videoParams().pixelAspectRatio();
                codec_context->pix_fmt = FFmpegUtils::overrideFFmpegDeprecatedPixelFormat(settings().videoParams().pixelFormat());
-//TODO!!!
-               codec_context->pix_fmt = AV_PIX_FMT_NV12;
+               // if QSV, set PixelFormat to NV12.
+               if (codec == ExportCodec::CodecQSVH264 || codec == ExportCodec::CodecQSVHEVC) 
+                       codec_context->pix_fmt = AV_PIX_FMT_NV12;
 //             codec_context->framerate = settings().videoParams().frameRate();
                codec_context->time_base = settings().videoParams().timeBase();

diff --git a/src/videorenderer.cpp b/src/videorenderer.cpp
index be3dd32..ee4ac79 100644
--- a/src/videorenderer.cpp
+++ b/src/videorenderer.cpp
@@ -76,8 +76,10 @@ bool VideoRenderer::init(void) {

        switch (video_codec) {
        case ExportCodec::CodecH264:
+       case ExportCodec::CodecQSVH264:
        case ExportCodec::CodecHEVC:
-               if (app_.settings().videoCRF() != -1) {
+       case ExportCodec::CodecQSVHEVC:
+               if (app_.settings().videoCRF() >= 0) {
                        // Set crf value
                        settings.setVideoOption("crf", std::to_string(app_.settings().videoCRF()));
                }

[progweb]

I have merge your dev and open discussion about hw support.

CRF isn't supported by QSV.

Discussion: https://github.com/progweb/gpx2video/discussions/29

[fpokryvk]

I know CRF isn't supported, but it also ignored bitrate settings unless --video-crf -1 was explicitly specified. I think it left the value -2, so it skipped to the branch where crf is set (not equal to -1).

[progweb]

If crf is equal - 2, then it uses bitrate min/max values for h264/hevc (no hw accel). I have to check this point. Maybe there is a bug :)

If you use video-codec option and don't set crf value, then gpx2video sets the default crf value depended on selected codec.

If you set video-crf to '-1', then you force to use min/max bitrate values.

If you set video-crf to positive value, then you use crf compression method and no use min/max bitrate values.

For hw accel (nvenc, qsv and vaapi) you can set 'preset' option and that's all.

[fpokryvk]

Weirdly enough, bitrate works on QSV too... But it seems to me, only video-bitrate and it must be set neraly 10x the actual value and max-bitrate set to -1... I can not find any reliable documentation of QSV... :/

[progweb]

All is now merged in master branch. Command line help is updated with documentation