fifonik / FFMetrics

Visualizes Video Quality Metrics (PSNR, SSIM & VMAF) calculated by ffmpeg.exe
570 stars 22 forks source link

[Not a bug] Interpreting results from Movavi Video Editor #58

Closed fu-hsi closed 3 years ago

fu-hsi commented 3 years ago

Are these results trustworthy? ;) All video clips are encoded with the same video editor. I collected all data from ffmetrics and use them for made below chart.

Maybe the program (Movavi) uses strange parameters for the hardware encoding. Although the bitrate is much higher, the quality is much worst. We spend more time for encoding but ultimately we get better quality and less file size.

If we use ffmpeg with h264_qsv for manual encoding with hardware acceleration we can able get much better quality than from video editor. I suppose this is not QuickSync fault, but video software problem (nothing was edited).

It's only for educational purpose. More bitrate doesn't give you better quality in Movavi Video Editor.

There is also a chance that maybe something is not working here, but I don't thing so. What do you think about it?

PS. English is not my native language, sorry about that ;) I reserve the right to correct mistakes.

Movavi Plus 2020

fifonik commented 3 years ago

It is known that HW encoders are much faster, but produce worse quality results on the same bitrate. Software is setting up the HW encoder, so it is very possible that the same HW encoder used from different programs gives you different quality on the same bitrate. Quality is also depend on exact HW used. For example, last AMD GPUs cannot use B-frames, so they produce quality even worse that older generations. Unfortunately, I cannot comment on QSV and NVENC as I do not have any of these technologies in my PC for a long time. I heard that modern hi-end NVidia GPUs have much better HW encoders that overperforms quality wise other HW encoders. However, I have not tested it myself.

It is also possible that the issue lies with metrics calculation. First: You may want to check FFMpeg Limitations section on project's home page (I've created #46 for this). Second: VMAF metric is affected by some image "improvements". This is why you may want to use NEG VMAF metric as some software could apply such improvements before encoding.

As for myself, I'm using HW encoding only to render drafts. All final renders I'm doing on CPU (x264/x265).

fu-hsi commented 3 years ago

I also render manually with HW using ffmpeg and with the same bitrate I get similal result to the software encoding but much faster. It's to big difference in quality for me:

ffmpeg -n -loglevel warning -hide_banner -stats -i 00043.MTS -pix_fmt yuv420p -c:v h264_qsv -q 23 -c:a copy "00043_50fps_qsv.MP4"

Now there is no noticeable differences to software encoding:

ffmpeg -n -loglevel warning -hide_banner -stats -i 00043.MTS -pix_fmt yuv420p -c:v libx264 -crf 23 -c:a copy "00043_50fps.MP4"

And yes, I know -q is not the same use -crf, but we get similar file output size and bitrate. No big difference between HW and Software.

Yes, HW quality is different in different generations of Intel processors.

fifonik commented 3 years ago

I do not really got your point. Are you saying that HW encoding is good for you? Well, use it then. However, it is not good for me (yet?).

P.S. Can I ask you to download my archive (~250MB) and using HW encoder render file 2012-11-22_095841.m2ts with average target video bitrate ~20 Mbps (you can use any profile that gives such average bitrate at the end). Then share your result please. I'd like to have a look.

fu-hsi commented 3 years ago

If we use ffmpeg directly, there is no big difference between HW and Software encoding with similar bitrate. There must be a problem with Video Editor and your app just prove that. Nothing else. All in all, I don't think I will use hardware acceleration in this editor anymore.

I downloaded your data (it's almost static and easier to encode I think) and used these commands:

ffmpeg -i 2012-11-22_095841.m2ts -c:v h264_qsv -b:v 20M -c:a copy "2012-11-22_095841.qsv.MP4"   
ffmpeg -i 2012-11-22_095841.m2ts -c:v libx264 -b:v 20M -c:a copy "2012-11-22_095841.x264.MP4"   

QSV - Intel Quick Sync Video.

File size and quality is very close. But when we use Movavi Editor for the same task, we get a big difference in quality and no difference in file size. Maybe Movavi adds some invisible filters that degrades image in some way which causes it to not be compared with the reference file. The aforementioned video editor also produces inaccurate bitrate (50.03 and 50.08):

Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], 19933 kb/s, 50.03 fps, 50 tbr, 12800 tbn, 100 tbc (default)

All results in one place: Results

This post is for informational purposes only, although not about ffmetrics :) Encoded files: https://drive.google.com/drive/folders/1U-srIfoyrkMv-gN-ibpOpwP8t-Tdn3CF

My eye doesn't see differences, but VMAF... ¯_(ツ)_/¯

fifonik commented 3 years ago

Thanks. This is probably the best quality created with HW encoder that I see.

fu-hsi commented 3 years ago

The last test on CyberLink 16 and Intel QuickSync Video enabled. Similar results to Movavi.

This is probably the best quality created with HW encoder that I see.

That's how it looks on my i3 (Ivy Bridge): 87.5 vs 80.6 VMAF. I close the ticket.

fifonik commented 3 years ago

That's how it looks on my i3 (Ivy Bridge): 87.5 vs 80.6 VMAF.

Thanks.

This is a bit unexpected as if I remember correctly, previously I asked people to render this using QSV and they provided files rendered with about the same intel CPU generation. However, I do not know what encoder they user, most probably not from ffmpeg. If it possible that QSV detection failed and ffmpeg switched back to CPU encoder? I do not know how to check it, and only speed should give some ideas about that. Was it big difference in speed between ffmpeg/qsv and ffmpeg/cpu on your system?

P.S I do have Intel system nearby, will definitely try to play with ffmpeg encoder.

fifonik commented 3 years ago

I've already rendered on the PC. QSV speed was much higher and quality wise they are similar. Wow.

fu-hsi commented 3 years ago

If it possible that QSV detection failed and ffmpeg switched back to CPU encoder?

I don't think so, because time consumption is very different, but who knows...

fu-hsi commented 3 years ago

I've already rendered on the PC. QSV speed was much higher and quality wise they are similar. Wow.

If you used ffmpeg for encoding, it's true. But when I use video editors, it's not.