Framerate is wrong for VVC (H266, libvvenc) encoded video file

[JohnTravolski]

On the latest ffmetrics beta (v1.5b10), it appears to get the framerate wrong for an H266 video file, but it is correct using FFMetrics 1.3.1.

The scores are very low because the frames are not aligned; notice how it stops at frame number 24 instead of 60 like 1.3.1 does.

You can test by taking a 16 bpc RGB PNG sequence and encoding it with:

ffmpeg -framerate 60 -ss 0 -i source_%04d.png -t 1 -y -vcodec rawvideo -pix_fmt rgb48le "lossless.nut"

to create a lossless reference source file, and then:

ffmpeg -framerate 60 -ss 0 -i  source_%04d.png -t 1 -y -c:v libvvenc -preset medium -tier high -qpa 0 -period 1 -threads 16 -vvenc-params bitrate=500M:wavefrontsynchro=1:tiles=2x2 "output.266"

to create the VVC encoded video file to check the stats of.

I'm on ffmpeg version 2024-10-10-git-0f5592cfc7-full_build-www.gyan.dev.

Using ffprobe on the VVC file yields:

[vvc @ 0000019e38010380] Stream #0: not enough frames to estimate rate; consider increasing probesize
Input #0, vvc, from '.\nonoise_libvvenc-2x2_preset_medium_tier_high_qpa_0_period_1_br_500_16thread2x2.266':
  Duration: N/A, bitrate: N/A
  Stream #0:0: Video: vvc (Main 10), yuv420p10le(tv), 3840x2160, 25 fps, 60 tbr, 1200k tbn

on the .nut file:

Input #0, nut, from '.\nonoise_lossless.nut':
  Metadata:
    encoder         : Lavf61.9.100
  Duration: 00:00:00.98, start: 0.000000, bitrate: 24292780 kb/s
  Stream #0:0: Video: rawvideo (RGB0 / 0x30424752), rgb48le, 3840x2160, SAR 1:1 DAR 16:9, 60 tbr, 61440 tbn
    Metadata:
      encoder         : Lavc61.22.100 rawvideo

I don't know why ffprobe shows 25 fps and 60 tbr on the .266 file. If I use the .mp4 container instead of .266 for the file extension for the VVC file, ffprobe will show 60 fps instead,

  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2mp41
    encoder         : Lavf61.9.100
  Duration: 00:00:00.48, start: 0.516016, bitrate: 130756 kb/s
  Stream #0:0[0x1](und): Video: vvc (vvc1 / 0x31637676), yuv420p10le(progressive), 3840x2160, 130723 kb/s, SAR 1:1 DAR 16:9, 60 fps, 60 tbr, 15360 tbn (default)
    Metadata:
      handler_name    : VideoHandler
      vendor_id       : [0][0][0][0]
      encoder         : Lavc61.22.100 libvvenc

but ffmetrics will stop at frame 28 instead of 24 and thus still calculate useless metrics. When using the .mp4 container, both versions of ffmetrics get it wrong and end at 28:

[fifonik]

Thanks for the report. the issue is re-produced. If I'm not mistaken, that issue caused by the fact that ffmpeg not reporting back input stream's framerate for that "lossless.nut".

I will try to make changes fo FFMetrics will guess it based on tbr/tbn/tbc

[fifonik]

Looks like there is another issue caused by the absence of the fps in FFMpeg output that caused FFMetrics crashed when you hover mouse over Media Info.

I fixed both issues and can supply executable for testing if you like.

[JohnTravolski]

Looks like there is another issue caused by the absence of the fps in FFMpeg output that caused FFMetrics crashed when you hover mouse over Media Info.

I fixed both issues and can supply executable for testing if you like.

Yes, I would be happy to test it.

[fifonik]

-file removed-

Here you are. Executable only (let me know when you download it, I will delete the archive from here). Ta.

[JohnTravolski]

Just tested it. I'm still only getting 29 frames (0 to 28) from the .mp4 file and 25 frames (0 to 24) from the .266 file, and consequently terrible scores.

It may be an issue with the VVC files themselves. Although that doesn't explain why the .266 file worked OK on the older ffmetrics version but not this one. Do you have a recent version of ffmpeg with the libvvenc encoder to test with?

[fifonik]

I'm still only getting 29 frames (0 to 28) from the .mp4 file and 25 frames (0 to 24) from the .266 file

I thought this is intentional as option -t 1 in provided command line tells ffmpeg only use 1 second... Also, as I see, the command you provided created Interlaced file. It is asking for problems.

[JohnTravolski]

That is correct, I'm using only one second, but I'm specifying a framerate of 60 fps. Hence there should be 60 frames. If I use a different codec, like libx265 or libx264, with the -t 1 option, I get all 60 in ffmetrics.

I'm reading your edit now; what is an interlaced file? If I'm doing something wrong with the way I encoded it and it's clear to you, please point it out so I'm aware of how to fix it. Thanks.

[fifonik]

Sorry, I have not investigated it too deep. I've noticed the fps detection issue and thought this is it. Will try to investigate it further.

[fifonik]

Here is the idet output for 2 these files:

As you see, the nut one was detected as interlaced, while 266 as progressive. This is why when metric is measured ffmpeg deinterlacing nut file so number of resulting frames halved.

Unfortunately, FFMetrics rely on the information provided by ffmpeg and I cannot fix the issue.

For this particular situation I think I can implement some improvement:

1. Improve interlace/progressive euristic so it will not show 'interlaced' for the 'nut' as in fact it was strange detection results -- number of BFF was about the same as number of TFF
2. If no interlaced/progressive information available, FFMetrics will not allow to run measurements (results will be most probably wrong anyway)
3. Add a new fields in program form for interlace/progressing and for framerate for situations where those were not detected. So user will be able to specify these values. Unfortunately, I'm strongly against adding these fields "per file". Just two field in form.

I cannot think of any other solution.

[fifonik]

BTW, I realised that I have not fixed the fps issue correctly. So fixed it as well as changed the frame type detection heuristics. As a result, for the "nut" file I have (see screenshot above) the field type is -unknown-/?.

For both files frame rate detected as 60 and so on plots you will be able to see 60 frames:

But as for the metric calculation -- it is up to ffmpeg and looks like it is failing in this particular case.

[fifonik]

Changed in 1.5.0 beta 12. The change will not fix your exact issue for the reason I explained earlier.

[JohnTravolski]

I think there's just something wrong with using the .mp4 container with libvvenc.

I just tested it and the metric calculation works OK for the .266 file now. Thanks.

[fifonik]

If you creating such files by yourself (for testing) -- you can slightly modify ffmpeg encoding options so interlacing info will be added into the files. You will not have issues like this while further processing files.

[JohnTravolski]

If you creating such files by yourself (for testing) -- you can slightly modify ffmpeg encoding options so interlacing info will be added into the files. You will not have issues like this while further processing files.

I'm not entirely sure what this means; I'm not familiar with "interlacing info". Are you able to provide a simple example so I can learn?

[fifonik]

You can check this post, for example: https://community.topazlabs.com/t/how-to-know-if-a-video-is-interlaced-or-progressive/67209/11 So I believe you need to provide correct -flags