Not showing the calculated PSNR and SSIM

[Count-MHM]

I was using an old build of FFMPEG. (ffmpeg-2022-08-10-git-8fc7f0fdec-full_build) And updated it to the newest one for better AV1 support. (ffmpeg-2024-02-04-git-7375a6ca7b-full_build)

But after updating FFMPEG, FFMetrics' PSNR and SSIM always show "N/A" even though it has calculated them.

ffmpeg-2022-08-10-git-8fc7f0fdec-full_build:

ffmpeg-2024-02-04-git-7375a6ca7b-full_build:

with ffmpeg-2022-08-10-git-8fc7f0fdec-full_build:

2024-02-09 01:05:04 ffmpeg.exe -nostdin -filters
2024-02-09 01:05:04 ffmpeg.exe -hide_banner -f lavfi -i color=c=black:s=320x240:d=1 -lavfi [0:v][0:v]libvmaf=eof_action=endall:n_threads=19:model=version=vmaf_v0.6.1 -f null -
2024-02-09 01:05:04 ffmpeg.exe -hide_banner -f lavfi -i color=c=black:s=320x240:d=1 -lavfi [0:v][0:v]libvmaf=eof_action=endall:n_threads=19:model=version=vmaf_v0.6.1neg -f null -
2024-02-09 01:05:04 ffmpeg.exe -hide_banner -f lavfi -i color=c=black:s=320x240:d=1 -lavfi [0:v][0:v]libvmaf=eof_action=endall:n_threads=19:model=version=vmaf_4k_v0.6.1 -f null -
2024-02-09 01:05:05 ffmpeg.exe -hide_banner -f lavfi -i color=c=black:s=320x240:d=1 -lavfi [0:v][0:v]libvmaf=eof_action=endall:n_threads=19:model_path='vmaf-models/vmaf_4k_v0.6.1.json' -f null -
2024-02-09 01:05:05 ffmpeg.exe -hide_banner -f lavfi -i color=c=black:s=320x240:d=1 -lavfi [0:v][0:v]libvmaf=eof_action=endall:n_threads=19:model_path='vmaf-models/vmaf_v0.6.1.json' -f null -
2024-02-09 01:05:05 ffmpeg.exe -hide_banner -f lavfi -i color=c=black:s=320x240:d=1 -lavfi [0:v][0:v]libvmaf=eof_action=endall:n_threads=19:model_path='vmaf-models/vmaf_v0.6.1neg.json' -f null -
2024-02-09 01:05:05 ffmpeg.exe -hide_banner -t 00:00:02 -i "F:\Raw Videos\Temp-03\Source.avi" -frames:v 30 -filter:v idet -f null -
2024-02-09 01:05:05 ffmpeg.exe -hide_banner -ss 1 -i "F:\Raw Videos\Temp-03\Source.avi" -frames:v 1 -f image2 -vf settb=AVTB,setpts=PTS-STARTPTS,scale=-1:200 C:\Users\MHM\AppData\Local\Temp\3afb112a-72bf-4e5d-a96b-14499098b3cf.bmp
2024-02-09 01:05:05 ffmpeg.exe -hide_banner -t 00:00:02 -i "F:\Raw Videos\Temp-03\AV1-CRF-05.mp4" -frames:v 30 -filter:v idet -f null -
2024-02-09 01:05:08 ffmpeg.exe -hide_banner -nostdin -i "F:/Raw Videos/Temp-03/AV1-CRF-05.mp4" -i "F:/Raw Videos/Temp-03/Source.avi" -lavfi [0:v]settb=AVTB,setpts=PTS-STARTPTS[main];[1:v]settb=AVTB,setpts=PTS-STARTPTS[ref];[main][ref]psnr=eof_action=endall:stats_file='-' -f null -
2024-02-09 01:05:11 ffmpeg.exe -hide_banner -nostdin -i "F:/Raw Videos/Temp-03/AV1-CRF-05.mp4" -i "F:/Raw Videos/Temp-03/Source.avi" -lavfi [0:v]settb=AVTB,setpts=PTS-STARTPTS[main];[1:v]settb=AVTB,setpts=PTS-STARTPTS[ref];[main][ref]ssim=eof_action=endall:stats_file='-' -f null -
2024-02-09 01:05:13 ffmpeg.exe -hide_banner -nostdin -i "F:/Raw Videos/Temp-03/AV1-CRF-05.mp4" -i "F:/Raw Videos/Temp-03/Source.avi" -lavfi [0:v]settb=AVTB,setpts=PTS-STARTPTS[main];[1:v]settb=AVTB,setpts=PTS-STARTPTS[ref];[main][ref]libvmaf=eof_action=endall:n_threads=19:log_fmt=json:log_path='C\:/Users/MHM/AppData/Local/Temp/FFMetrics/53bcbb13-7d7a-411d-b2db-6e655a3f3c6a.log':pool=Mean:model=version=vmaf_v0.6.1 -f null -

with fmpeg-2024-02-04-git-7375a6ca7b-full_build:

2024-02-09 01:07:28 ffmpeg.exe -nostdin -filters
2024-02-09 01:07:28 ffmpeg.exe -hide_banner -f lavfi -i color=c=black:s=320x240:d=1 -lavfi [0:v][0:v]libvmaf=eof_action=endall:n_threads=19:model=version=vmaf_v0.6.1 -f null -
2024-02-09 01:07:28 ffmpeg.exe -hide_banner -f lavfi -i color=c=black:s=320x240:d=1 -lavfi [0:v][0:v]libvmaf=eof_action=endall:n_threads=19:model=version=vmaf_v0.6.1neg -f null -
2024-02-09 01:07:28 ffmpeg.exe -hide_banner -f lavfi -i color=c=black:s=320x240:d=1 -lavfi [0:v][0:v]libvmaf=eof_action=endall:n_threads=19:model=version=vmaf_4k_v0.6.1 -f null -
2024-02-09 01:07:29 ffmpeg.exe -hide_banner -f lavfi -i color=c=black:s=320x240:d=1 -lavfi [0:v][0:v]libvmaf=eof_action=endall:n_threads=19:model_path='vmaf-models/vmaf_4k_v0.6.1.json' -f null -
2024-02-09 01:07:29 ffmpeg.exe -hide_banner -f lavfi -i color=c=black:s=320x240:d=1 -lavfi [0:v][0:v]libvmaf=eof_action=endall:n_threads=19:model_path='vmaf-models/vmaf_v0.6.1.json' -f null -
2024-02-09 01:07:29 ffmpeg.exe -hide_banner -f lavfi -i color=c=black:s=320x240:d=1 -lavfi [0:v][0:v]libvmaf=eof_action=endall:n_threads=19:model_path='vmaf-models/vmaf_v0.6.1neg.json' -f null -
2024-02-09 01:07:29 ffmpeg.exe -hide_banner -t 00:00:02 -i "F:\Raw Videos\Temp-03\Source.avi" -frames:v 30 -filter:v idet -f null -
2024-02-09 01:07:29 ffmpeg.exe -hide_banner -ss 1 -i "F:\Raw Videos\Temp-03\Source.avi" -frames:v 1 -f image2 -vf settb=AVTB,setpts=PTS-STARTPTS,scale=-1:200 C:\Users\MHM\AppData\Local\Temp\c5ade796-9b71-4161-b66a-5056c60b588d.bmp
2024-02-09 01:07:29 ffmpeg.exe -hide_banner -t 00:00:02 -i "F:\Raw Videos\Temp-03\AV1-CRF-05.mp4" -frames:v 30 -filter:v idet -f null -
2024-02-09 01:07:30 ffmpeg.exe -hide_banner -nostdin -i "F:/Raw Videos/Temp-03/AV1-CRF-05.mp4" -i "F:/Raw Videos/Temp-03/Source.avi" -lavfi [0:v]settb=AVTB,setpts=PTS-STARTPTS[main];[1:v]settb=AVTB,setpts=PTS-STARTPTS[ref];[main][ref]psnr=eof_action=endall:stats_file='-' -f null -
2024-02-09 01:07:32 ffmpeg.exe -hide_banner -nostdin -i "F:/Raw Videos/Temp-03/AV1-CRF-05.mp4" -i "F:/Raw Videos/Temp-03/Source.avi" -lavfi [0:v]settb=AVTB,setpts=PTS-STARTPTS[main];[1:v]settb=AVTB,setpts=PTS-STARTPTS[ref];[main][ref]ssim=eof_action=endall:stats_file='-' -f null -
2024-02-09 01:07:35 ffmpeg.exe -hide_banner -nostdin -i "F:/Raw Videos/Temp-03/AV1-CRF-05.mp4" -i "F:/Raw Videos/Temp-03/Source.avi" -lavfi [0:v]settb=AVTB,setpts=PTS-STARTPTS[main];[1:v]settb=AVTB,setpts=PTS-STARTPTS[ref];[main][ref]libvmaf=eof_action=endall:n_threads=19:log_fmt=json:log_path='C\:/Users/MHM/AppData/Local/Temp/FFMetrics/245e05a1-9cab-4c06-a323-7d307bef44b8.log':pool=Mean:model=version=vmaf_v0.6.1 -f null -

returning to the old version fixes the problem.

[Count-MHM]

Going back to the stable 6.1.1 also fixes the issue.

[fifonik]

Could you please run the following command with old and new ffmpeg: ffmpeg -f lavfi -i color=c=black:s=320x240:d=1 -f lavfi -i color=c=white:s=320x240:d=1 -lavfi [0:v][1:v]psnr:stats_file='-' -f null -

The command calculates PSNR over two generated videos (black and white). It is printing per frame values and at the very end -- per file value. The per file value is displayed by ffmetrics.

My current output in linux:

I assume that per file value format is changed in the latest FFMpeg (I will check it out when back to windows), this is why FFMetrics shows N/A.

[fifonik]

Checked it out.

Old ffmpeg: StdOut: n:636 mse_avg:18.99 mse_y:21.86 mse_u:10.32 mse_v:16.22 psnr_avg:35.34 psnr_y:34.74 psnr_u:38.00 psnr_v:36.03 StdErr: frame= 648 fps=280 q=-0.0 Lsize=N/A time=00:00:12.99 bitrate=N/A speed=5.62x
StdErr: video:299kB audio:7308kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown StdErr: [Parsed_psnr_4 @ 000001e27a7c9100] PSNR y:36.233649 u:38.270248 v:38.479232 average:36.836851 min:33.577308 max:39.757382

New ffmpeg: StdOut: n:636 mse_avg:18.99 mse_y:21.86 mse_u:10.32 mse_v:16.22 psnr_avg:35.34 psnr_y:34.74 psnr_u:38.00 psnr_v:36.03 StdErr: [Parsed_psnr_4 @ 000001c794e0f2c0] PSNR y:36.233649 u:38.270248 v:38.479232 average:36.836851 min:33.577308 max:39.757382 StdErr: [out#0/null @ 000001c7941400c0] video:304KiB audio:7308KiB subtitle:0KiB other streams:0KiB global headers:0KiB muxing overhead: unknown StdErr: frame= 648 fps=342 q=-0.0 Lsize=N/A time=00:00:12.96 bitrate=N/A speed=6.84x

The output format did not change. However, in the new ffmpeg I see the information about the latest frame (648 in my case) is following AFTER the information about overall file was returned. I have not expected it and this is causing the issue. Will think how to fix it. Thanks for the bug report.

[Count-MHM]

Thanks!

[fifonik]

Confirming that out of order lines returned by ffmpeg in stderr is exactly the cause of the issue. There is something else really strange with the new version: it is also returning many repeated lines in stderr. I think this is a bug in ffmpeg and I hope they will fix it soon.

Nevertheless, I've already made changes so FFMetrics will use some smarts and not expect that the very last line in stderr contains information about file metric.

[fifonik]

Should be fixed in 1.4.9 beta 1 (could you check it please?)

[Count-MHM]

Can confirm that 1.4.9 beta 1 works with ffmpeg-2024-02-04-git-7375a6ca7b-full_build and the newer ffmpeg-2024-02-15-git-a2cfd6062c-full_build. Thanks