[ENHANCEMENT] Please add test results

[ioctl-user]

Please consider adding BD-Rate curves using a few indicative video to compare this project with the vanilla svt-av1.

I mean tables and pictures like here: https://github.com/fraunhoferhhi/vvenc/wiki/Encoder-Performance

This could help people to see where the svt-av1-psy project is better.

[gianni-rosato]

Thanks for the request! As you likely know, our focus is perceptual fidelity per bit, not necessarily metric performance. I'll consider this if we can find compelling examples, but I believe visual comparisons would be more meaningful.

[ioctl-user]

Looks like some options: --presets -2 and -3, --variance-boost-strength, --variance-octile, tuned quantization matrices parameters could improve quality with objective metrics like PSNR or SSIM too.

From the other hand, there are well known metrics like VMAF set that have good correlation with the subjective quality perception. In addition you can show difference in any other metric like ssimulacra2 or so on.

By the way, you also can later suggest codec to the complex comparison [here] (http://www.compression.ru/video/codec_comparison/index_en.html) for the next year.

[gianni-rosato]

I would suggest you read the PSY Development page linked in the README.

While some of our improvements result in better metric scores, this is not and won't ever be our goal, which means metric scores may regress over time as we focus on improving visual fidelity. We look at specific metrics for PRs since it would be taxing to do manual community review of every potentially salient change. I would look elsewhere if you are interested in boosting metrics. VMAF in particular is well-known to be easily tricked by contrast-adaptive sharpening, which inherently isn't related to fidelity.

I will consider disclosing some BD-rate graphs for SSIMULACRA2, XPSNR, and potentially VMAF NEG at some point, but that will likely be outside of this repo. I'll leave this issue open to link to those comparisons if they ever arise.

[ioctl-user]

OK.

Publishing DB-rate graphs someday will be nice.

Anyway, just keep in mind option sending codec to the MSU Video Codecs Comparisons (see link above). They compare codecs across several dozens of metrics including blind subjective testing.

[gianni-rosato]

We have some stuff in the works here related to AVIF, but it won't show up on this repo if it ever materializes. Therefore, this is out of scope for this project.

[ioctl-user]

We have some stuff in the works here related to AVIF, but it won't show up on this repo if it ever materializes. Therefore, this is out of scope for this project.

Isn't this a mistake? Issue was about request of metrics and comparisons of the encoded video, not about AVIF.

[gianni-rosato]

We have some stuff in the works here related to AVIF

Regarding metrics as they pertain to still image coding. Nevertheless, this issue is closed - metrics will not be provided in the SVT-AV1-PSY repo at this time.