search

gianni-rosato / svt-av1-psy

The Scalable Video Technology for AV1 (SVT-AV1 Encoder and Decoder) with perceptual enhancements for psychovisually optimal AV1 encoding
BSD 3-Clause Clear License
198 stars 17 forks source link

[ENHANCEMENT] Please add test results #36

Open ioctl-user opened 5 months ago

ioctl-user commented 5 months ago

Please consider adding BD-Rate curves using a few indicative video to compare this project with the vanilla svt-av1.

I mean tables and pictures like here: https://github.com/fraunhoferhhi/vvenc/wiki/Encoder-Performance

This could help people to see where the svt-av1-psy project is better.

gianni-rosato commented 5 months ago

Thanks for the request! As you likely know, our focus is perceptual fidelity per bit, not necessarily metric performance. I'll consider this if we can find compelling examples, but I believe visual comparisons would be more meaningful.

ioctl-user commented 5 months ago

Looks like some options: --presets -2 and -3, --variance-boost-strength, --variance-octile, tuned quantization matrices parameters could improve quality with objective metrics like PSNR or SSIM too.

From the other hand, there are well known metrics like VMAF set that have good correlation with the subjective quality perception. In addition you can show difference in any other metric like ssimulacra2 or so on.

By the way, you also can later suggest codec to the complex comparison [here] (http://www.compression.ru/video/codec_comparison/index_en.html) for the next year.

gianni-rosato commented 5 months ago

I would suggest you read the PSY Development page linked in the README.

While some of our improvements result in better metric scores, this is not and won't ever be our goal, which means metric scores may regress over time as we focus on improving visual fidelity. We look at specific metrics for PRs since it would be taxing to do manual community review of every potentially salient change. I would look elsewhere if you are interested in boosting metrics. VMAF in particular is well-known to be easily tricked by contrast-adaptive sharpening, which inherently isn't related to fidelity.

I will consider disclosing some BD-rate graphs for SSIMULACRA2, XPSNR, and potentially VMAF NEG at some point, but that will likely be outside of this repo. I'll leave this issue open to link to those comparisons if they ever arise.

ioctl-user commented 5 months ago

OK.

Publishing DB-rate graphs someday will be nice.

Anyway, just keep in mind option sending codec to the MSU Video Codecs Comparisons (see link above). They compare codecs across several dozens of metrics including blind subjective testing.