MSU Video Quality Metrics Benchmark Invitation

[msm1rnov]

Hello! We kindly invite you to participate in our video quality metrics benchmark. You can submit MANIQA to the benchmark, following the submission steps, described here. The dataset distortions refer to compression artifacts on professional and user-generated content. The full dataset is used to measure methods overall performance, so we do not share it to avoid overfitting. Nevertheless, we provided the open part of it (around 1,000 videos) within our paper "Video compression dataset and benchmark of learning-based video-quality metrics", accepted to NeurIPS 2022.

[TianheWu]

Hi, thanks for inviting! MANIQA is the metric of image. I don't know whether it can valid the video quality.

[msm1rnov]

Thank you for your answer! Our benchmark includes both VQA and IQA methods. The participants page shows the type of each method. Image quality metrics are calculated for each frame independently. Then obtained scores are averaged throughout the whole video. Even such a simple temporal pooling strategy sometimes leads to correlation higher than that of video quality metrics (method UNIQUE as a great example).

[TianheWu]

Thanks！I will have a look!

[dmumtaz]

Hi Did you try it as a video quality metric? Please share the code.

[TianheWu]

@dmumtaz Not yet, I did some other researches previous. In the future, I will try it as a video quality metric！