Video compression picture types and quality metrics

[joachimpoutaraud]

In the future, it would be cool to add some video compression picture types visualizations, as well as full-reference quality metrics for evaluating the video files (i.e. VMAF, PSNR and SSIM).

There are many different image quality algorithms that help assess the quality of a given image, they can all be divided into three main categories: no-reference algorithms, reduced reference algorithms, and full-reference algorithms. The main difference between these types of image quality assessments are that the no-reference algorithms need a previously created dataset that is afterwards used to evaluate the degraded video, while the reduced reference algorithms aim to predict the visual quality of distorted images with only partial information about the reference images, and full-reference algorithms need both the degraded and the original video for comparison.

[joachimpoutaraud]

I have updated the toolbox and added the possibility to visualize the I/P/B frames of a video file. This can be useful to get a quick overview of the structure of the video stream: the GOP size, the number of I-frames, the number of consecutive B-frames, the dynamic range of bitrate over time, the estimated scene change points, the complexity of the content in different parts of the video, etc.

Now, you can get a nice visualization of the video compression pictures types using the following code:

source_video = musicalgestures.MgVideo('/path/to/video.avi') # create MgVideo object
source_video.info('frame') # returns a plot showing the IPB frames

More information on the wiki documentation

[joachimpoutaraud]

I have updated the toolbox and added the possibility to compute three video quality metrics, namely: VMAF, SSIM and PSNR. This can be useful for comparing the quality of video codecs or measuring the efficacy of encoding configuration between two video files (original vs processed).

For now the function is accessbile from the _utils.py script, where quality metrics can be computed using the following code:

from musicalgestures._utils import quality_metrics

quality_metrics('path/to/original/video.mp4', 'path/to/processed/video.mp4')
# Possible to specify a metric
quality_metrics('path/to/original/video.mp4', 'path/to/processed/video.mp4', metric='vmaf')