Netflix / vmaf

Perceptual video quality assessment based on multi-method fusion.
Other
4.65k stars 755 forks source link

add still model that uses motion_force_zero #1391

Open cosmin opened 1 month ago

cosmin commented 1 month ago

For some use cases it's important to use VMAF with motion forced to zero, such as very low FPS video where we know inter-frame motion is misleading. For example a montage of 1s images encoded at 30fps would have a VMAF motion of 0, but if the same thing is encoded at actually 1fps then inter-frame motion will be very high even though visually it will play the same.

However trying to pass motion_force_zero through ffmpeg doesn't seem to be working correctly, or at least I couldn't figure out how to do it properly. Instead it's easier to create a new model for still mode which passes this feature to the motion feature extractor.

nilfm99 commented 1 month ago

Thanks for the contribution. I don't see an issue with adding a new model, but I'll let @christosbampis take a look.

Regarding passing the flag via ffmpeg, I'll try it out and report back.

li-zhi commented 1 month ago

@cosmin there is already such model existed: https://github.com/Netflix/vmaf/blob/master/model/other_models/vmaf_v0.6.1mfz.json

Sorry the naming was not completely clear. Feel free to hook it up with ffmpeg.

nilfm99 commented 1 month ago

@cosmin Regarding the ffmpeg question, the short answer is that it's best to use a model that already has that feature, as you did. You can ad-hoc set features with options via the ffmpeg filter CLI, but those will be standalone features and won't be part of any model. For example, I computed standalone motion with motion_force_zero=true in this way (number of backslashes to escape depends on your system, I had to use 4):

ffmpeg -i dis5.y4m -i ref5.y4m -lavfi "libvmaf=log_fmt=json:log_path=cosmin.json:feature=name=motion\\\\:motion_force_zero=true" -f null -

Regarding the new model, thanks @li-zhi for linking the existing model, it is identical to the one in this PR.

cosmin commented 1 month ago

Any chance we could add this existing mfz model to built-in models? And perhaps a force motion zero of vmaf-neg. Happy to update the pull request.

li-zhi commented 1 month ago

@cosmin yes, feel free to make an vmaf-eng model with mfz, and add them to the built-in models. Thanks!