Setting focus point in cropped video based on content recognition

[forthrin]

Related to #12685:

When cropping landscape video to portrait, most of the time the focus point of the video (ie. the talking person) is usually around the center. But invariably focus is on something that is effectively off-screen because of the cropping. So:

Does FFmpeg have functionality that can analyze a video stream to find the most interesting point of focus?

a) If so and with neglible CPU impact, mpv could run this real-time and feed the output coordinates (x, y) to video-crop b) If too CPU heavy, imagine preprocessing video and storing (x, y) as stream metadata, to be picked up and set by mpv

This type of technology has been around for a while now to find people and faces in photos. Hopefully for this use case it doesn't require learning, but can be applied to any video stream with good results out of the box.

[kasper93]

This is something that you can do in script and certainly not something should be integrated in mpv core. The scripting is there to extend mpv functionality to such features.

[forthrin]

I follow you on that. So, anyone:

• Does FFmpeg have this type of filter? What is the filter called?
• Did someone write this type of script already? (URL?)

(Wouldn't know what phrase to search for...)