[NODES] Stabilize videos

m-sundman commented 5 years ago

It's very useful to stabilize shaky videos. Some features, in order of importance:

Make it 2-phase, separating the analyzing phase (which can take hours, or even days) from the smoothing/zoom/crop/rotate phase (which is quick and is often modified several times). This would save countless hours of redoing stabilization with slightly adjusted settings.
Support true cropping, meaning that the stabilized video is transparent (as opposed to, say, black) in the parts that are outside the input video.
Review and edit the analyzing phase, preferably supporting adding/removing/enabling/disabling control points (the areas that the algorithm tries to keep stable). This is to prevent the algorithm trying to match points which should not be stable, such as a large moving foreground object when trying to keep the background stable (or vice versa).

itsmattkc commented 5 years ago

This feature is currently in progress, the backend is mostly done (utilizing the libvidstab-based filter in FFmpeg, which is 2-phase) but the front end is still in progress. Thanks for filling a feature request and I'll try my best to accommodate your points.

m-sundman commented 5 years ago

In ffmpeg the crop option has only 2 values for how to handle out-of-frame pixels: (a) keep pixels from previous frame, or (b) fill with black. I would like to have 2 more: (c) use image from original, unstabilized frame, or (d) fill with transparency. (Actually, only options (a) and (d) are required, because with option (d) it's trivial to make (b) or (c) by just adding a layer below with black or the original video.)

Furthermore, I'm not sure how easy it is to edit the control points with libvidstab. It might be close to impossible. Maybe it would be possible to at least apply some kind of enable/disable mask to the filter, to prevent, say, a person walking in front of the camera to swoosh the image all over the place, or to discard any motion in certain parts of the video, to prevent the moving foreground from "stabilizing" the video so the background ends up moving.

alcomposer commented 5 years ago

OpenCV has huge potential to be used for (among other things) stabilisation.

itsmattkc commented 5 years ago

I have been looking at OpenCV (upon your suggestion in fact). While it would take longer to implement (and would require adding an extra somewhat large lib), I think the resulting stabilizer would be much faster (my libvidstab testing has been very slow, even with the analysis taking place beforehand). Plus it does seem OpenCV has a number of other features that could be utilized.

On Mon., 24 Dec. 2018, 6:55 pm alcomposer <notifications@github.com wrote:

OpenCV has huge potential to be used for (among other things) stabilisation.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/olive-editor/olive/issues/190#issuecomment-449699150, or mute the thread https://github.com/notifications/unsubscribe-auth/AghHY-uW6tSaXiJDma6XpeEakO0zfYGFks5u8IhqgaJpZM4ZfrR3 .

alcomposer commented 5 years ago

Awesome. Integrating OpenCV would really give so many benefits.

I believe (after doing ldd) on Davinci Resolve that its one of its dependencies. (I don't know if its used for stabilisation).

If OpenCV can analyse the clip, (which is always better than doing the whole media) then possibly the resultant information can be plugged into an OpenGL transform.

This would allow editing of control points, and masking areas.

jonno85uk commented 5 years ago

I think the resulting stabilizer would be much faster (my libvidstab testing has been very slow, even with the analysis taking place beforehand).

I've made a fork of vid.stab and made improvements in performance of the vidstabtransform, but only really noticeable when not encoding (i.e to x264) but certainly improves bicubic interpolation.

I did make a small improvement of the vidstabdetect filter as the amount of threads dedicated was limited to 0.8 of available threads. Not sure why it was set that way.

I need to get around to making a PR but my changes would require a change in ffmpeg for vidstabdtransform to use a bicublin interpolation method via and set the threadcount.

jonno85uk commented 5 years ago

If OpenCV can analyse the clip, (which is always better than doing the whole media) then possibly the resultant information can be plugged into an OpenGL transform.

You can analyse and transform a section of a media with libvidstab, at least via ffmpeg with the -ss and -to switches (not sure how to achieve the same via libav) . libvidstab works on frames given to it, not files.

m-sundman commented 5 years ago

OpenCV would indeed be a better solution. It has pluggable anti-wobbler and motion estimator, as well as in-painting for out-of-area pixels!

CodigoCristo commented 5 years ago

THE MOST IMPORTANT, A VIDEO STABILIZER IN OLIVE EDITOR...

https://www.youtube.com/embed/7Nf4bI_5hgI https://ai.googleblog.com/2012/05/video-stabilization-on-youtube.html

evertvorster commented 3 years ago

Hi there! I am new to Olive, and I like what I am seeing so far. It would be totally awesome if I was able to stabilize videos in-program, as I have a lot of shaky footage.

Currently I am running the git version of Olive, and am happy to do any testing required.

bp88dev commented 1 month ago

This would be an awesome feature to add!

olive-editor / olive

[NODES] Stabilize videos #190