search

georgmartius / vid.stab

Video stabilization library
http://public.hronopik.de/vid.stab/
Other
859 stars 109 forks source link

"wobbly rubber sheet" effect due to "slow" top-down scanning of camera? #131

Open AlainKnaff opened 1 month ago

AlainKnaff commented 1 month ago

Hi,

During a recent trip, I took a couple of videos of animals (marmots, ibexes) with extreme zoom (30x) with my Panasonic DC-TZ202.

As I don't have a steady hand the original videos were shaking quite a bit.

Your tool was godsend, and it could produce videos were the main subject is remarkably steady. Thanks for this wonderful program.

However, the background looks wobbly, a bit as if it was printed on a rubber sheet which was repeatedly pulled apart and released again.

Example:

Original video: http://alain.knaff.lu/Photos/2024-Barcelonette/Video.orig/P1158994.MP4

Processed video (with vid.stab with shakiness=10 and smoothing=0 , and finally cropped to 960x540): http://alain.knaff.lu/Photos/2024-Barcelonette/Video/P1158994.MP4

As these are zoom images, lens distortion is unlikely to be a factor (would come into play for wide angle shots)

A friend of mine thinks that this may be due to the finite time it takes the camera to "scan" the video from top down. I.e. topmost lines are scanned slightly earlier than lines at the bottom. So, if the camera is moving down image tend to get compressed vertically, whereas if it is moving up it tends to get extended. Could it be that vid.stab is compensating for purely geometric shake (i.e. translation, rotation, perspective), but not for this time-based artifact?

Thanks,

Alain

georgmartius commented 1 month ago

Dear Alain, yes, that is true. Rolling shutter, as it is called, it not considered in vid.stab, unfortunately. You might want to try the stabilizer in youtube, if that is an option for you. They compensate for that. Best, Georg

AlainKnaff commented 1 month ago

Thanks for your quick reply. Unfortunately youtube is not an option for me. However, thanks for giving me the name of this phenomenon ("rolling shutter"), that way I can start searching more easily for other solutions online :-)

AlainKnaff commented 1 month ago

Btw, is there any documentation of the file format of transforms.trf available? Maybe something could be done by preprocessing this file after the detect and before the transform step? Thanks

georgmartius commented 1 month ago

There is no proper documentation. But even if, it would not help, as the transformation that computed and are applied to compensate are rigid (apply to the whole frame).

AlainKnaff commented 1 month ago

What do you mean by "rigid"? Linear? If so, then it would be doable to compensate for first order effects (speed of panning, but not acceleration) which might already be an improvement. Indeed, for this we would only need shear (for horizontal panning) and scaling along one axis (for vertical panning), both of which are linear.

georgmartius commented 1 month ago

Yes, it is linear. You are right, that in principle linear transformations would be enough, but these shearings and scalings are not implemented, but might actually not so difficult to implement, but not by changing the transform file.