Is global_motions.trf data meaningless?

[karbiv]

A file created when "debug" option is 1, global_motions.trf; Those numbers in global_motions.trf look random.

[karbiv]

Actually, sorry, it makes sense now. I'm developing a script for a flawless stabilization of wide FOV cameras, converting translation to camera rotation(roll, yaw, pitch) using Hugin Stitcher command line interface.

[georgmartius]

Hi, this sounds fascinating. So you stabilize/ detect motion on each part of the image separately and then want to stitch it together?

[karbiv]

I didn't notice debug option at first, just used several lines in C to dump to a file t.x, t.y, t.alpha(angle).

Script: Converts fisheye(sony action cam) frames to rectilinear with Hugin, then a rectilinear video is created from those frames, for passing it to vidstab. I noticed that numbers in global_motion.trf(from debug option) do not differ much when full frame is stabilized or only the center part(crop filter in ffmpeg).

A rectilinear image is a tangent function of radius(from optical center). I naively thought that converting frame pixels translations(vidstab) to horizontal and vertical rotations in Hugin would do it. After converting pixel translations to rotations of camera in Hugin somewhat stabilizes wide frame, but it still looks shaky. I though that I calculate rotations not correctly.(tangent of video FOV from center, that tangent divided by width/2, and it's a coeff for translating pixels to rotations)

Maybe it's insufficient preciseness of stabilization data... So another idea was to separate left and right margins of a wide rectilinear frame, where movements are amplified, and stabilize them separately, and apply that stabilization data with coefficients. Or convert those parts in Hugin to the scale of the center part(by rotating them in Hugin towards optical center).

Anyway, I have an automatic script that does everything, I'm testing approaches. I'll upload it to github after exhausting these ideas.

[karbiv]

Checking again... It just struck me that data in global_motions.trf is before low pass filter(Gaussian or other). Very simple fact, but overlooked. It's an exploratory programming in Python, so some nuances were omitted for rapid cognition.

That might be it. The option is named "debug", of course. I think the issue can be closed. I'll continue with low pass filter.

[georgmartius]

Yes, the global_motions are just a dump of the detected motion, as far as I remember. The real processing (low-pass filtering etc) actually done in the second phase. One could modify the code to dump the "to be applied" transformations as well.

[karbiv]

I discovered that and implemented gaussian filter in Python for motion data in "global_motions.trf" file - same kernel and sigma parameter.(but also a simple dump fprintf(f, "...\n", t.x, t.y, t.alpha) for processed values in library's functions for comparison) Transformations are done in Hugin Stitcher - camera's small Euler rotation instead of plane pixel shifts, so practically only first phase(vidstab detect) is used.

The data in global_motions.trf file is relative motions of each frame(after first pass detect). Library converts it to absolute motions("relative" option) before passing to gaussian filter - cameraPathGaussian function.

I'm trying what can be done with raw data from global_motions.trf, filtering.

[georgmartius]

You have to apply the inverse of the difference between original camera path and the smoothed one. so transform by (smoothed_absolute_transform - detected_absolute_transform).

[karbiv]

That gives food for thought. Maybe something that I missed.

[karbiv]

You are talking about these lines(299-301) in transform.c::cameraPathGaussian filter function:

avg = mult_transform(&avg, 1.0/weightsum);
// high frequency must be transformed away
ts[i] = sub_transforms(&ts[i], &avg);

I did it at the beginning, in python implementation of Gaussian filter, but there still was some logical error. Resolved. Results are astonishing for me. Though I need to hone particular camera lens parameters in Hugin. I'll post comparison of videos.

[karbiv]

Original footage(fisheye): https://www.youtube.com/watch?v=-39CehRuflo

Stabilized with libvidstab and Hugin: https://www.youtube.com/watch?v=23wiSYf_ZZA

Wide stabilized video is in rectilinear projection(standard), and can be reprojected to a Panini(or similar) projection in Hugin, which geometrically would be less wide but with the same FOV.

I haven't seen software stabilization of wide FOV videos before. For some reason all demos are narrow, tunnel vision examples. Python program that I wrote(command line) needs some simplifications of its interface.

[georgmartius]

Great, the result looks really good.

[karbiv]

Yes, that stabilization works almost like magic.