search

gyroflow / gyroflow

Video stabilization using gyroscope data
https://gyroflow.xyz
GNU General Public License v3.0
6.22k stars 264 forks source link

360 video support #800

Closed jumpjack closed 3 months ago

jumpjack commented 3 months ago

Is there an existing feature request for this?

Description

I have a low budget 360 panoramic camera with 360x220 FOV. I know how to convert video/images from its fisheye format into equirectangular format, for viewing into panorama viewers and VR viewers, but I don't understand if this software can stabilize such videos. Unfortunateky the camera does not store gyro/compass data, but I was thinking to attach my android phone to it and start recording at same time the camera and the OpenCameraSensors app on the phone.

Then what...?

AdrianEddy commented 3 months ago

360 support in Gyroflow is not planned, unless someone contributes it. But I doubt it as it's a significant amount of work and a lot of refactoring would need to be done.

jumpjack commented 3 months ago

I can contribute telling you how to use ffmpeg to convert a fisheye image into equirectangular, and then virtualling moving around the camera by changing yaw, pitch and roll

Fisheye to equirectangular:

ffmpeg -i input.png -vf v360=fisheye:e:ih_fov=180:iv_fov=180 -y output.jpg

If you have a fisheye camera pointg up...

image

you will get a quite distorted image, because the ceiling is centered in the equirectangular image:

image

If you want the image to be seen like if camera was horizontal, you must add pitch parameter to compensate the 90° angle:

ffmpeg -i input.png -vf v360=fisheye:e:ih_fov=180:iv_fov=180:pitch=-90 -y output.jpg

... obtaining this:

image

You can also play with yaw and roll, but the three angles (yaw, pitch, roll) combine "weirdly" depending on which order you specify them: the default order used by ffmpeg is ypr , but you can change it adding parameter "rorder=YOUR_SEQUENCE":

ffmpeg -i input.png -vf v360=fisheye:e:ih_fov=180:iv_fov=180:pitch=-90:rorder=pry -y output.jpg

For example, to rotate camera left to put ceiling fan at center of image:

ffmpeg -i input.png -vf v360=fisheye:e:ih_fov=180:iv_fov=180:pitch=-90:yaw=-150:rorder=pry -y output.jpg

image

If you also want to point the camera up to the fan, you must change the pitch:

ffmpeg -i input.png -vf v360=fisheye:e:ih_fov=180:iv_fov=180:pitch=-150:yaw=-150:rorder=pry -y output.jpg

image

You can notice that to point the camera up again, you cannot put the pitch to 0, due to "interference" of changed yaw. Adding roll makes things even more complex...

Although the image looks very distorted, it is intended to be viewed into a VR viewer or into a panorama viewer like this:

https://renderstuff.com/tools/360-panorama-web-viewer/

Using last image, view will appear centered on the fun right from the beginning, which makes the "navigation" in the image with panning quite uncomfortable, but if you use the first image, with flat horizon, everything goes fine.

You can make experiments with ffmpeg online using this site: https://ffmpeg.lav.io/ If you use thumbnail images for testing, it's very fast. Don't forget to change extension of input file to "." to be able to load images rather than videos.

What I am nto able to do is how to use ffmpeg to generate a PRY triplet per each frame of a video, I only know how to process single images.