search

lubosz / gst-plugins-vr

Virtual Reality plugins for GStreamer
GNU Lesser General Public License v2.1
51 stars 19 forks source link

Using with 3d camera #19

Open albfan opened 2 years ago

albfan commented 2 years ago

I'm using this plugin with a 3d camera (insta360 air) and is almost working.

gst-launch-1.0 v4l2src device=/dev/video2 ! h264parse ! avdec_h264 ! autovideosink

Captura desde 2022-07-05 10-45-58

Using vrcompositor:

gst-launch-1.0 v4l2src device=/dev/video2 ! h264parse ! avdec_h264 ! decodebin ! glupload ! glcolorconvert ! vrcompositor ! gtkglsink

Captura desde 2022-07-05 10-40-21

I use pipeviz to see vrcompositor options:

Captura desde 2022-07-05 10-43-40

Is there a chance to add options to modify the transformation? the join of images creates an unwanted line and some black borders.

something like camera_type=360 and where to look at, like:

Captura desde 2022-07-05 10-52-01

I'm unsure about the parameters to define that.

lubosz commented 2 years ago

The gst-plugins-vr projection is currently very limited to equirect 360 mono. You can find more in my current iteration of gstreamer vr video in SPHVR: https://gitlab.com/lubosz/sphvr

But SPHVR in fact only focused on publishing formats, where all cameras where processed into one equirect image. In the case of your camera we have 2 cameras / viewports that show a half sphere. Currently I am just using multiple viewports for stereo. Will require some light patching of the projection in the SPHVR shaders. Also the SPHVR should go into gst-plugins-vr probably, as they are much more potent and don't work with vertex meshes but do the whole projection in GLSL. Will be thinking a bout a solution here.

albfan commented 3 weeks ago

Hi, I found a project where all the maths for my camera (insta360 air) https://github.com/careyer/Insta360-Air-remap/tree/master/Insta360-Air-remap. It end with a x and y map for ffmpeg:

https://trac.ffmpeg.org/wiki/RemapFilter

Do shaders https://gitlab.freedesktop.org/monado/demos/xrgears/blob/master/shaders/sky_plane_equirect.frag work in same way as maps from ffmpeg remap?