scihant / CTPanoramaView

A library that displays spherical or cylindrical panoramas with touch or motion based controls.
MIT License
1.04k stars 89 forks source link

Isn't VR360 just two VR180s (at least for stills)? #57

Open MarqueIV opened 3 years ago

MarqueIV commented 3 years ago

VR180 3D Image Support

Not so much an issue as it is a question. You say you only support 360 deg FoV angles, but all VR360 cameras that I know of are basically two VR180 cameras pointed away from each other, then rendered side-by-side. That's actually how the Vuze and Insta360 Vue switch between 360 mono and 180 stereo/3D views. You fold the 3D 180 cameras around so they're back-to-back and you get 360 footage.

As such, couldn't it be as simple as taking the 180 footage then adding a black rectangle for the 'second' 180 to properly show it with your library? i.e. your library would think it's still working with 360, just with a dead camera so half of the 'sphere' would be black.

If you're talking about the projection mode, then most VR180 cameras use the side-by-side 'circles' for video, but ironically for stills, they use equirectangular projection where the entire frame is filled, just distorted.

Now if you're speaking not about the projection issues, but the format of Google's VR180, for videos, it's as I said above... It has the two 'circles' on either side. But for stills, they do something a little different. Rather than store the images side-by-side, they only show the left eye and save it as a standard jpg. The view from the right eye is actually encoded and shoved in the metadata of the jpeg as a binary stream (along with some other metadata around lens distance and other projection-related data.) That's why you can open them with any standard Jpeg viewers.

...which brings me back to you here. At least for stills in VR180 3D format, you could grab the standard JPEG image, create a double-wide buffer, then render that image in the left side, leave the right side as just black, and treat it like it's a 360 equirectangular projection. Again, when it projects, you'll just see a dead/black half, but that's precisely what one would expect.

Now if you really wanted to get crazy, you could pull out that second 'right' image and render it in a second buffer, put two of your views side-by-side on the screen, using one buffer for the left and the other for the right, and bam... you now have a 3D VR180 viewer when used with a Cardboard viewer.

VR180 3D Sampe Images

Anyway, happy to help experiment with you on this if you want. If you do, to get you started, I'm including a raw VR180 3D JPEG image from the Lenovo Mirage Camera, as well as a second JPEG image where I manually pulled out the right eye and rendered it next to the image from the standard JPEG stream. Using this you may be able to see if you can figure out how to get your library to use VR180, at least in mono. If you do, again, slap a second viewer next to the first with the data from the right eye and congrats... you support VR180 3D still images, which is a start!

I myself personally like them better than videos because it's a moment frozen in time that you can look around as if you were there. Videos you're at the mercy of what the cameraman does/moves/goes. For instance, whenever I go apartment hunting, I bring the VR180 cam and snap images rather than videos. Then later, I can go back home and 'look around' again using my Quest 2 and Pigasus Viewer (after I extract them to the SBS view as I did here since Pigasus doesn't support the embedded right image. They support tons of projection modes and arrangements. Just none with the embedded data like VR180.) I basically take a photo from each 'corner' of a room, then down hallways and such, and I can virtually 'walk around', seeing everything in 3D and I can really plan things out how I want my furniture to go, etc. It's really great and has rejuvenated my use of the Lenovo Cam because again, videos, unless it's sitting on a tripod, are basically useless, to me anyway.

Files

Hallway.vr.jpeg Hallway_LR.jpg

Hallway.vr.jpeg Hallway_LR.jpg