search

facebookarchive / 360-Capture-SDK

A developer focused sample SDK that allows game and virtual Reality devs to be able to easily and quickly integrate 360 photo/video capture capability into their game apps.
https://github.com/facebook/360-Capture-SDK
Other
253 stars 75 forks source link

Stereoscopic Video tips #16

Open tdowgielewicz opened 7 years ago

tdowgielewicz commented 7 years ago

Hi people. I would like to implement stereoscopic video render with this sdk. As i find my self rather hobbist in Unity than pro. I want to ask for any tips how to approach this problem.

Regards

MosheBenZacharia commented 7 years ago

This plugin does not support stereoscopic recording, I would suggest "VR Panorama 360 PRO Renderer" from the Unity Asset Store: https://www.assetstore.unity3d.com/en/#!/content/35102

tdowgielewicz commented 7 years ago

Yup. It does not support it yet. However i would like to implement that feature. That's why i'm asking for good way to approach this problem :)

cg439 commented 7 years ago

There's a correct method of approaching this which involves doing things to get stereo renderings for each vertical slice so you can ensure you have stereo overlap in each direction, and there's an incorrect way you could approach this of doing the equivalent of having two "cameras" in the scene that simulate ipd distance between two eyes and do a 360 capture equivalent to top bottom/left right eye combinations.

The latter is likely something you could experiment with using the current SDK in a simple scene starting with offline capture of a scripted environment and merging the content to see if the 3D causes eye strain or not and is actually 3D looking around or even just in front of you.

The former requires a reworking of the core capture script logic.

Ideally you'd do something like multiple cameras for overlap and then use something like optical flow to properly create 3D content but that's unlikely to occur in realtime without multiple GPUs

cg439 commented 7 years ago

The latter is essentially how 180 stereo works, and why 180 stereo isn't great example of stereo as it only has stereo convergence directly in front of you. We'll be exploring stereo support likely later this year but probably after exploring 180 support.

Arakade commented 6 years ago

Hi @cg439 May I ask what's the latest on stereoscopic capture please? I notice fbc_setVodCaptureSettings() includes mention of STEREO_MODE. It's not settable in the current FBCaptureSDK.cs and adding a facility to set it seemingly has no effect (mono video is still produced). If your "latter" from above is proving too troublesome, could we perhaps have the "prior" option until it's ready? (perhaps with a separate flag that highlights its limitations?) Thanks

grancia commented 6 years ago

We're not supporting stereoscopic capture yet and not on the roadmap of this year either . How about testing with RGB-D which is giving you more immersive on watching video?

Arakade commented 6 years ago

Hi, @grancia Thanks for replying! Do you have info on uploading that to YouTube for 3D viewing? I was using https://support.google.com/youtube/answer/6316263 for over-under upload but don't see RGB-D mentioned there. (I did a little Googling but haven't found yet so would really appreciate any tips/links!) Thanks again.

cg439 commented 6 years ago

You can render the rgb-d to a typical stereoscopic 360 video since the d in rgb-d represents depth maps. Just choose a specific renderer with certain values for the camera state and have it render out both eyes, and you should be able to take those snapshots as a 360 3D video and then use typical stereoscopic metadata to upload to FB or Youtube.

Or you can try out some of the new rgbd-360 viewers. Perhaps look at reddit.com/r/6dof for example players of rgbd-360 videos.

On Sun, Jun 10, 2018 at 3:08 AM, Rupert Key notifications@github.com wrote:

Hi, @grancia https://github.com/grancia Thanks for replying! Do you have info on uploading that to YouTube for 3D viewing? I was using https://support.google.com/youtube/answer/6316263 for over-under upload but don't see RGB-D mentioned there. (I did a little Googling but haven't found yet so would really appreciate any tips/links!) Thanks again.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/facebook/360-Capture-SDK/issues/16#issuecomment-396037000, or mute the thread https://github.com/notifications/unsubscribe-auth/AFpcsHFAuBYZJh0VRrBV1NbfexcIbR-8ks5t7PAVgaJpZM4OcbO5 .