[TECHNICAL] Implement 360 Degree Video Display Feature

[tobiaswaggoner]

Description:

As a user, I want to start a 360-degree video recording within the application, use the app normally, and stop the recording, so that I can capture immersive video content of my interactions and automatically save it as an MP4 file.

Acceptance Criteria:

1. Start / Stop Recording UI:

   • The user has a way to start and stop the recording (wrist interface or such).
   • A visual indicator (e.g., recording icon) should appear on the screen to confirm that recording is running.

2. Save MP4 File:

   • Upon stopping the recording, the application should automatically save the recorded video as an MP4 file.
   • The file name should be automatically generated using a timestamp format (e.g., 360Capture_YYYYMMDD_HHMMSS.mp4).
   • The location is selected automatically, depending on the device.

3. Error Handling:

   • The recording process handles errors gracefully and ensures partial recordings are still saved if possible.

4. Performance:

   • The recording process does not significantly affect the application's performance or user experience.
   • Especially the FPS does not fall below 30 FPS on Quest 3 headsets

[tobiaswaggoner]

UNTESTET PROPOSAL BY CHATGPT

Setting up a 360-degree camera for recordings in Unity involves using a cubemap rendering technique. This captures the scene from multiple angles and stitches them together into a single panoramic image. Here's a step-by-step guide:

Step-by-Step Guide to Setting Up a 360-Degree Camera

1. Create a New Camera in Unity:

   • In your Unity project, create a new camera by right-clicking in the Hierarchy window and selecting Camera.
   • Rename this camera to something like 360Camera.

2. Position the Camera:

   • Position the camera at the desired location in your scene where you want to capture the 360-degree footage.
   • Ensure the camera's rotation is set to (0, 0, 0) for accurate capture.

3. Create and Configure a Render Texture:

   • In the Project window, create a new Render Texture by right-clicking and selecting Create -> Render Texture.
   • Configure the Render Texture to have the dimensions suitable for 360-degree captures. Common sizes are 4096x2048 or 8192x4096.

4. Script for Capturing 360-Degree Footage:

   • Create a new C# script (e.g., Camera360Capture.cs) and attach it to the camera.
   • Use the following script to capture the 360-degree footage:

using UnityEngine;

public class Camera360Capture : MonoBehaviour
{
    public int captureWidth = 4096;
    public int captureHeight = 2048;
    public string savePath = "360Capture.png";

    private Camera cam;
    private RenderTexture cubemap;
    private Texture2D equirect;

    void Start()
    {
        cam = GetComponent<Camera>();
        cam.fieldOfView = 90;
        cam.aspect = 1.0f;

        // Create the cubemap RenderTexture
        cubemap = new RenderTexture(captureWidth, captureHeight, 24, RenderTextureFormat.ARGB32);
        cubemap.dimension = UnityEngine.Rendering.TextureDimension.Cube;

        // Create the equirectangular texture
        equirect = new Texture2D(captureWidth, captureHeight, TextureFormat.RGB24, false);
    }

    void Update()
    {
        if (Input.GetKeyDown(KeyCode.C))
        {
            CaptureFrame();
        }
    }

    void CaptureFrame()
    {
        cam.RenderToCubemap(cubemap);
        cubemap.ConvertToEquirect(equirect);

        byte[] bytes = equirect.EncodeToPNG();
        System.IO.File.WriteAllBytes(savePath, bytes);

        Debug.Log("360 Capture saved to " + savePath);
    }

    void OnDestroy()
    {
        if (cubemap != null) cubemap.Release();
        if (equirect != null) Destroy(equirect);
    }
}

5. Configure the Camera:

   • Attach the Camera360Capture script to the 360Camera GameObject.
   • Adjust the capture width, height, and save path as needed.

6. Trigger Capture During Runtime:

   • Press the C key during runtime to capture the current view as a 360-degree image.

Explanation of the Camera Setup

• Cubemap Rendering: The RenderTexture is set to a cubemap, which captures the scene from six different directions (forward, backward, left, right, up, down). This allows for a complete 360-degree view around the camera.

• Field of View (FOV) and Aspect Ratio: The camera's FOV is set to 90 degrees, which is optimal for capturing a cubemap face. The aspect ratio is set to 1.0 to ensure square captures for each direction.

• Conversion to Equirectangular: After rendering to the cubemap, the texture is converted to an equirectangular format, which is a standard format for 360-degree images and videos.

• Saving the Image: The captured equirectangular image is saved as a PNG file to the specified path, allowing for easy access and further processing or viewing.

By following these steps, you can set up a 360-degree camera in Unity and capture panoramic images during runtime. This setup is useful for creating immersive content for VR applications, virtual tours, and more.