Implement a way to render a panorama from camera position

[RadiantUwU]

Describe the project you are working on

3D VR game where the phone acts as the HMD

Describe the problem or limitation you are having in your project

Having to render on a computer and display it on the mobile device.

Describe the feature / enhancement and how it helps to overcome the problem or limitation

This would allow me to send an image to the mobile device directly to be displayed with not a huge amount of performance related issues.

Describe how your proposal will work, with code, pseudo-code, mock-ups, and/or diagrams

RenderingServer.viewport_render_panorama(viewport: RID) -> Image

If this enhancement will not be used often, can it be worked around with a few lines of script?

No, as this will require many separate renders that might not even look correct using force_draw, slowing down rendering extensively.

Is there a reason why this should be core and not an add-on in the asset library?

This would pave a better way of allowing game makers to be able to use phone devices for VR related games.

[Calinou]

There's already RenderingServer.environment_bake_panorama(), but it only renders the radiance map, not what the Camera3D or a ReflectionProbe sees.

Also, this can be implemented with a script that uses a Camera3D node and a FOV of 90 degrees and a Viewport with a 1:1 aspect ratio to render a cubemap. It'll take at least 6 frames to perform the full cubemap rendering though.

Rendering actual equirectangular (2:1) images would require https://github.com/godotengine/godot-proposals/issues/3779 to be implemented. That said, it could also be done using a cubemap rendering (as mentioned above) and distorting it using a script or shader to create an equirectangular image. See https://danilw.github.io/GLSL-howto/cubemap_to_panorama_js/cubemap_to_panorama.html.

[bertodelrio256]

Also, this can be implemented with a script that uses a Camera3D node and a FOV of 90 degrees and a Viewport with a 1:1 aspect ratio to render a cubemap. It'll take at least 6 frames to perform the full cubemap rendering though.

Iv had trouble finding info on how to do this.. would you have a separate 1:1 viewport that you spawn in the scene with 6 cameras as children and then save the texture?

[bertodelrio256]

ok so i was able to create a script to do this...

this script will get the images from viewport children (one for each view angle) and taking shots, then loads it directly to a shader global (saves with the scene!):

#if TOOLS
using Godot;
using System;
[Tool]
public partial class CubemapGenerator : Node3D
{
    [Export]
    public bool takeshot = false;
    [Export]
    public NodePath globalshadervarspath;
    public SubViewport front_vp;
    public SubViewport back_vp;
    public SubViewport right_vp;
    public SubViewport left_vp;
    public SubViewport top_vp;
    public SubViewport bottom_vp;
    private int framecount = 0;
    private ShaderGlobalsOverride shaderGlobals;
    private Godot.Collections.Array<Image> images;
    private Node3D shotpos;
    private Camera3D front;
    private Camera3D back;
    private Camera3D right;
    private Camera3D left;
    private Camera3D top;
    private Camera3D bottom;

    public override void _Ready()
    {
        shotpos = GetNode<Node3D>("position");

        front = GetNode<Camera3D>("front_vp/front");
        back = GetNode<Camera3D>("back_vp/back");
        right = GetNode<Camera3D>("right_vp/right");
        left = GetNode<Camera3D>("left_vp/left");
        top = GetNode<Camera3D>("top_vp/top");
        bottom = GetNode<Camera3D>("bottom_vp/bottom");

        front_vp = GetNode<SubViewport>("front_vp");
        back_vp = GetNode<SubViewport>("back_vp");
        right_vp = GetNode<SubViewport>("right_vp");
        left_vp = GetNode<SubViewport>("left_vp");
        top_vp = GetNode<SubViewport>("top_vp");
        bottom_vp = GetNode<SubViewport>("bottom_vp");

        shaderGlobals = GetNode<ShaderGlobalsOverride>(globalshadervarspath);

        images = new Godot.Collections.Array<Image>{Image.Create(128,128, true, Image.Format.Rgba8),
        Image.Create(128,128, true, Image.Format.Rgba8),
        Image.Create(128,128, true, Image.Format.Rgba8),
        Image.Create(128,128, true, Image.Format.Rgba8),
        Image.Create(128,128, true, Image.Format.Rgba8),
        Image.Create(128,128, true, Image.Format.Rgba8)};
    }

    public void Capture()
    {
        images[0] = right_vp.GetTexture().GetImage();
        images[1] = left_vp.GetTexture().GetImage();
        images[2] = top_vp.GetTexture().GetImage();
        images[3] = bottom_vp.GetTexture().GetImage();
        images[4] = front_vp.GetTexture().GetImage();
        images[5] = back_vp.GetTexture().GetImage();

        Cubemap cb = new Cubemap();
        cb.CreateFromImages(images);
        shaderGlobals.Set("params/scene_cubemap", cb);
    }

    public override void _Process(double delta)
    {
        front.Position = GlobalPosition;
        back.Position = GlobalPosition;
        right.Position = GlobalPosition;
        left.Position = GlobalPosition;
        top.Position = GlobalPosition;
        bottom.Position = GlobalPosition;

        if(takeshot)
        {
            takeshot = false;
            Capture();
        }
    }
}
#endif

I originally tried to do it with one cam/viewport but the viewport would not update after rotating the cam.