Add support for creating interactable UIs in 3D space

Jendx commented 10 months ago

Describe the project you are working on

Meme game inspired by FNAF

Describe the problem or limitation you are having in your project

I wanted to setup player interactable screen with few buttons on it (vis ref photo). https://media1.giphy.com/media/v1.Y2lkPTc5MGI3NjExd3I0MDVleTFxOTIwZm54dmFqZXprYmpqZzkycHluN200ZmJucjdkciZlcD12MV9pbnRlcm5hbF9naWZfYnlfaWQmY3Q9Zw/YdJAFlCWvyrKhIoFzp/giphy.gif

Describe the feature / enhancement and how it helps to overcome the problem or limitation

Working on few mine learning projects in unreal, you were able to just draw 2D items (Widgets) into 3D space and when setting their collisions right, you were able to interact with them using raycasts.

I would like to have something like this in Godot as well. I saw the example project and though. Just having this like a node would be so much simpler.

Describe how your proposal will work, with code, pseudo-code, mock-ups, and/or diagrams

When you create new node, user will have to add few others nodes, that will be inserted into exported variables of root node

I have wrote code how it would work into abstract class below:

/// <summary>
/// Base class for Nodes that can be controlled by user's raycast (Basicaly Raycast replaces mouse)
/// For correct use create Node structure as follows:
/// <code>
/// L RaycastInteractable2DUiNode3D
/// LL SubViewport
/// LLL Control (Node containing UI for viewport)
/// LLL Camera (Render viewport texture into SubViewport)
/// LL MeshInstance3D (Create material with ViewPortTexture)
/// LLL Area3D (For tracking raycast. <b>DO NOT FORGET TO SET RAYCAST TO COLIDE WITH AREAS</b>)
/// LLLL ColisionShape3D
/// </code>
/// </summary>
[GlobalClass]
public abstract partial class RaycastInteractable2DUiNode3D : Node3D
{
    private bool isMouseHeld;
    private bool isMouseInside;
    private Vector3? lastMousePosition3D;
    private Vector2 quadSize;
    private Vector2 lastMousePosition2D = Vector2.Zero;

    [Export]
    protected Area3D uiArea;

    [Export]
    protected SubViewport subViewport;

    [Export]
    protected MeshInstance3D cameraViewDisplayMesh;

    protected RaycastInteractable2DUiNode3D() {}

    /// <summary>
    /// If event is mouse event & mouse is in area, the input will be forwarded into subViewport
    /// </summary>
    /// <param name="event"></param>
    public override void _UnhandledInput(InputEvent @event)
    {
        bool isMouseEvent = @event is InputEventMouse or InputEventMouseMotion or InputEventMouseButton;

        if (isMouseEvent && (isMouseInside || isMouseHeld))
        {
            HandleMouse((InputEventMouse)@event);
        }

        if (!isMouseEvent)
        {
            subViewport.PushInput(@event);
            return;
        }
    }

    /// <summary>
    /// Get's called everytime player's raycast hits valid object (Signal receiving method)
    /// </summary>
    /// <param name="colidedObject"></param>
    public void Player_onRayCastColided(Node colidedObject)
    {
        if (colidedObject.Name == uiArea.Name)
        {
            isMouseInside = true;
        }
    }

    private void HandleMouse(InputEventMouse @event)
    {
        isMouseInside = FindMouse(@event.GlobalPosition, out Vector3 position);

        HandleMouseInPosition(@event, position);
    }

    public void HandleSynteticMouseMotion(Vector3 position)
    {
        isMouseInside = true;

        HandleMouseInPosition(new InputEventMouseMotion(), position);
    }

    public void HandleSynteticMouseClick(Vector3 position, bool pressed)
    {
        isMouseInside = true;

        HandleMouseInPosition(
            new InputEventMouseButton() { ButtonIndex = MouseButton.Left, Pressed = pressed },
            position);
    }

    /// <summary>
    /// Handles mouse input in 2D space
    /// </summary>
    /// <param name="event"></param>
    /// <param name="position"></param>
    private void HandleMouseInPosition(InputEventMouse @event, Vector3 position)
    {
        quadSize = (cameraViewDisplayMesh.Mesh as QuadMesh).Size;

        if (@event is InputEventMouseButton)
        {
            isMouseHeld = @event.IsPressed();
        }

        Vector3 mousePosition3D;

        if (isMouseInside)
        {
            mousePosition3D = uiArea.GlobalTransform.AffineInverse() * position;
            lastMousePosition3D = mousePosition3D;
        }
        else
        {
            mousePosition3D = lastMousePosition3D ?? Vector3.Zero;
        }

        var mousePosition2D = new Vector2(mousePosition3D.X, -mousePosition3D.Y);

        mousePosition2D.X += quadSize.X / 2;
        mousePosition2D.Y += quadSize.Y / 2;

        mousePosition2D.X /= quadSize.X;
        mousePosition2D.Y /= quadSize.Y;

        mousePosition2D.X *= subViewport.Size.X;
        mousePosition2D.Y *= subViewport.Size.Y;

        @event.Position = mousePosition2D;
        @event.GlobalPosition = mousePosition2D;

        if (@event is InputEventMouseMotion)
        {
            (@event as InputEventMouseMotion).Relative = mousePosition2D - lastMousePosition2D;
        }

        lastMousePosition2D = mousePosition2D;

        subViewport.PushInput(@event);
    }

    private bool FindMouse(Vector2 globalPosition, out Vector3 position)
    {
        var camera = GetViewport().GetCamera3D();

        var from = camera.ProjectRayOrigin(globalPosition);
        var dist = FindFurtherDistanceTo(camera.Transform.Origin);
        var to = from + camera.ProjectRayNormal(globalPosition) * dist;

        var parameters = new PhysicsRayQueryParameters3D()
        {
            From = from,
            To = to, 
            CollideWithAreas = true,
            CollisionMask = uiArea.CollisionLayer,
            CollideWithBodies = false 
        };

        var result = GetWorld3D().DirectSpaceState.IntersectRay(parameters);

        position = Vector3.Zero;

        var didFindMouse = result.Count > 0;
        if (didFindMouse)
        {
            position = (Vector3)result["position"];
        }

        return didFindMouse;
    }

    private float FindFurtherDistanceTo(Vector3 origin)
    {
        var edges = new Vector3[]
        {
            uiArea.ToGlobal(new Vector3(quadSize.X / 2, quadSize.Y / 2, 0)),
            uiArea.ToGlobal(new Vector3(quadSize.X / 2, -quadSize.Y / 2, 0)),
            uiArea.ToGlobal(new Vector3(-quadSize.X / 2, quadSize.Y / 2, 0)),
            uiArea.ToGlobal(new Vector3(-quadSize.X / 2, -quadSize.Y / 2, 0)),
        };

        float farDistance = 0;

        foreach (var edge in edges)
        {
            var tempDistance = origin.DistanceTo(edge);
            farDistance = Mathf.Max(farDistance, tempDistance);
        }

        return farDistance;
    }
}

If this enhancement will not be used often, can it be worked around with a few lines of script?

It could be done with 178 lines of code if you know what you are doing

Is there a reason why this should be core and not an add-on in the asset library?

I believe this feature is often used in games and would be smaller issue for beginning Godot devs to implement and time saver for more experienced devs

AThousandShips commented 10 months ago

godotengine / godot-proposals