ImGuizmo in Vulkan

[ib00]

Has anybody successfully integrated ImGuizmo in Vulkan?

[hypernewbie]

yeah we got a Vulkan engine that has ImGuizmo. It just uses regular imgui draw commands which we've already tapped into our low level enginer draw code.

[matthiascy]

Hi, @hypernewbie, I've just started learning Vulkan, could you give me more details about the integration? Thanks.

[hypernewbie]

@matthiascy ImGuizmo uses pure imgui drawlist, so if you imgui running in Vulkan means ImGuizmo running in Vulkan with literally no additional effort.

One can simply use the Vulkan imgui backend as-is and simply link it to their engine, which is imgui's strongly recommended way of integration. We chose to do things the hard way and plug ImGui drawlist directly in custom engine layer, which eventually plugs into Vulkan commands.

[helynranta]

Hi guys! Did you ever get your integration working properly? I'm not sure if this is Vulkan, sdl, imgui or glm related, but I'm facing this very weird issue where it seems that for ImGuizmo X and Z axis are swapped for all Gizmos. It seems I got default imgui backend (with descriptors fixes from one of the issues) running with sdl2, should not be issue? I think there is something wrong with how I construct my matrices with glm. I got default glm::perspective for P and glm::lookAt for V in my MVP matrix. I have tried with GLM_FORCE_LEFT_HANDED also but it doesn't change anything. Can you give any educated guess of what could be happening here.

I've been stuck with this for a while now, been trying modifying model matrix with different transformation matrices, but it seems when I try that my program starts bugging differently depending on which direction camera is looking at, so I think there is something wrong with either projection or view matrix.

[CedricGuillemet]

@lerppana It looks like an LH/RH issue. From what I understand, it should work whatever the hand is. Maybe I'm wrong. To narrow down the issue, can you, in the example or in your code do 2 tests:

• use the view matrix from the sample (dump values and use them) and use it with the projection value you compute with glm
• use the projection matrix from the sample and use the view matrix you compute with glm I want to know which one is causing the issue

[helynranta]

@CedricGuillemet thank you for suggestions! I believe there was some talk here that this library is using row convention, and I believe glm uses column. So I got the Perspective and LookAt functions from example codes and can confirm that by mixing those with glm gizmo is not visible at all. Then I did the obvious and decided to just reconstruct all the matrices by using your functions and not mix with glm and this is the result: Numbers there are actually correct, placing and rotation of gizmo are not. Here is the relevant part, I'm pretty sure I'm doing some obvious stupid mistake here. Looks like gizmo is pointing to my object and not camera.

static float view_matrix[16];
static float projection_matrix[16];
static float model_matrix[16];

auto aspect_ratio = ((float)swap_chain->extent.width) / ((float)swap_chain->extent.height);
Perspective(
    projection.perspective.field_of_view,
    aspect_ratio,
    projection.near_clipping_plane,
    projection.far_clipping_plane,
    projection_matrix);

LookAt(
    glm::value_ptr(camera_transform.get_position()),
    glm::value_ptr(projection.looks_at),
    glm::value_ptr(core::up),
    view_matrix);

ImGuizmo::SetOrthographic(projection.type == component::projection::type::orthographic);
ImGuizmo::SetDrawlist();

float windowWidth = (float)ImGui::GetWindowWidth();
float windowHeight = (float)ImGui::GetWindowHeight();
ImGuizmo::SetRect(ImGui::GetWindowPos().x, ImGui::GetWindowPos().y, windowWidth, windowHeight);

auto& pos = transform.get_position();
auto& s = transform.get_scale();

static float translation[3], rotation[3], scale[3];
translation[0] = pos.x;
translation[1] = pos.y;
translation[2] = pos.z;

// skipping rotation for now cos I use quaternions in my own code

scale[0] = s.x;
scale[1] = s.y;
scale[2] = s.z;

ImGuizmo::RecomposeMatrixFromComponents(translation, rotation, scale, model_matrix);

ImGuizmo::Manipulate(
    view_matrix,
    projection_matrix,
    mCurrentGizmoOperation,
    mCurrentGizmoMode,
    model_matrix,
    NULL,
    useSnap ? &snap.x : NULL);

if (ImGuizmo::IsUsing())
{
    ImGuizmo::DecomposeMatrixToComponents(
        model_matrix,
        translation,
        rotation,
        scale);

    transform.set_position(translation[0], translation[1], translation[2]);
}

Related to this, could it be possible to also have these functions exposed in ImGuizmo.h. I believe they are also defined in source files?

[helynranta]

I have found a solution to my issues. It was RH/LH issue after all. I can mix GLM matrices with ImGuizmo but it seems that I need to render my frame in RH and then pass LH version of perspective and lookAt matrix to ImGuizmo. Position of Gizmo matches if I do it other way around, but then when I attempt to pick rotation gizmo circles it does not work correctly. I don't mind because I was rendering my scene with LH setup anyway, but might cause issues if someone is using RH setup for rendering. Or then this is some weird quirk in my own rendering code, cannot be sure :D

If someone finds this issue, here is code snippets of how you can choose specific handed setups with GLM:

auto view = glm::lookAtRH(eye, at, up);

auto aspect_ratio = windowWidth / windowHeight;
auto perspective = glm::perspectiveRH(
    glm::radians(projection.perspective.field_of_view),
    aspect_ratio,
    near_clipping_plane,
    far_clipping_plane);

ImGuizmo::Manipulate(
    glm::value_ptr(view),
    glm::value_ptr(perspective),
    mCurrentGizmoOperation,
    mCurrentGizmoMode,
    model_matrix,
    NULL,
    useSnap ? &snap.x : NULL);

[Makogan]

@lerppana, could I ask you how you integrated ImgUIzmo? I am also trying to integrate with vulkan, but unlike you, I cannot even get the guizmos to respond to user input at all. No highlighting, no numbers, nothing. It would be extremely helpful for me to see a snippet to compare with what I have.

[helynranta]

Sorry for a late response. Did you get your implementation working? I would quess there is a couple ways you can start looking at this.

• what is your backend? Does all other imgui functionality work correctly (do buttons respond to mouse)? I use official sdl2 backend + Vulkan backend if that helps.

• how did you do the drawing? I do one phase in which I draw the game image. I then go and draw imgui stuff separately and in my game window (in editor) I first do draw my game image and then draw the widget. Could be that you have forgotten the ImGuizmo::SetRect? I think that tells ImGuizmo where the image should be. My drawing of guizmo is couple of comments up 👆🏻.

My game engine project is quite old and I do it at my free time so it is kind of messy atm. If you need more help I can try to extract important parts of rendering to image. But there is propably good tutorials out there that do a better job at it 😅

[v3c70r]

I happened to encountered the same issue when I try to use ImGuizmo with Vulkan. It turns out the ImGuizmo uses OpenGL NDC coordinate where Y-axis is from -1 to 1. While Vulkan's NDC's Y-axis is from 1 to -1. (Indeed it's a handedness issue) A quick fix is to flip the projection matrix's [1][1] element before pass it to ImGuizmo functions.

projMat[1][1] *= -1.0f

Ref: https://computergraphics.stackexchange.com/questions/12448/vulkan-perspective-matrix-vs-opengl-perspective-matrix