`WantSetMousePos` warping leaks mouse events from previous position into next frame

Paril commented 1 week ago

Version/Branch of Dear ImGui:

1.90.6 docking

Back-ends:

imgui_impl_sdl2 + imgui_impl_opengl3

Compiler, OS:

Win11 MSVC 2022

Full config/build information:

Dear ImGui 1.90.6 (19060)
--------------------------------
sizeof(size_t): 8, sizeof(ImDrawIdx): 2, sizeof(ImDrawVert): 20
define: __cplusplus=199711
define: _WIN32
define: _WIN64
define: _MSC_VER=1929
define: _MSVC_LANG=201402
define: IMGUI_HAS_VIEWPORT
define: IMGUI_HAS_DOCK
--------------------------------
io.BackendPlatformName: imgui_impl_sdl2
io.BackendRendererName: imgui_impl_opengl3
io.ConfigFlags: 0x00000443
 NavEnableKeyboard
 NavEnableGamepad
 DockingEnable
 ViewportsEnable
io.ConfigViewportsNoDecoration
io.ConfigInputTextCursorBlink
io.ConfigWindowsResizeFromEdges
io.ConfigMemoryCompactTimer = 60.0
io.BackendFlags: 0x00001C0E
 HasMouseCursors
 HasSetMousePos
 PlatformHasViewports
 HasMouseHoveredViewport
 RendererHasVtxOffset
 RendererHasViewports
--------------------------------
io.Fonts: 1 fonts, Flags: 0x00000000, TexSize: 512,64
io.DisplaySize: 900.00,800.00
io.DisplayFramebufferScale: 1.00,1.00
--------------------------------
style.WindowPadding: 8.00,8.00
style.WindowBorderSize: 1.00
style.FramePadding: 4.00,3.00
style.FrameRounding: 0.00
style.FrameBorderSize: 0.00
style.ItemSpacing: 8.00,4.00
style.ItemInnerSpacing: 4.00,4.00

Details:

When WantSetMousePos is used, regardless of what the values of MousePos/MousePosPrev are set to, it seems that if multiple movement events are queued by the OS it will leak some of those events into the next frame causing deltas to be reported incorrectly.

It's difficult to narrow down exactly what's going on, but I have a bit of logging going on in my app, here's what I've accumulated. During a "good" warp (the expected behavior), the log looks like this:

move from 106,1 to 106,0 (delta 0,1) (io 106,0)
move from 106,0 to 106,-1 (delta 0,1) (io 106,-1)
wrap from 106,-1 to 106,664 (delta 0,1) (new io 106,664)
move from 106,664 to 106,663 (delta 0,1) (io 106,663)
move from 106,663 to 106,662 (delta 0,1) (io 106,662)

this is by moving upwards, when the -1 border is hit it wraps around to the other side. Looks good so far. However, in a lot of cases, there seems to be a 'ghost' mouse position that came from the state before the warp:

move from 117,6 to 116,0 (delta 1,6) (io 116,0)
move from 116,0 to 115,-6 (delta 1,6) (io 115,-6)
wrap from 115,-6 to 115,659 (delta 1,6) (new io 115,659)
move from 115,659 to 112,-23 (delta 3,682) (io 112,-23)
wrap from 112,-23 to 112,642 (delta 3,682) (new io 112,642)
move from 112,642 to 110,-41 (delta 2,683) (io 110,-41)
wrap from 110,-41 to 110,624 (delta 2,683) (new io 110,624)
move from 110,624 to 109,617 (delta 1,7) (io 109,617)
move from 109,617 to 108,610 (delta 1,7) (io 108,610)

Notice how the first wrap is correct, moving the mouse to the bottom, but the next mouse position that is given to us from io is based on the old position; then it seems to correct itself later, but this kind of causes a feedback loop where it can get trapped wrapping back & forth a few times.

It tends to happen more if you are more forceful/quick with mouse movements; if you move 1 pixel at a time, it never happens. I managed to get it to happen in an isolated environment (code in the sample box below). It is easiest to reproduce by moving somewhat quickly near the borders, going back & forth. The sample will 'pause' for 4 seconds when it gets a delta that is too large to make any sense (larger than the size of the bounding rectangle).

Screenshots/Video:

No response

Minimal, Complete and Verifiable Example code:

    ImGui::SetNextWindowSize({ 640, 640 });
    ImGui::Begin("Test");
    ImGui::InvisibleButton("Test", ImGui::GetContentRegionAvail());

    auto min = ImGui::GetItemRectMin();
    auto max = ImGui::GetItemRectMax();

    ImGui::GetForegroundDrawList()->AddRect(min, max, 0xFFFFFFFF);

    // stored state
    static bool down = false;
    static ImVec2 downPos = {};

    auto &io = ImGui::GetIO();

    bool isWithinViewport = ImGui::IsItemHovered() &&
                            io.MousePos.x >= min.x && io.MousePos.y >= min.y &&
                            io.MousePos.x < max.x && io.MousePos.y < max.y;

    static ImVec2 badDelta;
    static float badDeltaTime = 0;

    if (down)
    {
        if (!ImGui::IsMouseDown(ImGuiMouseButton_Left))
        {
            down = false;
            badDeltaTime = 0;
        }
        else
        {
            ImVec2 delta { io.MousePos.x - downPos.x, io.MousePos.y - downPos.y };

            // will hit when the delta exceeds threshold
            if (fabsf(delta.x) >= 640 || fabsf(delta.y) >= 640)
            {
                badDeltaTime = 4.0f;
                badDelta = delta;
                //__debugbreak();
            }

            ImGui::SetCursorScreenPos(ImVec2 { (float) (min.x + (max.x - min.x) / 4.0f), (float) (min.y + (max.y - min.y) / 2.0f) });

            if (badDeltaTime > 0)
            {
                badDeltaTime -= io.DeltaTime;
                ImGui::GetForegroundDrawList()->AddRect(min, max, 0xFF0000FF, 0.f, 0, 8.0f);
                ImGui::Text("%4g %4g", badDelta.x, badDelta.y);
            }
            else
            {
                ImGui::Text("%4g %4g", delta.x, delta.y);
            }

            // wrap
            downPos = io.MousePos;
            while (downPos.x < min.x)
                downPos.x += (max.x - min.x);
            while (downPos.x > max.x)
                downPos.x -= (max.x - min.x);
            while (downPos.y < min.y)
                downPos.y += (max.y - min.y);
            while (downPos.y > max.y)
                downPos.y -= (max.y - min.y);

            if (io.MousePos.x != downPos.x ||
                io.MousePos.y != downPos.y)
            {
                io.MousePos = io.MousePosPrev = downPos;
                io.MouseDelta = {};
                io.WantSetMousePos = true;
            }
        }
    }
    else if (io.WantCaptureMouse && isWithinViewport)
    {
        if (ImGui::IsMouseClicked(ImGuiMouseButton_Left))
        {
            down = true;
            downPos = io.MousePos;
        }
    }

    ImGui::End();

Paril commented 1 week ago

Also, worth noting that io.MouseDelta is also incorrect during these warps and will match the calculated delta above, so I don't know if I have any alternatives...

EDIT: the issue also happens with the GLFW backend, so it's not SDL-specific.

ocornut commented 1 week ago

Linking to #228 for completeness as it suggest using similar code. Will investigate this issue when I can. Thank you!

Paril commented 1 week ago

No problem! I managed to work around it for now using SDL's relative mouse mode directly, but hopefully this is a simple issue; UI controls that wrap around are pretty common and handy.

ocornut / imgui