Incorrect touch inputs with SDL2 in Windows

Version/Branch of Dear ImGui:

Version: 1.72 Branch: master

Back-end/Renderer/Compiler/OS:

Back-ends: imgui_impl_sdl.cpp (+ imgui_impl_opengl3.cpp) Compiler: mingw64-g++ (cross compiled from Linux) Operating System: Windows 10

My Issue/Question:

On some systems, Dear ImGui will not respond properly to touch screen events (through the touch emulated mouse, which has the special mouse ID SDL_TOUCH_MOUSEID) with the SDL2 backend. If you first touch the window, release, then touch the screen somewhere else, the window will jump to the new location, as if the touch had started at the end of the previous touch, then teleported.

After some investigation, I discovered that the culprit seems to be SDL_GetGlobalMouseState combined with some Windows shenanigans. From the SDL2 wiki:

Note: SDL_GetMouseState() returns the mouse position as SDL understands it from the last pump of the event queue. This function, however, queries the OS for the current mouse position, and as such, might be a slightly less efficient function. Unless you know what you're doing and have a good reason to use this function, you probably want SDL_GetMouseState() instead.

On the systems that are affected (and I can't tell what causes it - I've used two W10 computers with the same screen, and it only happened on one of them), Windows does not properly update the variables that SDL gets the global mouse state from, until the touch is released or moved sufficiently. This means that on the frame where you get the SDL_MOUSEBUTTONDOWN event (and SDL_GetMouseState reports the button clicked), ImGui reads the mouse position from the previous touch, still present in SDL_GetGlobalMouseState. If this old position is on an ImGui window, it thinks the window was clicked (and then dragged very quickly when the global mouse state updates). (The opposite is also true - if the previous touch was outside the window, the next touch will not register as a click on the ImGui window.)

I assume this has to do with special gesture commands, like holding down to right-click, but I can't tell why it works on some computers and not on others. In any case, the SDL2 event queue works fine, so SDL_GetMouseState works as it should. As such, the issue can be resolved by editing imgui_impl_sdl.cpp to remove the code that uses SDL_GetGlobalMouseState instead of SDL_GetMouseState:

//#if SDL_HAS_CAPTURE_AND_GLOBAL_MOUSE && !defined(__EMSCRIPTEN__) && //!defined(__ANDROID__) && !(defined(__APPLE__) && TARGET_OS_IOS)
//    SDL_Window* focused_window = SDL_GetKeyboardFocus();
//    if (g_Window == focused_window)
//    {
//        // SDL_GetMouseState() gives mouse position seemingly based on the last window entered/focused(?)
//        // The creation of a new windows at runtime and SDL_CaptureMouse both seems to severely mess up with that, so we retrieve that position globally.
//        int wx, wy;
//        SDL_GetWindowPosition(focused_window, &wx, &wy);
//        SDL_GetGlobalMouseState(&mx, &my);
//        mx -= wx;
//        my -= wy;
//        io.MousePos = ImVec2((float)mx, (float)my);
//    }
    // SDL_CaptureMouse() let the OS know e.g. that our imgui drag outside the SDL window boundaries shouldn't e.g. trigger the OS window resize cursor.
    // The function is only supported from SDL 2.0.4 (released Jan 2016)
//    bool any_mouse_button_down = ImGui::IsAnyMouseDown();
//    SDL_CaptureMouse(any_mouse_button_down ? SDL_TRUE : SDL_FALSE);
//#else
    if (SDL_GetWindowFlags(g_Window) & SDL_WINDOW_INPUT_FOCUS)
        io.MousePos = ImVec2((float)mx, (float)my);
//#endif

However, going by the comment in the code, this could have unwanted consequences, so it's not a very good bug fix.

Standalone, minimal, complete and verifiable example: Unfortunately I can't really provide one, as I don't currently have access to the hardware, and as I said, it only happens on some computers, seemingly. What I did to check it was basically

ImGuiIO& io = ImGui::GetIO();
ImGui::Begin("MouseTest");
    ImGui::Text("ImGui mouse: %f %f",io.MousePos.x, io.MousePos.y);
    int x,y;
    SDL_GetMouseState(&x,&y);
    ImGui::Text("SDL mouse: %d %d", x, y);
    SDL_GetGlobalMouseState(&x,&y);
    ImGui::Text("SDL global mouse: %d %d", x, y);
ImGui::End();

The normal SDL mouse position updated as soon as the touch happened, but the other two only updated after some wriggling of the finger, or at touch release.

I don't really think this is anything worth fixing, per se, (especially if the library will eventually support touch directly through an event driven system as mentioned here #2334) but it's good to know for those who want to use touch screens and are experiencing this issue.

Happy fifth anniversery!

ocornut / imgui

Incorrect touch inputs with SDL2 in Windows #2729