Modifier keys with python backend

[Aman-Anas]

I was using the opengl backend with a game engine context, and was able to get almost everything running correctly. However, there are some parts of the backend examples (glfw, sdl, etc) that need to be updated with the new API, such as modifier keys.

For example, I was not able to get this to work

io.key_ctrl = ctrl_pressed
io.key_alt = alt_pressed
io.key_shift = shift_pressed

but this worked without errors.

io.add_key_event(imgui.Key.im_gui_mod_ctrl, ctrl_pressed)
io.add_key_event(imgui.Key.im_gui_mod_alt, alt_pressed)
io.add_key_event(imgui.Key.im_gui_mod_shift, shift_pressed)

Also, mouse button and position events should use the new io.add_mouse_button_event(), io.add_mouse_wheel_event() and io.add_mouse_pos_event() but they seem to work the old way as well.

Not that big of a deal, but the modifier keys didn't work until I updated it and it's only really documented in the code, so I thought it was worth mentioning that it needs to be updated 😀

[pthom]

Hello,

First, thank you very much for posting this! Following your instructions, I was unable to get it to work on my side.

For example, if I try to work on the python SDL backend (bindings/imgui_bundle/python_backends/sdl_backend.py), I could modify it like this:

1. Add this to the key map:

       key_map[SDL_SCANCODE_LCTRL] = imgui.Key.left_ctrl
       key_map[SDL_SCANCODE_RCTRL] = imgui.Key.right_ctrl
       key_map[SDL_SCANCODE_LSHIFT] = imgui.Key.left_shift
       key_map[SDL_SCANCODE_RSHIFT] = imgui.Key.right_shift
       key_map[SDL_SCANCODE_LALT] = imgui.Key.left_alt
       key_map[SDL_SCANCODE_RALT] = imgui.Key.right_alt
       key_map[SDL_SCANCODE_LGUI] = imgui.Key.left_super
       key_map[SDL_SCANCODE_RGUI] = imgui.Key.right_super

Comment this out:

            # io.key_shift = (SDL_GetModState() & KMOD_SHIFT) != 0
            # io.key_ctrl = (SDL_GetModState() & KMOD_CTRL) != 0
            # io.key_alt = (SDL_GetModState() & KMOD_ALT) != 0
            # io.key_super = (SDL_GetModState() & KMOD_GUI) != 0

And then try to display the modifiers key: add this inside bindings/imgui_bundle/python_backends/examples/example_python_backend_sdl2.py

                io = imgui.get_io()
                imgui.text(f"""
                Keyboard modifiers: 
                    {io.key_ctrl=}
                    {io.key_alt=} 
                    {io.key_shift=} 
                    {io.key_super=}""")

However, in this case modifiers keys are not displayed on my side.

Would you like to be credited in this repo by adding a PR? I would very much welcome it.

Btw, if you have an alternative additional backend which may be of use for others, feel free to post it also ;-)

[Aman-Anas]

I checked whether modifier keys are displayed via the "inputs" section in the show_demo_window panel, near the bottom.

Testing with the glfw example in bindings/imgui_bundle/python_backends/examples by default it doesn't show the modifier keys registered. (Also, I'm getting an error if scrolling in that windows sometimes about AttributeError: 'imgui_bundle._imgui_bundle.imgui.IO' object has no attribute 'mouse_wheel_horizontal', and it crashes it. Would be fixable with add_mouse_wheel_event() I think)

After adding this to the keyboard callbacks

        if (action != glfw.PRESS) and (action != glfw.RELEASE):
            return

        if (glfw_key == glfw.KEY_LEFT_CONTROL) or (
                glfw_key == glfw.KEY_RIGHT_CONTROL):
            io.add_key_event(imgui.Key.im_gui_mod_ctrl, action == glfw.PRESS)

        if (glfw_key == glfw.KEY_LEFT_ALT) or (
                glfw_key == glfw.KEY_RIGHT_ALT):
            io.add_key_event(imgui.Key.im_gui_mod_alt, action == glfw.PRESS)

        if (glfw_key == glfw.KEY_LEFT_SHIFT) or (
                glfw_key == glfw.KEY_RIGHT_SHIFT):
            io.add_key_event(imgui.Key.im_gui_mod_shift, action == glfw.PRESS)

It then shows up. Also need to add the super key, lol

I can do a PR for this, sure. I'm not sure if they should be added into the keymap or handled separately, since maybe you want the actual key value "shift etc" rather than the modifier version in the keymap... I should probably check some C++ Dear Imgui examples I guess.

[pthom]

OK, thanks for providing more info, I will work on it, as I would prefer it to be in a keymap.

[Aman-Anas]

Maybe it would make sense to have a second, modifier keymap and check that as well in the keyboard callback.

[Aman-Anas]

It does seem like the cpp Dear Imgui glfw backend simply has some addKeyEvent() calls for modifiers, no need to keep it the exact same though, if a keymap is cleaner.

static void ImGui_ImplGlfw_UpdateKeyModifiers(GLFWwindow* window)
{
    ImGuiIO& io = ImGui::GetIO();
    io.AddKeyEvent(ImGuiMod_Ctrl,  (glfwGetKey(window, GLFW_KEY_LEFT_CONTROL) == GLFW_PRESS) || (glfwGetKey(window, GLFW_KEY_RIGHT_CONTROL) == GLFW_PRESS));
    io.AddKeyEvent(ImGuiMod_Shift, (glfwGetKey(window, GLFW_KEY_LEFT_SHIFT)   == GLFW_PRESS) || (glfwGetKey(window, GLFW_KEY_RIGHT_SHIFT)   == GLFW_PRESS));
    io.AddKeyEvent(ImGuiMod_Alt,   (glfwGetKey(window, GLFW_KEY_LEFT_ALT)     == GLFW_PRESS) || (glfwGetKey(window, GLFW_KEY_RIGHT_ALT)     == GLFW_PRESS));
    io.AddKeyEvent(ImGuiMod_Super, (glfwGetKey(window, GLFW_KEY_LEFT_SUPER)   == GLFW_PRESS) || (glfwGetKey(window, GLFW_KEY_RIGHT_SUPER)   == GLFW_PRESS));
}

[pthom]

Ok, done in ff1689df502e75750f164d285f47f80a56d85084

New mouse API implemented in d6da7391b6a3e4157b7060385a9e7b429c9a9c8f