Keyboard / Controller Support?

[velvitonator]

(ADMIN EDIT July 2016: see https://github.com/ocornut/imgui/issues/323#issuecomment-233785300 for the current todo list / ongoing tasks)

I've been looking to make our ImGUI accept controller input, for use on consoles. Something like hitting d-pad down to select the next item down, etc.

From what I could tell, however, the only way to do such a thing would be to spoof mouse control. Is this the case? If so, is there a way to ask for the rectangle of a given item outside of the update loop?

I also couldn't find any way to programmatically set the focus of an item other than input text fields; did I miss something?

[ocornut]

Before anything else, have you looked into using a system like Synergy + usynergy.c to have your game console use your PC mouse/keyboard ? I'd wholly recommend that.

If so, is there a way to ask for the rectangle of a given item outside of the update loop?

Not sure what you mean by "the update loop" nor why you would need that if you are spoofing mouse control? If you are going to spoof mouse control just make your controller adjust the MousePos and MouseDown (buttons) values.

You can call GetItemRectMin() / GetItemRectMax() to obtain the bounding rectangle of the last item (which might be clipped and outside of view, testable with IsItemVisible()) but I don't see how that would help here.

I also couldn't find any way to programmatically set the focus of an item other than input text fields; did I miss something?

No it's not available yet, there's no generalized concept of focus in fact.

Now if you really want controller support. There's different way we can approach this:

• Emulating mouse with game controller (or keyboard) would be easy to implement and functional immediately. Obviously it'd be rather awkward and underwhelming interaction-wise, but for casual use it may be enough.
• If you only want this for simple sort of UI like list of items you can emulate the concept of "whats focused" on user side and handle interaction yourself (e.g. Button X pressed while Item N is selected).
• The correct thing would be to fully handle keyboard/controller navigation and per-widget focus. It's not a simple feature, probably a bunch of work to get it right (I'd say 1 week optimistically). It is a very desirable feature but I don't know when / if I'll ever be able to do it.

I have rename your topic to keyboard/controller because the problems are the same for keyboard controls and many people are likely to come from the angle of desiring keyboard control.

[velvitonator]

Before anything else, have you looked into using a system like Synergy + usynergy.c to have your game console use your PC mouse/keyboard ? I'd wholly recommend that.

Yeah, we've got this up and running (except for keyboard support, since I've yet to hook up the manual character-input stuff). We want to add controller as a good native option, so users don't have to switch input devices.

Not sure what you mean by "the update loop" nor why you would need that if you are spoofing mouse control? If you are going to spoof mouse control just make your controller adjust the MousePos and MouseDown (buttons) values.

Apologies; I meant that we could do something like move the mouse down "an item" instead, but that starts to get a bit complicated for client code, especially since we'd need this to be something for everyone to use, to make their own debug GUIs. With that approach, we'd need the screen-rect of the options etc. This is all certainly possible for us to do, it's just a matter of the effort we want to expend on it. Hence, I'm hoping to clarify the available options before embarking on something time-consuming. =)

Emulating mouse with game controller (or keyboard) would be easy to implement and functional immediately. Obviously it'd be rather awkward and underwhelming interaction-wise, but for casual use it may be enough.

Yeah, this is looking to be the only option in the timeframe we're looking at. Since we also have Synergy, I'm not convinced adding this is worth it; I know I'd easily switch to mouse + keys as soon as I had to, rather than bear analog-stick mouse cursor!

If you only want this for simple sort of UI like list of items you can emulate the concept of "whats focused" on user side and handle interaction yourself (e.g. Button X pressed while Item N is selected).

Certainly! But getting it to reflect in the UI is the tricky bit =/ Also, we already have a couple of in-game debug UIs using ImGUI, and they have quite a bit of variety. =P

The correct thing would be to fully handle keyboard/controller navigation and per-widget focus. It's not a simple feature, probably a bunch of work to get it right (I'd say 1 week optimistically). It is a very desirable feature but I don't know when / if I'll ever be able to do it.

Hmm, I'll be honest: I'm so accustomed to working with closed-source things that simply adding the feature to ImGUI didn't occur to me. I think it's a little out of scope for the timeframe I have, but it's certainly worth considering in the longer term, should we find that controller support is indispensable for whatever reason.

I have rename your topic to keyboard/controller because the problems are the same for keyboard controls and many people are likely to come from the angle of desiring keyboard control.

Good idea!

[ocornut]

Apologies; I meant that we could do something like move the mouse down "an item" instead, but that starts to get a bit complicated for client code, especially since we'd need this to be something for everyone to use,

It's trivial to store the rectangle of the hovered item (we can add that to the library), so you could move the cursor outside of it but you wouldn't know by how much as you don't know where the next item. An hybrid approach to move outward the item by the amount of its width/height and then move smoothly if there's no item under the cursor. It would be easy to implement and may be a little better than a cursor but probably still mediocre. If you are on PS4, I haven't tried it but I suspect the touch pad is precise enough to emulate a mouse.

Hmm, I'll be honest: I'm so accustomed to working with closed-source things that simply adding the feature to ImGUI didn't occur to me. I think it's a little out of scope for the timeframe I have, but it's certainly worth considering in the longer term, should we find that controller support is indispensable for whatever reason.

FYI if your project/company has the fund I'm more than happy to provide custom work on imgui to develop that sort of non-trivial feature (been looking for possible ways to fund its development as I can't really afford it any more myself, at least not at the pace I was working on it a few months ago). Something to consider, may be more optimal than you doing it and it'll benefit the sustainability of the library.

It's one of the those feature that's touching quite a lot of elements in the code - quite a complex feature to do right. If you to want to tackle it yourself I can also guide you.

[velvitonator]

Sorry for the silence; I forgot the re-mark the mail as unread to get back to it.

Your idea to use the PS4 touchpad was pretty effective--it's somewhat finicky, but much better than a stick, and it wasn't too difficult to hook up the sticks and pad for scrolling, either.

So, we're good for now; thanks for the suggestion!

[ocornut]

I don't have an ETA yet but I've been working on this lately. When it starts to be semi usable I will push it to a public branch, right now it is way too prototypey.

[ocornut]

Some progress report, been improving support for the popup stack, combos, menus, sliders, visualizing selection, avoiding confusion between keyboard and mouse, etc. There's still a billion things to do, it is a little overwhelming.

[Pagghiu]

This is VERY interesting :grin:

[MrMarkie]

This looks great. We'd like to use it for interaction with debug UI in PSVR. It's exactly the kind of thing we need for this. When are you looking to make it available?

[ocornut]

When it is done @MrMarkie :) I've been working on my non-existent spare time until now, but because this feature is now sponsored (a first) I'm aiming before end of July. Will probably start testing and iterating on API and edge cases mid-july.

[MrMarkie]

Hi. Is there any news on this feature? I'd be happy to help test it if you have a preview version

[ocornut]

I was thinking about pushing a branch but the IO api is utterly temporary/broken yet and it's still missing important features. I'll work on it this week-end. From Monday if you're happy with experimenting with it (at the cost of minor breakage in the following weeks) it would be indeed quite helpful if you want to try it and submit feedback. You can also e-mail me if you want to move this off-line for back and forth.

[MrMarkie]

Absolutely. Let me know when / where I can get it. Look forwards to trying it out. Markie On 15 Jul 2016 17:35, omar notifications@github.com wrote:I was thinking about pushing a branch but the IO api is utterly temporary/broken yet and it's still missing important features. I'll work on it this week-end. From Monday if you're happy with experimenting with it (at the cost of minor breakage in the following weeks) it would be indeed quite helpful if you want to try it and submit feedback. You can also e-mail me if you want to move this off-line for back and forth.

—You are receiving this because you were mentioned.Reply to this email directly, view it on GitHub, or mute the thread.

[ocornut]

Quick update GIF

I'm not finished but I'll be pushing first test version to a branch tonight, along with instructions for those interested.

[MrMarkie]

This looks excellent!! On 19 Jul 2016 22:28, omar notifications@github.com wrote:Quick update GIF

I'm not finished but I'll be pushing first test version to a branch tonight, along with instructions for those interested.

—You are receiving this because you were mentioned.Reply to this email directly, view it on GitHub, or mute the thread.

[ocornut]

Hello,

I have started to push gamepad/keyboard controls this into a work-in-progress branch: https://github.com/ocornut/imgui/tree/navigation

Brain dump, some of it may be hard to digest :)

It's been a big chunk of work and it isn't finished (probably 50+ changes left to apply, bug fixes and tweaks), BUT i think it is ready for early evaluation. If you are willing to help and checkout this branch it would be super helpful.

If you don't have time to look it and help, come back in 2-4 weeks and it'll be in a better state.

I will keep working on this mostly in the upcoming week-ends. Even if incomplete it could be merged in master in a few weeks if everything works out nicely, and then I can keep improving/fixing afterwards. Any test with testing would obviously accelerate merging into master.

The development of this feature has been sponsored by Insomniac Games (thank you!).

• 1) If you don't setup the new inputs, in theory you shouldn't notice any visible difference. That's the theory and I have most probably added bugs and undesirable side-effects. So even just checking out the branch and using it without navigation would already be helpful so you can report bugs.
• 2) THE INITIAL FOCUS IS ON USING A GAME CONTROLLER! It also works with keyboard but essential features like alt/ctrl style shortcuts (#456) are not done at all. Also the user expectation with keyboard is much higher/harsher than with gamepad (with keyboard we more naturally expect a standard os-like experience which dear imgui doesn't provide). Thirdly, depending on your application using keyboard probably requires more fine-tuned controls of how inputs are dispatched and shared between your app/game and different parts of your imgui interfaces. We may need better options/tooling for that.
• 3) It is very probably missing or failing at something that will appear very big and obvious to you. I've nailed many issues and big technical problems but barely started scrapping the surface of all possible usage scenarios. So I have the basic blocks and will rearrange them and add lots of missing glue. If you want to give it a try you will probably stumble on a few issues and maybe a big gap somewhere. I am looking forward to hear about them and improve the situation gradually.
• 4) The actual scoring function for navigating the graph hasn't been given a lot of love yet and is failing to behave as expected in various cases of intricate, tighly packed or loosely spread layouts. But it works for basic layouts. Failure usually mean that you press a direction and don't end up exactly where you intended. This will be worked out in the following weeks, and please don't hesitate to send screenshots of cases that aren't working too well.
• 5) I have tried to split this work into commits so it might be easier to bisect bugs. 5a. A few various commits already pushed to master during the last month. Some commits not directly in navigation code but required for navigation made sense to push earlier to increase empirical testing, so I did that (sorry). 5b. A dozen of small commits in the Navigation branch. 5c. One big commit for most of it that didn't make sense to break apart. +820 lines which in dense ImGui terms is actually quite a lot (to get a rough idea I have already spend more than 50 hours working on this and I'm not done).
• 6) The good news is that the addition to the public API are rather minimal. ImGui only grows by 22 lines. 3 functions for now, IsItemFocused(), IsAnyItemFocused(), SetItemDefaultFocus(), 1 window flag ImGuiWindowFlags_NoNav, a bunch of keys (read instructions below), 1 IO setting NavMovesMouse, 3 IO outputs WantMoveMouse NavUsable NavActive.
• 7) ALL THE PUBLIC FACING API, MEMBERS, NAMES, ARE WORK-IN-PROGRESS and deemed to change before we merge into master. Please be as critical and inquisitive as possible. In particular, I am NOT expecting to keep the current way of passing inputs (currently using the KeyMap[] KeyDown[] array). It works but it is rather misleading. I just used what was easily available because I want to nail most features before deciding on public facing API. Nailing the right input scheme, the semantic of keys or events will necessitate doing a bit of back and forth. I have in my task list to redesign the input system of the library and I may use the opportunity to transition those new events to that. Note of those expected breakage should be a big problem but expect some breakage in the following weeks.
• 8) I haven't polished the value tweaking, not exposed the slower/faster factor, not exposed the repeat delay/rate (they are factors of the keyboard delay/rate). So there are a handful of factors hardcoded in code. For now I'd rather have them hardcoded than expose them without much thought put into it. We may as well want to introduce some mechanism where the tweaking speed accelerate non-linearly, or even take advantage of analog triggers for tweaking.
• 9) The current control scheme I am using on a DualShock 4 is: D-Pad up/down/left/right: navigate, tweak values L/R: slow down/speed up tweaking values Analog: manual scroll (currently digital, haven't wired all of it) Cross button: press button, hold to tweak/activate widget, enter child, etc. Circle button: close popup, exit child, clear selection, etc. Square button(TAP): access menu, collapsing, window options, etc. Square button(HOLD)+Dpad: Resize window Square button(HOLD)+Analog: move window Square button(HOLD)+L/R change window focus, ALT-TAB style Triangle button: text input (requires user backend reading back io.WantTextInput to possibly display an OS keyboard display). (edited July 30)

Note that I didn't yet add Moving or Collapsing windows yet. The code for those is now trivial to add but the key question is how to decide on an input layout and how to expose it to the end-user so it is decently configurable. Things like ALT-TAB window focus alteration requires by design holding one button while pressing another (because of the MRU logic) so I don't yet know how to suitably expose all those inputs. Currently the L/R triggers are mapped on ImGuiKey_NavTweakSlower ImGuiKey_NavTweakFaster but that's also used to change focus. So while for a gamepad ImGuiKey_NavTweakSlower==ImGuiKey_NavPrevWindow and ImGuiKey_NavTweakFaster==ImGuiKey_NavNextWindow, on a keyboard we are likely to want to use Alt/Shift for Slower/Faster and CTRL+TAB/CTRL+SHIFT+TAB, etc. So there's work to rearrange those.

I expect we can also use analog stick and that will also weight in the design to pass all those inputs.

• 10) There is an useful option io.NavMovesMouse, currently off by default. When on, the mouse cursor can be moved in ImGui::NewFrame() when directional navigation is used. It does so by overwriting io.MousePos and setting io.WantMoveMouse=true. It is up to your backend when that flag is set to apply the new mouse position in your OS. Right now you need to apply it on the next frame (which is awkward, I will rework the code to handle this delay). Depending on your application/use case this option may be best.
• 11) I have spent a fair amount of time trying to get mouse+keyboard to coexist nicely but there's still probably one million edge cases to sort out. Note that I've been mostly testing with io.NavMovesMouse=true so there's probably issues I have overlooked that mostly appears with it set to false.
• 12) If you are running this on VR, some suggestions: if you want to display ImGui as a static overlay (not affected by head rotation) you may want to reduce DisplaySize and avoid rendering over your entire framebuffer. You can also increase the style.DisplaySafeAreaPadding value. Popups should stay within this rectangle while you can still partly move regular window outside. It may be just better to display it within the 3D world.
• 13) INSTRUCTIONS This is my test diff in imgui_impl_glfw.cpp (not committed). To be used as rough guidance. As discussed above I am currently passing all events through the keyboard array which is rather confusing and misleading.

In ImGui_ImplGlfw_NewFrame()

    io.NavMovesMouse = true;

    // Setup events/key mapping within the existing keyboard array.
    int avail_key = GLFW_KEY_LAST;
    io.KeyMap[ImGuiKey_NavActivate]     = avail_key++;
    io.KeyMap[ImGuiKey_NavCancel]       = avail_key++;
    io.KeyMap[ImGuiKey_NavWindowing]    = avail_key++;
    io.KeyMap[ImGuiKey_NavInput]        = avail_key++;
    io.KeyMap[ImGuiKey_NavLeft]         = avail_key++;
    io.KeyMap[ImGuiKey_NavRight]        = avail_key++;
    io.KeyMap[ImGuiKey_NavUp]           = avail_key++;
    io.KeyMap[ImGuiKey_NavDown]         = avail_key++;
    io.KeyMap[ImGuiKey_NavTweakFaster]  = avail_key++;
    io.KeyMap[ImGuiKey_NavTweakSlower]  = avail_key++;

    // Update Keyboard Nav
    const bool TEST_KEYBOARD = false;
    io.KeysDown[io.KeyMap[ImGuiKey_NavActivate]]    = TEST_KEYBOARD && io.KeysDown[GLFW_KEY_SPACE];
    io.KeysDown[io.KeyMap[ImGuiKey_NavCancel]]      = TEST_KEYBOARD && io.KeysDown[GLFW_KEY_ESCAPE];
    io.KeysDown[io.KeyMap[ImGuiKey_NavWindowing]]   = TEST_KEYBOARD && false;
    io.KeysDown[io.KeyMap[ImGuiKey_NavInput]]       = TEST_KEYBOARD && io.KeysDown[GLFW_KEY_ENTER];
    io.KeysDown[io.KeyMap[ImGuiKey_NavLeft]]        = TEST_KEYBOARD && io.KeysDown[GLFW_KEY_LEFT];
    io.KeysDown[io.KeyMap[ImGuiKey_NavRight]]       = TEST_KEYBOARD && io.KeysDown[GLFW_KEY_RIGHT];
    io.KeysDown[io.KeyMap[ImGuiKey_NavUp]]          = TEST_KEYBOARD && io.KeysDown[GLFW_KEY_UP];
    io.KeysDown[io.KeyMap[ImGuiKey_NavDown]]        = TEST_KEYBOARD && io.KeysDown[GLFW_KEY_DOWN];
    io.KeysDown[io.KeyMap[ImGuiKey_NavTweakFaster]] = TEST_KEYBOARD && io.KeyShift;
    io.KeysDown[io.KeyMap[ImGuiKey_NavTweakSlower]] = TEST_KEYBOARD && io.KeyAlt;

    // Update Joystick Nav
    if (glfwJoystickPresent(GLFW_JOYSTICK_1))
    {
        int buttons_count = 0;
        const unsigned char* buttons = glfwGetJoystickButtons(GLFW_JOYSTICK_1, &buttons_count);
        if (buttons_count > 0  && buttons[0]  == GLFW_PRESS) io.KeysDown[io.KeyMap[ImGuiKey_NavActivate]]= true;
        if (buttons_count > 1  && buttons[1]  == GLFW_PRESS) io.KeysDown[io.KeyMap[ImGuiKey_NavCancel]]  = true;
        if (buttons_count > 2  && buttons[2]  == GLFW_PRESS) io.KeysDown[io.KeyMap[ImGuiKey_NavWindowing]]  = true;
        if (buttons_count > 3  && buttons[3]  == GLFW_PRESS) io.KeysDown[io.KeyMap[ImGuiKey_NavInput]] = true;
        if (buttons_count > 10 && buttons[10] == GLFW_PRESS) io.KeysDown[io.KeyMap[ImGuiKey_NavUp]]      = true;
        if (buttons_count > 11 && buttons[11] == GLFW_PRESS) io.KeysDown[io.KeyMap[ImGuiKey_NavRight]]   = true;
        if (buttons_count > 12 && buttons[12] == GLFW_PRESS) io.KeysDown[io.KeyMap[ImGuiKey_NavDown]]    = true;
        if (buttons_count > 13 && buttons[13] == GLFW_PRESS) io.KeysDown[io.KeyMap[ImGuiKey_NavLeft]]    = true;
        if (buttons_count > 4  && buttons[4]  == GLFW_PRESS) io.KeysDown[io.KeyMap[ImGuiKey_NavTweakSlower]] = true;
        if (buttons_count > 5  && buttons[5]  == GLFW_PRESS) io.KeysDown[io.KeyMap[ImGuiKey_NavTweakFaster]] = true;
    }

Also in ImGui_ImplGlfw_NewFrame(), honor io.WantMoveMouse request:

    // Setup inputs
    // (we already got mouse wheel, keyboard keys & characters from glfw callbacks polled in glfwPollEvents())
    if (glfwGetWindowAttrib(g_Window, GLFW_FOCUSED))
    {
        if (io.WantMoveMouse)
        {
            // Requested to move mouse (when using directional navigation with 'NavMovesMouse=true'). Intentionally applied on following frame to compensate for render lag.
            glfwSetCursorPos(g_Window, (double)io.MousePos.x, (double)io.MousePos.y);
        }
        else
        {
            double mouse_x, mouse_y;
            glfwGetCursorPos(g_Window, &mouse_x, &mouse_y);
            io.MousePos = ImVec2((float)mouse_x, (float)mouse_y);   // Mouse position in screen coordinates (set to -1,-1 if no mouse / on another screen, etc.)
        }
    }
    else
    {
        io.MousePos = ImVec2(-1,-1);
    }

If you can spend an hour setting it up and reporting confusion or issues it'll be helpful!

Thanks!

Extra musing.. Unfortunately some of the code to support that feature has become rather fragile, not in the sense that it is crippled with bugs (hopefully it isn't), but many of those changes requires too much expertise and knowledge of the codebase, which isn't a good thing at all. The testing suite and documentation will need to take more priority in the future and I above everything want to steer this project away from ever being a bus-factor-1 project. I will return to this topic later and looking forward to gather any help I can to make the testing suite happen! (#435).

[ocornut]

Some of my todo list.

• [ ] A. Sort-out/finalize all the input bindings correctly -> toward exposing explicit gamepad vs keyboard to be able to tweak controls accordingly.
• [x] A. Move windows
• [x] A. Collapse windows
• [x] A. Reaching menubar vs. scrolling is super awkward, use a button to access different layers?
• [x] B. NavWindowing with a popup menu doesn't select the expected window
• [x] A. Tooltip position feedback loop (e.g. see vertical slider demo)
• [x] A. Columns: "Up/Down isn't staying in the same column. This is just a two column setup, pressing up ends up on the right column and down on the left column."
• [x] B. Explicit manual scrolling (e..g mapped on a analog stick): how to reapply suitable focus? currently only possible when no item is navigable.
• [x] A. Possible navigation issues when using ImGuiListClipper
• [x] B. InputText (single-line) allow up/down to move out of active widget.
• [x] A. Can't use windowing option when no window is selected (clicked on void)
• [x] B. Menus: expecting Left/Right to navigate parent like windows does.
• [x] B. Expose styling properly for the colors used by NavWindowing and the focused widget highlight
• [x] C. Combo: Holding button too long when selecting reopens the combo box.
• [ ] B. Menus: Navigating menus is still awkward in multiple ways.
• [ ] B. Investigate crossing over the boundaries of child windows, in particular those without scroll. Introduce a window flag to flatten child in term of navigation.
• [ ] C. Menubars inside modals windows are acting weird (broken in master as well)
• [ ] A. Problem various problem with graph navigation/scoring functions, currently biased toward vertical layouts
• [ ] C. Using scrolling should activate scrollbar in a way the user can tell programmatically (e.g. Log window)
• [ ] C. NavHighlight clipping issue within child window
• [ ] C. Merge all the old FocusIdx tabbing stuff into the new system
• [ ] B. Resizing window will currently fail with certain types of resizing constraints/callback applied
• [ ] C. Popup: introduce a default validation button e.g. SetItemDefaultValidation() activable from anywhere in the window with Enter. > Now can use ImGui::PushItemFlag(ImGuiItemFlags_SelectableDontClosePopup, true); / ImGui::PopItemFlag()
• [ ] C. TreeNode: NavLeft to close, NavRight on closed node to open. How would it behave with buttons/items after a closed treenode, and/or multiple columns?
• [ ] C. Drag/Slider: experiment with keeping item active when activated, using cancel to stop editing.
• [ ] B. Popup: add options to disable auto-closing popups when using a MenuItem/Selectable (#126)
• [ ] C. Lost of currently focused widget when using buttons that changes labels on click (obvious, but only made apparent with directional navigation- can we automagically work around it?)
• [ ] B. Bug with keeping visibility of navigated them within horizontal scrollbar. Stuck nav (visible in Horizontal Scrolling demo).

The big commit already includes fixes/changes for probably a hundred different things, so the left-over may be quite easy :) As long as we focus on gamepad this is fairly scoped and we can see the end of the tunnel (full keyboard friendlyness is another thing).

[unpacklo]

I haven't integrated any of this into my own code, just looking at your branch directly, but it seems like setup will be very simple.

The only hiccup I had was I had to install DS4Windows to get the controller inputs to be mapped correctly for your setup. I guess Sony hasn't provided any official drivers for the DS4... shame! However, I noticed that the touch pad works and I can click/move windows with it, but it looks like its a DS4Windows feature since I can move the mouse on my entire system and click things with it.

Navigation with the controller produces the expected results for most things, but the one area I've noticed which can yield very poor navigation is switching windows. I suspect it's because you may be cycling through windows somehow in memory? But most of the time I'm trying to cycle through them in some spatial fashion on screen. This may be exacerbated by the fact that I was moving the windows around with the mouse in between.

Will be a little while before I have more detailed feedback, but this is very promising!

[michaelbartnett]

Easy to get going, I liked the sub-selection in the style color editor. Something that felt like it was missing was a way to jump to the parent in a tree widget. That's tricky though, since I'd normally expect NavLeft to do that, which doesn't necessarily make sense if the children of that tree node have horizontal elements. I'm not doing very complex things with imgui at the moment, so I'll keep my comment brief.

Here's a quick hack for people wishing to test it out with an Xbox 360 controller on Mac:

    if (glfwJoystickPresent(GLFW_JOYSTICK_1))
    {
        struct GamepadMap
        {
            enum {
                DPAD_UP, DPAD_DOWN, DPAD_LEFT, DPAD_RIGHT,
                FACE_UP, FACE_DOWN, FACE_LEFT, FACE_RIGHT,
                SHOULDER_LEFT, SHOULDER_RIGHT,
                LAST
            };

            unsigned char map[LAST];

            bool checkpress(const unsigned char* buttons, int buttons_count, int btn)  { return buttons_count > btn && buttons[map[btn]] == GLFW_PRESS; }
        };

        static GamepadMap ds4win_map = {{10, 12, 13, 11, 3, 0, 2, 1, 4, 5}};
        static GamepadMap x360macos_map = {{0, 1, 2, 3, 14, 11, 13, 12, 8, 9}};
        // GamepadMap &padmap = ds4win_map;
        GamepadMap &padmap = x360macos_map;

        int buttons_count = 0;
        const unsigned char* buttons = glfwGetJoystickButtons(GLFW_JOYSTICK_1, &buttons_count);

        if (padmap.checkpress(buttons, buttons_count, GamepadMap::FACE_DOWN))      io.KeysDown[io.KeyMap[ImGuiKey_NavActivate]]= true;
        if (padmap.checkpress(buttons, buttons_count, GamepadMap::FACE_RIGHT))     io.KeysDown[io.KeyMap[ImGuiKey_NavCancel]]  = true;
        if (padmap.checkpress(buttons, buttons_count, GamepadMap::FACE_LEFT))      io.KeysDown[io.KeyMap[ImGuiKey_NavWindowing]]  = true;
        if (padmap.checkpress(buttons, buttons_count, GamepadMap::FACE_UP))        io.KeysDown[io.KeyMap[ImGuiKey_NavInput]] = true;
        if (padmap.checkpress(buttons, buttons_count, GamepadMap::DPAD_UP))        io.KeysDown[io.KeyMap[ImGuiKey_NavUp]]      = true;
        if (padmap.checkpress(buttons, buttons_count, GamepadMap::DPAD_RIGHT))     io.KeysDown[io.KeyMap[ImGuiKey_NavRight]]   = true;
        if (padmap.checkpress(buttons, buttons_count, GamepadMap::DPAD_DOWN))      io.KeysDown[io.KeyMap[ImGuiKey_NavDown]]    = true;
        if (padmap.checkpress(buttons, buttons_count, GamepadMap::DPAD_LEFT))      io.KeysDown[io.KeyMap[ImGuiKey_NavLeft]]    = true;
        if (padmap.checkpress(buttons, buttons_count, GamepadMap::SHOULDER_LEFT))  io.KeysDown[io.KeyMap[ImGuiKey_NavTweakSlower]] = true;
        if (padmap.checkpress(buttons, buttons_count, GamepadMap::SHOULDER_RIGHT)) io.KeysDown[io.KeyMap[ImGuiKey_NavTweakFaster]] = true;
    }

[EDIT] NavLeft not NavRight >_<

[ocornut]

@Roflraging Yeah, DS4window provide mouse emulation. I expect native PS4 developers to implement a similar setup to emulate a fallback mouse, even awkwardly.

Navigation with the controller produces the expected results for most things, but the one area I've noticed which can yield very poor navigation is switching windows. I suspect it's because you may be cycling through windows somehow in memory? But most of the time I'm trying to cycle through them in some spatial fashion on screen. This may be exacerbated by the fact that I was moving the windows around with the mouse in between.

Switching windows is indeed a previous/next thing based on the current z-order (~most recently used window). It currently works this way because it compares to Windows' ALT-TAB and because spatial navigation wouldn't work very well with lots of overlapping windows. It would work better if we could show a preview of the windows arranged in a linear fashion, the same way Mac/Windows does it, but that seems like lots of extra work and the contents of imgui windows aren't really fit for easy identification after resizing down. If you have any idea on how to improve this let me know. I don't think it is a big issue however.

Also thanks @michaelbartnett. Eventually we should provide similar code in the committed example that has some sort of nice remapping table, even if the code itself ends up being commented out.

I've got a bunch of e-mail feedback from @pdoane which he may copy here later.

[pdoane]

I did an integration into one of my test projects - mostly just a property grid and a couple of modal dialog boxes. My interest is in keyboard navigation which as you say is going to be more complicated, but even the controller support is already a good foundation.

Integration was easy. I followed your GLFW sample and was up and running in a few minutes. No major issues and a lot of the navigation already felt natural.

Menus:

• I'm expecting Left/Right while in one menu to go to the next one
• I don't have keyboard shortcuts manually implemented so that's not hooked up.

Modal Dialogs:

• Escape closed the dialog box. Good!
• Enter didn't didn't accept the dialog. I don't remember a flag for it but I'm assuming we'll need to semantically identify the Ok button.
• I want control over the initial focus. There's a text input field that would be the natural thing to be active on opening of the dialog.

Text input:

• After I've moved the highlight over a text input field, I'm expecting that I could start typing. Or at least to activate it and start typing.

Multi-column navigation:

• Left/Right seems to be working ok.
• Up/Down isn't staying in the same column. This is just a two column setup, pressing up ends up on the right column and down on the left column.

Drag controls:

• Having to hold the activate key while pressing left/right feels awkward.

List box:

• Having the selection be different from what I'm navigating is appropriate for some contexts and not others.

Tree Nodes:

• I am expecting Left/Right to open/close the node. Not sure how that would interact with a multi-column environment though.

Navigation override:

• I have a list box with a text input above it that is a filter. I'm expecting to be able to type into the filter box and then press the down key to transfer into navigating within the list box. Maybe that is the appropriate behavior for single-line text input?

[unpacklo]

It currently works this way because it compares to Windows' ALT-TAB and because spatial navigation wouldn't work very well with lots of overlapping windows.

So I think part of the reason why I have the spatial expectation at all is because of the controller bindings. Having it cycle through with left + right bumpers give me some sort of expectation that I should be cycling left/right on the screen. I'll have to try with different binds to see if it makes it better for me, but I suspect that the preview of the cycling order as in alt+tab makes the most difference.

I do tend to agree that this is not a huge issue, I personally don't have more than 2 or 3 windows at any time, if that.

[ocornut]

@pdoane:

Escape closed the dialog box. Good! Enter didn't didn't accept the dialog. I don't remember a flag for it but I'm assuming we'll need to semantically identify the Ok button. I want control over the initial focus. There's a text input field that would be the natural thing to be active on opening of the dialog.

There's a new function SetItemDefaultFocus() for that. So you could use that on the OK button as well if you are happy with just regularly "activating" it (mapped to "Space" in my example) vs a window "global" Enter to identify OK button. Perhaps doable as an extra function let's say SetItemDefaultValidate() which would record the ID and that can be activated from anywhere in the window with Enter?

TextInput: After I've moved the highlight over a text input field, I'm expecting that I could start typing. Or at least to activate it and start typing.

Right now you can type in by pressing NavInput (Enter) which works on sliders, drags.. we could make NavActivate function on TextInput as well, can't tell if that would feel more or less consistent?

Drag controls: Having to hold the activate key while pressing left/right feels awkward.

I will experiment with keeping the control activated until pressing NavCancel but I think the current scheme allows for faster interactions.

Tree nodes: I am expecting Left/Right to open/close the node. Not sure how that would interact with a multi-column environment though.

Yes need to look at columns before we can try that on trees. EDIT Problem is also that one can have items next to a closed tree node (I often do that myself).

and then press the down key to transfer into navigating within the list box. Maybe that is the appropriate behavior for single-line text input?

Will try.

[ocornut]

@pdoane

Multi-colum navigation Up/Down isn't staying in the same column. This is just a two column setup, pressing up ends up on the > right column and down on the left column.

Please generally provide screenshots or repro because that's a general navigation issue and may not be tied to columns. It works for example in the simple columns part of the demo, but acts weirdly as you described in the "Property Example" sample. In that case it is related to the Selectable() on the left incorrectly sticking out too far and the navigation scoring calculation not applying clipping over the item rectangle.

[ocornut]

@pdoane I have pushed some fixes, it would be nice if you could confirm if you columns issues are resolved or improved.

[ocornut]

API BREAKING CHANGES

• Renamed ImGuiKey_NavWindowing to ImGuiKey_NavMenu

Pushed various changes above. Most notable, the NavMenu button (recommended "Square" on a DualShock4 controller) allows toggling between the scrollable area of a window and the menubar. Holding the NavMenu button still allows resizing and selecting window focus.

I will add moving and collapsing window once I figure out the input scheme for that and the menu/layer system was a necessary step forward. May adopt Windows style NavMenu+left to access a window menu to select among those options, which also would be keyboard compatible (rather than taking advantage of multiple sticks).

[unpacklo]

Finally got around to checking out these controller changes in a non-trivial imgui setup.

Some thoughts:

• Will there be support for analogous middle mouse and right mouse action for the controller? In some of the GUIs I have, we make extensive use of middle mouse and right mouse to aid in selection/deselection of items. Seems like we need some sort of alternate select? Biggest concern for me is making sure that the GUI code that does rely on middle or right mouse doesn't need to handle a second separate code path for the controller.
• Fast scrolling/next item? Some of the scrolling areas we have are very large, and to cycle through the items at the default rate can be tedious. Obviously, a controller based GUI would probably rely on large menus less, but GUIs already set up, this might be very convenient.
• Support to pop up to top level child. In some of our windows, we make extensive use of child regions and popping to the top can require several button hits. Not necessarily something that needs to be supported directly at the input level, but might be useful to expose as a function call? I'm not sure. Probably not a huge priority.

I would write more, but it is very late for me, I'll try to get more thoughts tomorrow. Biggest thing I've realized is that having a GUI already set up with the mouse + keyboard expectation doesn't really work with the controller. It's workable, but not ideal! Many of the issues I currently have is much more of a problem with the GUI design fundamentally not matching the input method rather than legitimate issues with your implementation.

[ocornut]

@Roflraging

Will there be support for analogous middle mouse and right mouse action for the controller?

You can always map mouse buttons to free buttons of your controller (if there are any?), which may or not cover you enough if you have io.NavMovesMouse activated and your binding is honoring io.WantMoveMouse.

Fast scrolling/next item? Some of the scrolling areas we have are very large, and to cycle through the items at the default rate can be tedious. Obviously, a controller based GUI would probably rely on large menus less, but GUIs already set up, this might be very convenient.

"Faster next item" is a little cumbersome to get right, but I agree we need manual scrolling in (there's already support for scrolling but only when there's no item to interact with). The question is how to return back from "scrolling" to "selecting a widget" and how is said widget selected (perhaps based on relative widget position of where we were before scrolling?)

Currently in check-list above as - [ ] B. Explicit manual scrolling (e..g mapped on a analog stick): how to reapply suitable focus? currently only possible when no item is navigable.

Support to pop up to top level child.

There's different issues and possible solution here. If you can post screenshots (or e-mail privately, I have a NDA with your company) I could look at the specific case. If they are childs with no visible scrolling I think we could adopt a scheme where we can cross through parent-child borders. It is also possible we need extra window flags or options to parametrize those behaviors as well.

[ocornut]

Still uncertain about how to express/configure inputs, it appears too difficult to treat gamepad and keyboard as a same source and gets in the way of lots of possible improvements

Instead of attempting to expose semantic-only I may bite the bullet and expose separate explicit sets of inputs for gamepad and keyboard, and we can optimize input scheme specifically for those (with first focus on gamepad).

For now I have added 4 new digital inputs currently mapped to left-analog stick on my dual shock:

ImGuiKey_NavScrollLeft, // e.g. Analog left
ImGuiKey_NavScrollRight,// e.g. Analog right
ImGuiKey_NavScrollUp,   // e.g. Analog up
ImGuiKey_NavScrollDown, // e.g. Analog down

Used to:

• Manually scroll a window. At discussed above, the scheme used if that if you scroll until your current focused item is out of view and resume navigation from there, I won't jump back to navigate from focused item but rather resume navigation from the visible point closest to where the focused item was. This is a little awkward but allows making good use of scrolling. The standard behavior (at least what Windows does) is to resume from focused item so mouse-wheeling followed by keyboard lose the scrolling point.
• During manual scroll you can hold L/R to slow down or accelerate scrolling. (currently ImGuiKey_NavTweakSlower/ImGuiKey_NavTweakFaster, again naming/semantic is making it rather ugly) . Speed currently constant.
• When holding the NavMenu input (Square button) which is used to access menus, you can use this stick to move windows. Analog inputs aren't exposed so moving speed is constant yet but if I go the way discussed above of splitting gamepad/keyboard I will probably add them.
• Also you can use the NavMenu input (Square button) to access the Collapse arrow as well as the Close buttons.

With those changes, apart from the big questions mark of how to expose inputs, it is starting to be quite usable with a gamepad. There's probably hundreds of upcoming incremental changes obviously, but you can do stuff without a keyboard/mouse around.

Reminder: todo list is here https://github.com/ocornut/imgui/issues/323#issuecomment-233785300

The ugly binding for GLFW+DualShock4 with DS4Window is: We shall make that less ugly, probably using a dedicated input visualization/config window. Currently under "Keyboard,Mouse,etc." in the demo window there's a panel that shows all pressed inputs.

// FIXME-NAVIGATION
// Setup events/key mapping within the existing keyboard array.
int avail_key = GLFW_KEY_LAST;
for (int n = ImGuiKey_NavActivate; n < ImGuiKey_NavLast_; n++)
{
    io.KeyMap[n] = avail_key++;
    io.KeysDown[io.KeyMap[n]] = false;
}

// Update Keyboard Inputs
if (1)
{
    io.KeysDown[io.KeyMap[ImGuiKey_NavActivate]]    = io.KeysDown[GLFW_KEY_SPACE];
    io.KeysDown[io.KeyMap[ImGuiKey_NavCancel]]      = io.KeysDown[GLFW_KEY_ESCAPE];
    io.KeysDown[io.KeyMap[ImGuiKey_NavInput]]       = io.KeysDown[GLFW_KEY_ENTER];
    io.KeysDown[io.KeyMap[ImGuiKey_NavLeft]]        = io.KeysDown[GLFW_KEY_LEFT];
    io.KeysDown[io.KeyMap[ImGuiKey_NavRight]]       = io.KeysDown[GLFW_KEY_RIGHT];
    io.KeysDown[io.KeyMap[ImGuiKey_NavUp]]          = io.KeysDown[GLFW_KEY_UP];
    io.KeysDown[io.KeyMap[ImGuiKey_NavDown]]        = io.KeysDown[GLFW_KEY_DOWN];
    io.KeysDown[io.KeyMap[ImGuiKey_NavTweakFaster]] = io.KeyShift;
    io.KeysDown[io.KeyMap[ImGuiKey_NavTweakSlower]] = io.KeyAlt;
}

// Update Joystick Inputs
int axes_count = 0;
const float* axes = glfwGetJoystickAxes(GLFW_JOYSTICK_1, &axes_count);
if (glfwJoystickPresent(GLFW_JOYSTICK_1))
{
    int buttons_count = 0;
    const unsigned char* buttons = glfwGetJoystickButtons(GLFW_JOYSTICK_1, &buttons_count);
    if (buttons_count > 0  && buttons[0]  == GLFW_PRESS) io.KeysDown[io.KeyMap[ImGuiKey_NavActivate]]= true;
    if (buttons_count > 1  && buttons[1]  == GLFW_PRESS) io.KeysDown[io.KeyMap[ImGuiKey_NavCancel]]  = true;
    if (buttons_count > 2  && buttons[2]  == GLFW_PRESS) io.KeysDown[io.KeyMap[ImGuiKey_NavMenu]]    = true;
    if (buttons_count > 3  && buttons[3]  == GLFW_PRESS) io.KeysDown[io.KeyMap[ImGuiKey_NavInput]]   = true;
    if (buttons_count > 10 && buttons[10] == GLFW_PRESS) io.KeysDown[io.KeyMap[ImGuiKey_NavUp]]      = true;
    if (buttons_count > 11 && buttons[11] == GLFW_PRESS) io.KeysDown[io.KeyMap[ImGuiKey_NavRight]]   = true;
    if (buttons_count > 12 && buttons[12] == GLFW_PRESS) io.KeysDown[io.KeyMap[ImGuiKey_NavDown]]    = true;
    if (buttons_count > 13 && buttons[13] == GLFW_PRESS) io.KeysDown[io.KeyMap[ImGuiKey_NavLeft]]    = true;
    if (buttons_count > 4  && buttons[4]  == GLFW_PRESS) io.KeysDown[io.KeyMap[ImGuiKey_NavTweakSlower]] = true;
    if (buttons_count > 5  && buttons[5]  == GLFW_PRESS) io.KeysDown[io.KeyMap[ImGuiKey_NavTweakFaster]] = true;

    if (axes_count > 0 && axes[0] < -0.5f) io.KeysDown[io.KeyMap[ImGuiKey_NavScrollLeft]]  = true;
    if (axes_count > 0 && axes[0] > +0.5f) io.KeysDown[io.KeyMap[ImGuiKey_NavScrollRight]] = true;
    if (axes_count > 1 && axes[1] < -0.5f) io.KeysDown[io.KeyMap[ImGuiKey_NavScrollDown]]  = true;
    if (axes_count > 1 && axes[1] > +0.5f) io.KeysDown[io.KeyMap[ImGuiKey_NavScrollUp]]    = true;
}

[ocornut]

FYI I have been working on how to pass inputs, including analog inputs, in a way that would work for both gamepad and future keyboard. That'll probably be the last major change before the thing can be marked "usable beta".

[ocornut]

Ok I made a first-pass breaking commit. EDIT typos, fixes

• All the ImGuiKey_NavXXX were removed from the ImGuiKey enum, now using ImGuiNavInput_XXX (ImGuiNavInput enum).
• The input data is stored in the io.NavInputs[] float array.
• Float allows for analog inputs! For keyboard you can use 0.0f/1.0f for anything analog the whole 0.0f..1.0f is available. Up to the user to remap raw joystick values, imgui will test >0.0f.. The example GLFW code below does a naive linear remap of 0.3f..0.9f to 0.0f..1.0f.
• There's no weird remapping indirection, and you don't have to touch your io.KeyDown[] array which is meant to be your native keyboard data.
• The new enums are currently called ImGuiNavInput_PadActivate, so that explicitly include the word PAD. This is because some of the input won't make sense for keyboard and vice-versa, they'll need to be separated. You can still totally map your keyboard keys on the Pad inputs and it'll work but you'll know that you are using an interface that has been tweaked for gamepad.

Here's the enum

enum ImGuiNavInput_
{
    ImGuiNavInput_PadActivate,      // press button, tweak value                    // e.g. Circle button
    ImGuiNavInput_PadCancel,        // close menu/popup/child, lose selection       // e.g. Cross button
    ImGuiNavInput_PadInput,         // text input                                   // e.g. Triangle button
    ImGuiNavInput_PadMenu,          // access menu, focus, move, resize             // e.g. Square button
    ImGuiNavInput_PadUp,            // move up, resize window (with PadMenu held)   // e.g. D-pad up/down/left/right
    ImGuiNavInput_PadDown,          // move down
    ImGuiNavInput_PadLeft,          // move left
    ImGuiNavInput_PadRight,         // move right
    ImGuiNavInput_PadScrollUp,      // scroll up, move window (with PadMenu held)   // e.g. right stick up/down/left/right
    ImGuiNavInput_PadScrollDown,    // "
    ImGuiNavInput_PadScrollLeft,    //
    ImGuiNavInput_PadScrollRight,   //
    ImGuiNavInput_PadFocusPrev,     // next window (with PadMenu held)              // e.g. L-trigger
    ImGuiNavInput_PadFocusNext,     // prev window (with PadMenu held)              // e.g. R-trigger
    ImGuiNavInput_PadTweakSlow,     // slower tweaks                                // e.g. L-trigger
    ImGuiNavInput_PadTweakFast,     // faster tweaks                                // e.g. R-trigger

    ImGuiNavInput_COUNT,
};

Here's my current GLFW binding code for DualShock4 with DSWindow:

// Setup directional navigation events/key mapping
bool nav_uses_keyboard = true;
bool nav_uses_gamepad = true;
memset(io.NavInputs, 0, sizeof(io.NavInputs));

// Enable to allow ImGui moving mouse cursor when using keyboard/gamepad navigation
io.NavMovesMouse = false;

// Update Keyboard Inputs
if (nav_uses_keyboard)
{
    #define MAP_KEY(NAV_NO, KEY_NO) { if (io.KeysDown[KEY_NO]) io.NavInputs[NAV_NO] = 1.0f; }
    MAP_KEY(ImGuiNavInput_PadActivate,      GLFW_KEY_SPACE);
    MAP_KEY(ImGuiNavInput_PadCancel,        GLFW_KEY_ESCAPE);
    MAP_KEY(ImGuiNavInput_PadMenu,          GLFW_KEY_LEFT_ALT);
    MAP_KEY(ImGuiNavInput_PadInput,         GLFW_KEY_ENTER);
    MAP_KEY(ImGuiNavInput_PadUp,            GLFW_KEY_UP);
    MAP_KEY(ImGuiNavInput_PadDown,          GLFW_KEY_DOWN);
    MAP_KEY(ImGuiNavInput_PadLeft,          GLFW_KEY_LEFT);
    MAP_KEY(ImGuiNavInput_PadRight,         GLFW_KEY_RIGHT);
    MAP_KEY(ImGuiNavInput_PadTweakSlow,     GLFW_KEY_LEFT_ALT);
    MAP_KEY(ImGuiNavInput_PadTweakSlow,     GLFW_KEY_RIGHT_ALT);
    MAP_KEY(ImGuiNavInput_PadTweakFast,     GLFW_KEY_LEFT_SHIFT);
    MAP_KEY(ImGuiNavInput_PadTweakFast,     GLFW_KEY_RIGHT_SHIFT);
    #undef MAP_KEY
}

// Update Gamepad Inputs
if (nav_uses_gamepad)
{
    #define MAP_BUTTON(NAV_NO, BUTTON_NO)       { if (buttons_count > BUTTON_NO && buttons[BUTTON_NO] == GLFW_PRESS) io.NavInputs[NAV_NO] = 1.0f; }
    #define MAP_ANALOG(NAV_NO, AXIS_NO, V0, V1) { float v = (axes_count > AXIS_NO) ? axes[AXIS_NO] : V0; v = (v - V0) / (V1 - V0); if (v > 1.0f) v = 1.0f; if (io.NavInputs[NAV_NO] < v) io.NavInputs[NAV_NO] = v; }
    int axes_count = 0, buttons_count = 0;
    const float* axes = glfwGetJoystickAxes(GLFW_JOYSTICK_1, &axes_count);
    const unsigned char* buttons = glfwGetJoystickButtons(GLFW_JOYSTICK_1, &buttons_count);
    MAP_BUTTON(ImGuiNavInput_PadActivate,   0);
    MAP_BUTTON(ImGuiNavInput_PadCancel,     1);
    MAP_BUTTON(ImGuiNavInput_PadMenu,       2);
    MAP_BUTTON(ImGuiNavInput_PadInput,      3);
    MAP_BUTTON(ImGuiNavInput_PadUp,         10);
    MAP_BUTTON(ImGuiNavInput_PadDown,       12);
    MAP_BUTTON(ImGuiNavInput_PadLeft,       13);
    MAP_BUTTON(ImGuiNavInput_PadRight,      11);
    MAP_BUTTON(ImGuiNavInput_PadFocusPrev,  4);
    MAP_BUTTON(ImGuiNavInput_PadFocusNext,  5);
    MAP_BUTTON(ImGuiNavInput_PadTweakSlow,  4);
    MAP_BUTTON(ImGuiNavInput_PadTweakFast,  5);
    MAP_ANALOG(ImGuiNavInput_PadScrollUp,   1,  +0.3f,  +0.9f);
    MAP_ANALOG(ImGuiNavInput_PadScrollDown, 1,  -0.3f,  -0.9f);
    MAP_ANALOG(ImGuiNavInput_PadScrollLeft, 0,  -0.3f,  -0.9f);
    MAP_ANALOG(ImGuiNavInput_PadScrollRight,0,  +0.3f,  +0.9f);
    #undef MAP_BUTTON
    #undef MAP_ANALOG
}

This is me toying with a macro based version, but I won't keep that. The naive data driven version didn't encourage custom control flow and I'm pretty sure that beginner users will feel restricted to any provided data structure, whereas I want to encourage people to tweak their input. So I'll may just keep something like the above but with a helper function, the question is wether we can provide a similar structure for all the bindings or not. Even if that code isn't part of imgui itself it's nice to figure out something more elegant.

[ocornut]

whereas I want to encourage people to tweak their input.

In particular, I am looking at mechanism/idioms to make it easy to selectively transfer input from one part of the app (imgui/tools) to another (game) which may vary per peripheral, per platform, etc. Right now the nav_uses_keyboard nav_uses_joystick flags in that demo code are part of the backend but it would be nice to provide a standard interface in imgui to allow app to pass info to the bindings.

[ocornut]

I am stuck on this one, so will be thinking aloud:

[ ] Popup: add options to disable auto-closing popups when using a MenuItem/Selectable

@MrMarkie asked for it under those terms:

"Also, when I select a check option from a sub menu, the sub menu closes. I wonder if it should stay open until you press cancel? It's just that setting a bunch of check boxes one after another could be quite laborious"

This is indeed desirable when using MenuItem that are meant to be toggled/checked. But not desirable for other MenuItem that are meant to be activated, such as a typical "Open.." or "Quit" item.

The problem is that the two MenuItem() APIs don't allow to differentiate one from case another.

bool MenuItem(const char* label, const char* shortcut = NULL, bool selected = false, bool enabled = true);
bool MenuItem(const char* label, const char* shortcut, bool* p_selected, bool enabled = true);

We could fairly assume that the bool* version is always a checkable when the pointer isn't NULL and not a checkable when NULL. But in many situations the bool version can be used (most commonly for code that doesn't use bool for storage or need to access the data via accessors). And then passing 'false' is ambiguous, we can't tell if the item is a checkable or not. Particularly common as if you just want to create an "Open" item you are likely to just either pass false or NULL inconsistently.

A) If we were to disable auto-closing popup solely based on passing a non-NULL pointer to the bool* version that would make both overloads behave inconsistently which isn't so great. It would solve 90% use cases for most people but make the whole thing feel inconsistent :(

B) We could introduce new flags to the function, e.g. MenuItemEx() which could fit the additional information we want (either "can be checked" or "dont close popup"/"close popup"). That seems a little overkill just for this purpose but we can keep this idea brewing. Current API was designed as one of the rare function to take two bools, which is generally not great API design but in this case made a lot of sense as they are very commonly used and we want to make code terse, but that means we can't just add a new flag to an existing flag set. And we are not going to add a third bool while I'm alive. So MenuItemEx(const char* label, const char* label, ImGuiMenuItemFlags flags) could be a candidate, for again, probably not willing to add that just for this. Even with that added, it would make thing a little more painful to use on the users' side.

C) Third solution, perhaps more approachable, would be to add a new window state flag to enable-disable the behavior of auto-closing popups. The problem is to settle on what's the correct way to name this API. The behavior that's currently used but not exposed publicly is completely arbitrary ("Selectable/MenuItem closes current popup when their parent is a popup, unless ImGuiSelectableFlags_DontClosePopups flag is set"). If we expose it as an option in the public API it comes with extras expectations: why just Selectable/MenuItem? Can I configure it separately for different types of widget?

D) Fourth solution, if you need this behavior just use Checkbox() and not MenuItem().

[ocornut]

@MrMarkie following post above, I have now added a non-publicly exposed flag, if you include <imgui_internal.h> you can do in your menu: ImGui::PushItemFlag(ImGuiItemFlags_SelectableDontClosePopup, true); / ImGui::PopItemFlag(). To disable closing the menu. You can still call CloseCurrentPopup() to close a menu explicitly in code. Putting it in imgui_internal.h for now means I have more flexibility with changing the name/semantic/rules of this later. In particular we might want to split the behavior between mouse and gamepad/keyboard.

[ocornut]

MOVING TO #787