Certain mouse events don't work

[Nielsbishere]

My guess is that point->box checks are broken, but it can also obviously also be my code. The following work fine: sliders, progress bars. Trying to use anything else however, won't work correctly. Other things such as radio, push buttons, sliders and checkboxes don't work. Sliders however do work sometimes, but barely. https://imgur.com/a/VQYFFRs The code can be found at on my repo but can only be built on Windows with CMake.

NK code

enum { EASY, NORMAL, HARD };
static int op = EASY, active[3]{ 1, 0, 1 }, selected{};
static float value = 0.6f;
static int i =  20;
static usz test{};
List<const c8*> names = {
    "Cheese",
    "Peanuts"
};
if (nk_begin(ctx, "Show", nk_rect(50, 50, 220, 300),
             NK_WINDOW_BORDER|NK_WINDOW_SCALABLE|NK_WINDOW_MOVABLE|NK_WINDOW_CLOSABLE)) {
    // fixed widget pixel width
    nk_layout_row_static(ctx, 30, 80, 1);
    if (nk_button_label(ctx, "Button")) {
        oic::System::log()->debug("Hi");
    }
    // fixed widget window ratio width
    nk_layout_row_dynamic(ctx, 30, 2);
    if (nk_option_label(ctx, "Easy", op == EASY)) op = EASY;
    if (nk_option_label(ctx, "Normal", op == NORMAL)) op = NORMAL;
    if (nk_option_label(ctx, "Hard", op == HARD)) op = HARD;
    nk_layout_row_dynamic(ctx, 30, 2);
    nk_checkbox_label(ctx, "Yes?", active);
    nk_checkbox_label(ctx, "No?", active + 1);
    nk_checkbox_label(ctx, "Maybe?", active + 2);
    nk_layout_row_dynamic(ctx, 30, 2);
    nk_combobox(ctx, names.data(), int(names.size()), &selected, 30, nk_vec2(1, 1));
    // custom widget pixel width
    nk_layout_row_begin(ctx, NK_STATIC, 30, 2);
    {
        nk_layout_row_push(ctx, 50);
        nk_label(ctx, "Volume:", NK_TEXT_LEFT);
        nk_layout_row_push(ctx, 110);
        nk_slider_float(ctx, 0, &value, 1.0f, 0.1f);
        nk_progress(ctx, &test, 100, 1);
    }
    nk_layout_row_end(ctx);
}
nk_end(ctx);

Render code (after nk code)

if (!refresh && previous.data())
    refresh = previous.size() != ctx->memory.needed || memcmp(previous.data(), ctx->memory.memory.ptr, previous.size());

//Convert to draw data

if (refresh) {
    refresh = false;
    static const struct nk_draw_vertex_layout_element vertLayout[] = {
        { NK_VERTEX_POSITION, NK_FORMAT_FLOAT,  vertexLayout[0].offset },
        { NK_VERTEX_TEXCOORD, NK_FORMAT_FLOAT, vertexLayout[1].offset },
        { NK_VERTEX_COLOR, NK_FORMAT_R8G8B8A8, vertexLayout[2].offset },
        { NK_VERTEX_LAYOUT_END }
    };
    struct nk_convert_config cfg = {};
    cfg.shape_AA = NK_ANTI_ALIASING_ON;
    cfg.line_AA = NK_ANTI_ALIASING_ON;
    cfg.vertex_layout = vertLayout;
    cfg.vertex_size = vertexLayout.getStride();
    cfg.vertex_alignment = 4 /* Detect? */;
    cfg.circle_segment_count = 22;
    cfg.curve_segment_count = 22;
    cfg.arc_segment_count = 22;
    cfg.global_alpha = 1.0f;
    cfg.null = atlasTexture;
    nk_buffer cmds, verts, idx;
    nk_buffer_init(&cmds, &allocator, NK_BUFFER_DEFAULT_INITIAL_SIZE);
    nk_buffer_init(&verts, &allocator, NK_BUFFER_DEFAULT_INITIAL_SIZE);
    nk_buffer_init(&idx, &allocator, NK_BUFFER_DEFAULT_INITIAL_SIZE);
    nk_convert(ctx, &cmds, &verts, &idx, &cfg);
    vbo.release();
    ibo.release();
    primitiveBuffer.release();
    auto vboStart = (u8 *)verts.memory.ptr;
    auto iboStart = (u8 *)idx.memory.ptr;
    vbo = {
        g, NAME("NK VBO"),
        GPUBuffer::Info(Buffer(vboStart, vboStart + verts.needed), GPUBufferType::VERTEX, GPUMemoryUsage::LOCAL)
    };
    ibo = {
        g, NAME("NK IBO"),
        GPUBuffer::Info(Buffer(iboStart, iboStart + idx.needed), GPUBufferType::INDEX, GPUMemoryUsage::LOCAL)
    };
    primitiveBuffer = {
        g, NAME("Primitive buffer"),
        PrimitiveBuffer::Info(
            BufferLayout(vbo, vertexLayout),
            BufferLayout(ibo, BufferAttributes(GPUFormat::R16u))
        )
    };
    commands->clear();
    commands->add(
        BindPipeline(pipeline),
        SetClearColor(Vec4f { 0, 0.5, 1, 1 }),
        BeginFramebuffer(target),
        SetViewportAndScissor(),
        ClearFramebuffer(target),
        BindPrimitiveBuffer(primitiveBuffer),
        BindDescriptors(descriptors)
    );
    const nk_draw_command *cmd {};
    nk_draw_index offset {};
    nk_draw_foreach(cmd, ctx, &cmds) {
        if (!cmd->elem_count) continue;
        Texture t = Texture::Ptr(cmd->texture.ptr);
        auto r = cmd->clip_rect;
        commands->add(
            r.w == 16384 ? SetScissor() : SetScissor({ u32(r.w), u32(r.h) }, { i32(r.x), i32(r.y) }),
            DrawInstanced::indexed(cmd->elem_count, 1, offset)
        );
        offset += u16(cmd->elem_count);
    }
    commands->add(
        EndFramebuffer()
    );
    nk_buffer_free(&cmds);
    nk_buffer_free(&verts);
    nk_buffer_free(&idx);
    g.present(target, swapchain, commands);
    u8 *prev = (u8*)ctx->memory.memory.ptr;
    previous = Buffer(prev, prev + ctx->memory.needed);
} else
    g.present(target, swapchain);
//Reset
nk_clear(ctx);

Input events

//Receive events
nk_input_begin(ctx);
//Could potentially use a callback system for efficiency TODO:
bool processedMouse{};
for(auto *dvc : vi->devices)
    if (dvc->getType() == InputDevice::Type::KEYBOARD) {
        //Only loop through nuklear keys
        for (usz i = 0; i < NKey::count; ++i) {
            //Get our key
            String name = NKey::nameById(i);
            usz keyId = Key::idByName(name);
            if (keyId == Key::count) continue;
            //Send to NK
            auto state = dvc->getState(InputHandle(keyId));
            if (state == InputDevice::PRESSED)
                nk_input_key(ctx, nk_keys(NKey::values[i]), 1);
            else if (state == InputDevice::RELEASED)
                nk_input_key(ctx, nk_keys(NKey::values[i]), 0);
        }
    } else if(dvc->getType() == InputDevice::Type::MOUSE) {
        if (processedMouse) continue;
        f64 x = dvc->getCurrentAxis(MouseAxis::AXIS_X);
        f64 y = dvc->getCurrentAxis(MouseAxis::AXIS_Y);
        f64 px = dvc->getPreviousAxis(MouseAxis::AXIS_X);
        f64 py = dvc->getPreviousAxis(MouseAxis::AXIS_Y);
        if (px == x && py == y) continue;
        //Only loop through nuklear keys
        for (usz i = 0; i < NMouseButton::count; ++i) {
            //Get our key
            String name = NMouseButton::nameById(i);
            usz keyId = MouseButton::idByName(name);
            if (keyId == MouseButton::count) continue;
            nk_input_button(ctx, nk_buttons(i), int(x), int(y), int(dvc->getCurrentState(ButtonHandle(keyId))));
        }
        nk_input_motion(ctx, int(x), int(y));
        nk_input_scroll(ctx, nk_vec2(f32(dvc->getCurrentAxis(MouseAxis::AXIS_WHEEL)), 0));
        processedMouse = true;
    }
nk_input_end(ctx);

[dumblob]

Don't have time to analyze your code (hopefully someone else will), but there are several things in Nuklear to consider when having this issue (the following is not exhaustive).

1. The current implementation has some known bugs - see e.g. https://github.com/vurtun/nuklear/pull/925 or https://github.com/vurtun/nuklear/pull/803 .
2. It might be a backend issue in combination with your underlying system.
3. It might be due to potential padding issues https://github.com/Immediate-Mode-UI/Nuklear/pull/2 .
4. Something else.
5. An arbitrary combination of the above.

[Nielsbishere]

It was partially because of my implementation. But it worked after I got rid off NK_KEYSTATE_BASED_INPUT and sent it to nk through callbacks, so perhaps that is an issue?

[dumblob]

I remember some issues with the two input "modes" (one reacting on rising edge and the other on the falling edge). You have to use the correct one for your chosen backend. And of course, there might still be some bugs. Feel free to read all demo/s to get a glimpse how to approach that.

[Nielsbishere]

Now I noticed what you were talking about; clicking through a tooltip or checkbox. It should consume it instead of keep checking if the mouse is in the area

[dumblob]

Yep, that was the effect of incorrect usage of

/// NK_KEYSTATE_BASED_INPUT         | Define this if your backend uses key state for each frame rather than key press/release events

and also maybe some bug(s).

[Nielsbishere]

I don't use that anymore

[Nielsbishere]

It's because mouse events aren't consumed or because the order of when a click is used is wrong. This means that if you have two elements on top of each other, it should only be consumed for the top.

[dumblob]

That sounds to me like the known problem with overlapping widgets/windows (especially popups and menus). Everything overlapping doesn't have first class support in Nuklear and is 1 frame late thus making place for weird states and bugs. This is not easily solvable in an elegant way (there is actually a way - see https://github.com/Immediate-Mode-UI/Quarks ).

Could this be the reason for the poor experience you're describing?

[Nielsbishere]

Yes, it is fixed in https://github.com/Immediate-Mode-UI/Nuklear/pull/14