The future of Nuklear - brainstorming ideas

dumblob commented 2 years ago

@BieHDC wrote on 2021-12-20:

This should be the discussion place of what should be done and etc as the discussion started in this issue.

This is also my first time directly working with others together and figuring out this new interface.

dumblob commented 2 years ago

@dumblob wrote on 2021-12-20:

Will reply later (hopefully at the latest in a few days but ... anything can happen).

Btw. could this be made public? I think we shouldn't hide such discussions :wink:.

dumblob commented 2 years ago

@BieHDC wrote on 2021-12-20:

I dont know, i am literally using this interface first time too. But i am busy the next days anyway, so...

dumblob commented 2 years ago

@RobLoach wrote on 2021-12-25:

One thing I'd love to see revamped is the gamepad/keyboard-only support. While you can accomplish it, it takes some work. Would be nice to have it just work.

Hejsil commented 2 years ago

With my null-termination-be-gone branch, it's pretty clear that I'm a big fan of having an API that is friendly and efficient to bindings from other languages. This would include:

API that does not use null terminated strings
libc is optional (nuklear already forfills this)
Mark none nullable pointers as such with some annotation (there are compiler attributes for this)
- This is useful for tools that generates bindings based on header files

One other thing I have as a wishlist item is a better allocator interface. The current one has the same interface as malloc/free, but this kind of interface requires the allocator keeps track of the allocation length, which can be extra unnecessary overhead for some allocation schemes.

Hejsil commented 2 years ago

Another one: A very low level API for nk_label and other text widgets that provides some sort of string builder, which can then be used to implement printf formatting and the like. Idk the exact API, but something like this is what I'm thinking.

struct nk_command_text_builder builder = nk_label_builder_begin(ctx);
nk_command_text_builder_append(&builder, "prefix - ");
nk_command_text_builder_append(&builder, some_string_variable);
nk_command_text_builder_append(&builder, " - suffix");
nk_label_builder_end(&builder);

The idea here being that the string will be built in place on the nk_command_text that will be emitted. This would allow other languages to wrap this API and use their standard formatting APIs.

dumblob commented 2 years ago

A very low level API for nk_label and other text widgets that provides some sort of string builder, which can then be used to implement printf formatting and the like...

Just saw this and find it interesting - especially if extended to allow for custom string "type" - e.g. https://github.com/josephg/librope (or its "successor" https://github.com/josephg/jumprope-rs ).

iperov commented 2 years ago

Hi guys. Is it possible to create brand new UI concept, something between [Immediate mode UI] and [Full state UI]. Keep simplicity of UI declaration, but provide more flexibility, such as animations?

dumblob commented 2 years ago

@iperov yes, that's one of the few things I myself had in mind - but didn't have time to write a concise description of what I think. I'll get to it at some point.

Feel free to take a look at https://github.com/traffaillac/traffaillac.github.io/issues/1 and https://github.com/Pauan/rust-signals/discussions/31 in the meantime.

If you have any ideas, speak out - the more feedback the better :wink:.

iperov commented 2 years ago

I have an idea, but I don't know how relevant it is because I don't design GUI engines. For example when we declare a sequence of layout and widgets gui.vertical_layout_start() gui.label() gui.horizontal_layout_start() gui.label() gui.button()

the engine should automatically build internal layout/widget objects/structures? and they won't reset on the next pass if the widget declarations match the internal representation?

also can user get access to such structures to define additional logic? for example some pseudocode

button_obj = gui.button()
if button_obj.clicked():
   ... instant click process right here

button_obj.call_on_hover( on_button_hover ) - call function if mouse is hovering button

func on_button_hover (button_obj.):
   time = button_object.mouse_hovering_elapsed_time()
   button_obj.set_color(  RGB()*time  )
   if time >= 1.0:
     button_obj.set_icon ( icon_ready )
   if time < 1.0:
     button_obj.disable_click()

also 

gui.button.set_default_on_button_hover (on_button_hover) - to set handler for all buttons

so it is ok when some widget sequences are changed and do not match internal representation, they will just recreated, and all animation will reset.

also I would like to have a library for python as well. Because machine learning requires visualization, and QT is too redundant for that purpose.

threedslider commented 2 years ago

Hi,

Nuklear is very cool project, I will use it surely. My request is : will be it as 100% full customization with 100% visual for my C++ project ?

dumblob commented 2 years ago

My request is : will be it as 100% full customization with 100% visual for my C++ project ?

The current Nuklear is very customizable. If you can specify technical shortcomings you experience, then please describe them in detail here. Otherwise please use the Nuklear issue tracker to discuss everything else. Thanks!

dumblob commented 2 years ago

To keep in mind the other side of the spectrum: https://wiki.csswg.org/ideas/mistakes (it's missing some - e.g. ::first-line which is subject to font-size thus creating a loop...).

threedslider commented 2 years ago

The current Nuklear is very customizable. If you can specify technical shortcomings you experience, then please describe them in detail here. Otherwise please use the Nuklear issue tracker to discuss everything else. Thanks!

Ok cool ! Thank you too and keep it up ^^

dumblob commented 2 years ago

Some very interesting thoughts about "command" drawing (Nuklear currently also uses commands) and different kinds of such commands etc. from a highly experienced person: https://github.com/red/REP/issues/103#issuecomment-1024922637 .

BieHDC commented 2 years ago

Sorry for the long silence, but i finally got time again. Here are some of the keypoints i would like to have:

Only 2 layouting systems, a simple one like nk_layout_dynamic and one for complex ones. Consider the default chat application: left sidebar spanning from top to bottom 64px wide, then you have a subspace with a top bar holding channel description. Then you have another subspace with input bar at the bottom and a send button, and the rest in the middle for text. Something like this should be "easily" doable, but right now we lack the "take the rest of the remaining space and treat it as new window" kind of command (which is kinda what my subspace pr is trying to do). This also means that instead of being top left to buttom down only layouting, you should be able to specify the anchor point (i think thats what its called). In this example for the text input and send button you would anchor to bottom left.
I would also make much more use of caching. So you still stay immediate mode but first time code path it initialises memory from heap. This makes saving and accessing the state much easier and should remove some "hacky" things as annotated in current nuklear.
Having simple modular widgets to build larger more complex widgets. For example being able to connect a bunch of text input widgets to build a bigger excel sheet. This was something that was personally causing me issues since some widgets have a mind of their own. Basically have a nice way to make complex widgets.

Thats what i remember right now, please give feedback.

dumblob commented 2 years ago

Images (incl. icons, buttons, etc.) shall be better dealt with - see e.g. https://github.com/Immediate-Mode-UI/Nuklear/pull/444 .

dumblob commented 2 years ago

Transparency - masks etc. could be useful...

dumblob commented 2 years ago

Tiling images (as background etc.) could be desirable for some use cases (neo)Nuklear tries to cover. See https://github.com/Immediate-Mode-UI/Nuklear/pull/444 .

iperov commented 2 years ago

@dumblob do you have any resources to read about internal UI implentations and architectures?

dumblob commented 2 years ago

@iperov not much as of now. But if you have anything potentially related, please speak up - if not for me, then it's guaranteed valuable for others.

ryuukk commented 2 years ago

My suggestions:

animation/tweening/easing support
better theming support (per control/id style)
better font styling support (outline, shadow)

BieHDC commented 2 years ago

So over the last months i tried to get something to show multiple times, but eventually it becomes overwhelming. IRL work probably did not help ether dealing with that. Did one of you get any examples done? Because to be honest, i feel like writing something new from scratch that aligns more with my mindset.

dumblob commented 2 years ago

I for myself did not code any examples yet - I am still in the brainstorming phase looking for proper abstractions. I am especially focused on dynamic behavior as over the years I started to realize that having solved the dynamic behavior (i.e. anything coming and changing over time), everything else is suddenly much easier to find abstractions for (layouting, styling, customizing, etc.).

But if you have anything (even non-functional), feel free to post a link to your repo so that we can take a look at your ideas to distil your thoughts :wink:.

iperov commented 2 years ago

have you seen any new UI related projects?

twisted-nematic57 commented 2 years ago

Add an Apple macOS backend

dumblob commented 2 years ago

@iperov I am trying to incrementally keep up with UI & UX development all over the world. So yes and no to your question :wink:. I have seen new projects/repos but unfortunately no new approaches.

iperov commented 2 years ago

@dumblob What new approaches are you suggesting?

dumblob commented 2 years ago

To focus first only on dynamic behavior (i.e. anything changing over time) as I outlined in my comment above: https://github.com/Immediate-Mode-UI/neonuklear/issues/1#issuecomment-1202577136 .

zecruel commented 2 years ago

Hi! My sugestions:

Add a multiline text edit widget with automatic line wrap. I made an implementation of this widget (in the distant past), but it needs a lot of improvement: https://github.com/zecruel/nuklear/blob/master/src/nuklear_edit.c
I think it is necessary to review the text rendering methods. On long texts, the entire string is rendered, there is no optimization to process what is currently being displayed.
Still about text rendering, it would be nice to include more support for multiple languages. For example, including support for rendering "backwards" texts, such as Arabic and Hebrew.

dumblob commented 2 years ago

Hi @zecruel, thanks for chiming in! Feel free to make a draft PR (to the standard Nuklear repository - https://github.com/Immediate-Mode-UI/Nuklear ) with your multiline text edit widget and we will discuss it in more depth. That would be a great addition IMHO.

Also I wonder whether the two other points could be implemented already now in the current Nuklear (as neonuklear is only brainstorming and I can say for sure for myself that I will not have the time to develop it in the upcoming few years). Both seem technically possible (see e.g. https://github.com/nothings/stb/issues/686 regarding Arabic & Thai).

zecruel commented 2 years ago

Got it, you're looking for more disruptive ideas.

I insist on the issue of text rendering, which is much more complex than it seems. Nuklear's default mode of "baking font" needs to be rethought to provide more flexibility for rendering rich text.

Multilingual support is also a challenge. "Eastern" languages are computationally difficult, both for direct rendering and for interactive text input.

dumblob commented 2 years ago

@zecruel I can not agree more. Feel free to make proposals (even to the current Nuklear) - with or without code - how would you imagine the better support for fonts. Note also that there were some people trying to utilize the option of using different font libraries than the default stb. If your situation allows for this, feel free to give another font library a try and we will be more than happy to assist and hear your feedback and/or any contributions. Thanks a lot!

iperov commented 2 years ago

In my experience of creating architectures, it is impossible to create a universal architecture that allows you to do everything and still have a high level of abstraction.

iperov commented 2 years ago

I like the QT widgets architecture that I use in python. The downside is that I build GUIs in my mind, and layout functions are not well designed for that. It took me a few months to learn layout and widget behavior to build GUIs quickly enough by imagination.

redthing1 commented 1 year ago

https://github.com/vurtun/nuklear/issues/828

A very nice and simple tab system like this, built in.

iperov commented 1 year ago

Inside any Immediate-mode-GUI (IMGUI), a Retained-GUI (RGUI) is implemented in order to store the widget states. But as we see in the example of DearIMGUI - the implementation is just awful. Therefore why implement RGUI when you can use a ready-made RGUI framework, like QT/wx/Tk?

Technically IMGUI is just marking up RGUI widgets and getting the events of each widget directly into the markup function. If the markup of a widget doesn't match a specific widget, it just gets recreated to the right widget and the event fetches will already be in the next markup.

In that case, instead of redrawing the GUI screen at 30-60FPS , the layout will only be rebuilt, and the RGUI framework itself won't cause costly redrawing every frame.

At this point you can create completely new lib.

But I've gone further and am building an experimental GUI in Python that doesn't need to invoke a rebuild of the markup 60 times per second.

Instead of getting the events in the markup function, I made "abstract backend controls" that implement primitive data management and work without a connected GUI to the backend. Examples of such controls: Number (for SpinBox), Path (for file dialog), Signal (for button), SingleChoice (for ComboBox). In the markup function we mark the widgets referring to these Controls. We do not manipulate with widgets, but with controls. For example Signal.emit() is controlled by the Pushbutton widget. In the backend class I subscribe to the Signal.on_signal event and handle the pressing of the button, changing the state of the class and optionally call the GUI markup again, which can change the widget structure.

Example

https://user-images.githubusercontent.com/8076202/205911270-f0f77af6-ac18-44f0-8734-098e33a075aa.mp4

example of video player buttons markup:

...
with gui.hstack() as hst, gui.size(XGUI.ESize.S3), gui.color(XGUI.EColor.red):
    gui.spacer().expand(h=True)

    gui.pushbutton(self._con_seek_begin_signal, image=gui.get_ionicon_image(XGUI.EIonIcon.play_skip_back_circle_outline))
    gui.pushbutton(self._con_seek_backward_signal, image=gui.get_ionicon_image(XGUI.EIonIcon.play_back_circle_outline))

    play_btn = gui.pushbutton(self._con_play_signal, image=gui.get_ionicon_image(XGUI.EIonIcon.play_circle_outline), tooltip='Play')
    stop_btn = gui.pushbutton(self._con_stop_signal, image=gui.get_ionicon_image(XGUI.EIonIcon.pause_circle_outline), tooltip='Stop' )
    if fp.is_playing:
        play_btn.disable()
    else:
        stop_btn.disable()

    gui.pushbutton(self._con_seek_forward_signal, image=gui.get_ionicon_image(XGUI.EIonIcon.play_forward_circle_outline))
    gui.pushbutton(self._con_seek_end_signal, image=gui.get_ionicon_image(XGUI.EIonIcon.play_skip_forward_circle_outline))

    gui.spacer().expand(h=True)

with gui.hstack() as hst:
    gui.spacer()
    with gui.hstack() as hst:
        hst.compact()
        gui.spinbox(self._con_frame_index)
        gui.spinbox(self._con_frame_count, read_only=True)
    gui.spacer()
...

example of backend controls definition

self._con_seek_begin_signal.on_signal.subscribe(self._on_con_seek_begin_signal)

self._con_seek_backward_signal = boxcon.Signal()
self._con_seek_backward_signal.on_signal.subscribe(self._on_con_seek_backward_signal)

self._con_play_signal = boxcon.Signal()
self._con_play_signal.on_signal.subscribe(self._on_con_play_signal)

self._con_stop_signal = boxcon.Signal()
self._con_stop_signal.on_signal.subscribe(self._on_con_stop_signal)

self._con_seek_forward_signal = boxcon.Signal()
self._con_seek_forward_signal.on_signal.subscribe(self._on_con_seek_forward_signal)

self._con_seek_end_signal = boxcon.Signal()
self._con_seek_end_signal.on_signal.subscribe(self._on_con_seek_end_signal)

self._con_frame_index = boxcon.Number(range=(0,fp.frame_count-1), control_func=self._on_con_frame_index)
self._con_frame_index.set_number(fp.frame_idx, bypass_control=True)

self._con_frame_count = boxcon.Number(range=(fp.frame_count-1, fp.frame_count-1))
self._con_frame_count.set_number(fp.frame_count-1, bypass_control=True)

example of Play signal handler

def _on_con_play_signal(self):
    ... # some work
    self._con_error_stream.emit('Play start')
    self._update_gui() # - request to rebuild GUI

I think the DearIMGUI era is coming to an end, and we can create something better.

dumblob commented 1 year ago

@iperov yeah, good old signals (slots, events, ... however you call that). Unfortunately that concept is usually implemented too low level and thus highly error prone and becomes cumbersome (already in mid-complex UIs).

Therefore streams/live_variables/data_flows/... concept (one can view it as "high level" signals/slots/...) shall to be introduced as described e.g. in https://github.com/vlang/ui/issues/7#issuecomment-575102513 (I briefly touched this in the discussion linked above in https://github.com/Immediate-Mode-UI/neonuklear/issues/1#issuecomment-1017675458 ).

iperov commented 1 year ago

highly error prone

not in my lib.

dumblob commented 1 year ago

@iperov where is your lib hosted? Let us move the discussion about "error prone" etc. of signals/slots/events APIs to your repository.

iperov commented 1 year ago

But you started this discussion

iperov commented 1 year ago

I was a bit surprised that you were talking about errors at all, discussing abstract concepts. It seems that programming in C seriously changes your perception and openness to new paradigms. Good thing that after 15 years of C/C++ programming I don't go into that filth.

dumblob commented 1 year ago

How far shall neonuklear go with its API abstractions? There is https://github.com/linebender/glazier and similar projects which strive to make it easier to focus only on one thing in UI libs.

ryuukk commented 1 year ago

how is glazier relevant?

nuklear should remain non bloatware, keep it simple, fast, extensible and portable renderer agnostic IMGUI library, nothing else

dumblob commented 1 year ago

@ryuukk sure, Nuklear will stay as it is. But here we discuss neonuklear - i.e. some form of a successor (if any).

If we find out there is no need for more disruptive APIs/ideas than the current Nuklear offers, then neonuklear will not take off and thus will not exist at all :wink:.

Googler1 commented 1 year ago

Something else.

dumblob commented 1 year ago

@Googler1 could you elaborate?

Nuklear and Dear ImGui are most probably the only most portable (supporting most platforms) medium-capable (feature-wise) GUI libs in the world. Do you mean neonuklear shall continue in this league or do you mean something else?

MichaelKv commented 1 year ago

Brainstorming ideas? On a GUI library? In 21st century? This is really sad if you really need them. But OK. Let it be so.

It shall not be just a library. It shall be at least a framework – and infrastructure with widgets and primitives for scientific, CAD (architecture, electronics, microelectronics, IT, etc.), hardware (medical, houses, IoT, etc.) and so on primitives developed by independent companies. And it must not be a monolithic one.
It shall not be C. C++ is the must. Starting a new big ambitious project in C is just unlikely. And it is enormously expensive. C++ means exceptions, multithreading, coroutines, etc. The framework might probably not use all these directly but shall at least be aware or them.
You will need to invest into its architecture and design a lot. I.e., it shall not have graphic primitives’ IDs generated from programmer’s provided strings and things like that as in Dear ImGui. All the things must be simple, intuitive and direct as visiting a crocodile farm.

iperov commented 1 year ago

@MichaelKv

On a GUI library? In 21st century?

Especially in the 21st century!

I am currently writing a layer on top of Qt, because there is NO gui library that meets my requirements for new software.

disagree. Most "infrastructure" implemented by other libraries such as numpy, opencv, etc. No need to reinvent the wheels.
disagree. C++ has gotten dirtier lately. C is the only parent that keeps the language clean. C programs can be ported to other languages in pure form in languages like V, Zig...
agree. DearImGui is piece of shit.

iperov commented 1 year ago

By the way, does anyone know if there are GUI projects where the rendering is done entirely on GLSL shaders? There are SDF functions https://iquilezles.org/articles/distfunctions2d/ , with which you can draw anything in maximum quality, as well as animation.

dumblob commented 1 year ago

@iperov I am only aware of the following:

https://github.com/zauonlok/renderer https://github.com/nokola/nonaquad https://github.com/audulus/vger

They use shaders to varying degrees, but I am not sure if for everything.

Immediate-Mode-UI / neonuklear

The future of Nuklear - brainstorming ideas #1