Nuklear on Microwindows on Aquila

[ghaerr]

Hi!

I ended up looking at your AquilaOS last week and found it well written. Then, I got redirected to your website and read a few of your articles... and found out about Nuklear. Wow what a super good-looking and capable immediate mode GUI!!! I immediately started thinking that I should port it to Microwindows, my small framebuffer-based graphical windowing system, and consider how it could be made the standard UI for that platform. Now, a week later, I have completed the initial port, and have had some great ideas about the next steps to make this work as a completely client/server environment where each process would have its own Nuklear context, but have the drawing all performed by Microwindows, which runs on top of raw frame buffer, X11, SDL, etc. This seemed to be what you were looking for in your blog articles, and this may be a way to do it without basically having to write a windowing system all over again, but still get the benefits of a standard UI for Aquila. We have a terminal emulator also for shell tasks.

I'm not sure if you're interested, but if so, I'd be happy to talk about how each Nuklear "app" could run as as separate process, while still looking exactly as though they all currently are implemented within one window.

I've attached a screenshot showing the standard MIcrowindows window frame, with Nuklear running inside it, as a single process for the moment. The window frame will be replaced with one differently drawn that will exactly resemble the internal Nuklear frame, but with the Nuklear application's window flags set to no titlebar, no border, no minimize or close box, an no movement. When the user drags the Microwindows frame around, it will look like Nuklear moving it but the windowing system will be doing the blit-based window move instead.

-- Greg

[thethumbler]

Hello Greg, I'm very happy to hear from you :)

Actually I've had that in mind and actually tried to work on Nuklear/Microwindows, it's very minimalist and delivers great results. However, I didn't have much time to work on it and currently I'm baffled with exams, it would be very nice to do it though.

I already have the basic requirements to run Microwindows on Aquila but haven't tried porting it as of yet, I'll work on it when I have the time (between the exams, maybe) and let you know if I get it to work successfully.

Keep me updated about your Nuklear/Microwindows port, looking forward to have it on Aquila.

[ghaerr]

Hi,

I finished porting Nuklear GUI to Microwindows. Nuklear apps now run separately client/server and Microwindows displays window headers and close boxes identical to Nuklear. It all now works except for window resizing via border-dragging. I have a Nano-X wrapper that basically #includes any Nuklear UI application .c file which then can be run talking to the Nano-X server as a separately linked process.

[thethumbler]

Hello,

I managed to run the Embedded MW and NanoX with the demos, I made only slight modifications to the framebuffer screen driver and disabled Keyboard & Mouse support, so it just renders the window.

I then compiled the code with the server/client model, and the server creates the socket, binds and listens to it, but I haven't implemented select() yet (I mainly used threading for async I/O before), so that would be the next step. I'll implement and add support for Keyboard and Mouse, create a fork of your repo, push the modifications and the config file to build for Aquila and send you a PR.

Thank you for sharing this with me, I'm excited about it.

[ghaerr]

That is very cool, and you make fast work!!

If you pull the latest from my repo, the above code should run unmodified and automatically draw using the updated Nuclear UI look and feel :)

You don't have to have select(), although that will block the CPU of course when no events. The nanox/srvmain.c::GsSelect() function can be changed using #if's from the config file or Arch.rules (HAVE_SELECT), to use an alternate method for the time being. The last GsSelect() uses polling for you to continue without having to implement select yet.

When you're ready to run actual Nuklear apps, take a look at what I've done in demos/nuklear/demo-nuklear.c. I like it for the time being, but I'm having to know the Nuklear application size ahead of time in order to create the Nano-X window to the right size. I'm trying to stay out of the nk_begin/end main GUI loop to provide maximum portability, but I'm not convinced my current implementation is the best. I didn't implement the window resize drag yet for the same reason that I want to stay out of the GUI loop.

Let me know what help you need, I'm glad you're considering using this for AquilaOS!

[ghaerr]

Another related topic: Microwindows needs a better terminal emulator, do you think it would be worthwhile for me to port your fbterm to Nano-X? The emulators we currently have were not well written and don't implement scrollback buffers, and the fonts leave something to be desired, although they will run vi.

I read somewhere you were considering replacing fbterm, what were those reasons?

I haven't taken a hard look at fbterm, but Nano-X implements a "buffered" window type that draws into a back buffer until a flush event. We also implement direct framebuffer windows but need a better method for non-hardware frame buffer access. Does fbterm implement scroll back and work well for modern requirements but with a small footprint?

[thethumbler]

fbterm uses libvterm to handle terminal sequences, libvterm provides events to draw/destroy sections of the screen, fbterm has no scrollback buffer (although it can be implemented) and the fonts are fixed size. My problem with fbterm is that some sequences don't work well, that's not a problem of libvterm though, but rather my tty driver inside the kernel. I'd suggest starting from fbterm only as a reference point for libvterm integration.

I wanted to use an in-house terminal sequences handling library instead of libvterm, just to learn more about it, libvterm is very powerful and can be a good fit for your purposes.

[thethumbler]

Hello,

Now we have Microwindows/Aquila!

I managed to get mouse & keyboard to work (using select(2)) and here is a video to show it off.

This is a rapid prototype though, the keyboard driver is just the TTY driver but using stdin instead of opening a new file, and a new mouse driver is written for Aquila. The server/client model is working as well.

I didn't try with Nuklear yet, but I'm sure it will work. I'll clean the code and push ASAP, probably in a few days.

[ghaerr]

Now we have Microwindows/Aquila!

I managed to get mouse & keyboard to work (using select(2)) and here is a video https://youtu.be/yhz7a5mJ1Aw to show it off.

This is a rapid prototype though, the keyboard driver is just the TTY driver but using stdin instead of opening a new file, and a new mouse driver is written for Aquila. The server/client model is working as well.

I didn't try with Nuklear yet, but I'm sure it will work. I'll clean the code and push ASAP, probably in a few days.

Very cool!!

Sure, send a PR over. I have been writing lots of code but none in the drivers/ area. You should definitely try pulling the latest tree, then running a few of the bin/demo-nuklear-* demos (simultaneously). I now have window resizing working with the new nuklear-style draw code.

I also rewrote the nuklear wrapper last night, it requires a call from in-between nk-begin/nk-end but seems to work quite well, and doesn’t require any other modifications to get any nuklear app running on… soon: AquilaOS!

[thethumbler]

Hello @ghaerr

I have sent you a PR that adds AquilaOS support, I have yet to push my changes to aquila and setup a guide on how to build a toolchain for aquila. It works though and I suppose my modifications to your code base is minimal.

I have a problem though, when I close any nuklear application I always get a segmentation fault (MW applications don't have the same behavior). Here is the log with DEBUG enabled, I didn't get to debug it yet, but your insight would be very much helpful.

[ghaerr]

I'm pretty sure that problem is related to a double-free() memory management mixup when deallocating a window's resources (in this case, the list of client's interested in a window's events).

I thought I had that fixed, but obviously not. In looking at it, my debug code is still intact: there are two cases "FREE 1" and "FREE 2" in nanox/srvutil.c and nanox/srvnet.c respectively. Try commenting out one or the other free() and see whether that stops the problem (and likely starts a memory leak).

[ghaerr]

When you say "MW applications don't have this behavior", the win32 mw apps certainly won't as they don't use the nanox/.c window API. Can you confirm that other Nano-X app's behavior, it would be interesting to know.

All Nano-X applications, when run on the NX server with compiled-in window manager (NANOWM=Y) support, do a GrSelectEvents on the root window for child updates (GR_EVENT_TYPE_CHLD_UPDATE) in order to see a new client window map event and use that for dressing a window an reparenting. The select event request on the root window and subsequent window container creates another wp->eventclients list for the window manager for the client window. I am thinking that this eventclients list may be getting freed twice in the process of deallocating the client window and its window manager-owned frame window.

[thethumbler]

It was actually due to bad handling of a special case in Aquila kernel itself, when the client terminates the socket the buffers are freed and further access to them would sigfault. And since Aquila has that rather inconvenient strategy of blaming the process for everything, it sigfaults nano-X even though it originated in kernel code. It's fixed now.

[ghaerr]

Well that's good to know! In the meantime looking at this, I could not get it to duplicate, although I did find the original location of a free() that was commented out for debugging. I'll uncomment it to fix the memory leak.

[thethumbler]

Hello @ghaerr

I started working on a new terminal emulator based on libvterm, and already getting results. Will release it tomorrow probably, when I get (almost) everything working.

[ghaerr]

Cool! I just added direct client-side framebuffer or window back-buffer mmap access, which will greatly help with display update speed, if you're using that method. I'll take a look at it when released.