Improvements for DTS UI/UX

tym2k1 commented 7 months ago

The problem you're addressing (if any)

The Dasharo Tool Suite's menu aesthetics could benefit from a better interface. The existing implementation lacks visual appeal and experiences formatting issues across different aspect ratios (like below when ran in QEMU).

Describe the solution you'd like

A comprehensive list of TUI libraries is available in the form of rothgar/awesome-tuis repository, along with example projects implementing TUIs that could serve as useful references. It might be also possible to port a custom font to the minimal linux image and set it as the default in the TTY. This font could be also patched to provide icons.

(Some libraries even have limited support of images so the Dasharo logo could be visible from the TTY)

Use of available open-source libraries could potentially streamline the process of implementing those changes.

It's important though for this changes to not make the Automated testing more difficult and even treat it as a window of opportunity to make them easier to implement in the future.

Where is the value to a user, and who might that user be?

Improving the interface's aesthetics could make the user experience more inviting, particularly for those initially deterred by a rough appearance. This enhancement may change perceptions of Dasharo and firmware updates, making the idea of managing your firmware seem more seamless and less daunting.

Describe alternatives you've considered

There were attempts at this by @macpijan at https://github.com/Dasharo/dasharo-issues/discussions/361#discussioncomment-5177841. Some discussion also ensued if the style should be changed prior to incorporating changes related to showing users more information on the DTS welcome message at https://github.com/Dasharo/dasharo-issues/issues/540.

Additional context

No response

tym2k1 commented 7 months ago

As there are several limitations posed by the Linux VT it might be worth considering implementing projects like fbterm or KMSCON into DTS image. Neither of them is present in the OpenEmbedded Layer Index and KMSCON seems to be mostly abandoned by now but fbterm is actively maintained by Debian.

tym2k1 commented 7 months ago

Also for some eye-candy a plymouth splash screen could be added with the dasharo logo or even some modified dasharo logo that would specify that it is Dasharo Tool Suite that is booting :)

pietrushnic commented 7 months ago

What about firewalls and embedded devices like MinnowBoard Turbot? How that would work?

macpijan commented 7 months ago

It would not :)

pietrushnic commented 7 months ago

So, either this new UX design would support both wither it has to support serial/text only.

tym2k1 commented 7 months ago

In that case I propose two solutions:

introduce a separate grub bootmenu option. So we'd end up with one for minimal serial installation and one for more enhanced graphical interface
Keep one bootmenu option.
- I think it's possible to create TUIs that will work with serial they would just need to utilise control sequences (like ANSI escape codes). For instance, the ANSI escape sequence to clear the screen is \033[2J. The TUI could send such sequence over the serial connection but the terminal emulator would have to support them. The $TERM env variable informs both the system and applications about the type of terminal emulator in use, essentially describing its capabilities, including supported control sequences. For TUIs to function optimally over serial connections, it's crucial that the TERM variable is correctly set on both the device and the host machine. This ensures that when TUI applications send control sequences over the serial connection, they are sending sequences that are compatible with the terminal emulator being used on the host. Of course the TUI libraries would need to support this to some extent but that needs further research.
- It might be possible within the software to detect whether it is operating in a TTY environment or a serial connection, allowing it to automatically switch between different interfaces accordingly
  - Below is a simple PoC of the this (tested it on meta-rte and got Detected a serial connection. as the tty evaluates over minicom to /dev/ttyS0)

#!/bin/bash

# Check if stdout (file descriptor 1) is attached to a terminal
if [ -t 1 ]; then
    echo "Running in a TTY environment."

    # Attempt to distinguish a serial connection (simplistic approach)
    TTY=$(tty)
    if [[ "$TTY" == /dev/ttyS* || "$TTY" == /dev/ttyUSB* ]]; then
        echo "Detected a serial connection."
        # Logic for serial connection
    else
        echo "Detected a non-serial terminal."
        # Logic for non-serial TTY
    fi
else
    echo "Not running in a TTY environment."
    # Logic for non-TTY
fi

It's important to avoid creating additional future work by ensuring that the core components of both interfaces are shared. This approach would prevent the duplication of efforts in the long run. Thus, a scenario in which separate tests are created for the serial and graphical interfaces should be deemed a failure.

pietrushnic commented 7 months ago

Would that work, e.g., for PC Engines? I recommend MrChromebox UI was exercised; maybe it could have an automatic selection of printing on both display and serial; then on display, there would be something more sophisticated/beautiful, but let's be honest, the top need is UX, not artistic design - it may but doesn't have to mean the same. Usability is the top priority. Also, the critical question is the resources needed for implementation.

Final technical decision I would leave to @macpijan

macpijan commented 7 months ago

introduce a separate grub bootmenu option. So we'd end up with one for minimal serial installation and one for more enhanced graphical interface

Most use-cases of DTS does not even use GRUB, it is booted via iPXE.

I think it's possible to create TUIs that will work with serial they would just need to utilise control sequences (like ANSI escape codes). For instance, the ANSI escape sequence to clear the screen is \033[2J. The TUI could send such sequence over the serial connection but the terminal emulator would have to support them. The $TERM env variable informs both the system and applications about the type of terminal emulator in use, essentially describing its capabilities, including supported control sequences. For TUIs to function optimally over serial connections, it's crucial that the TERM variable is correctly set on both the device and the host machine. This ensures that when TUI applications send control sequences over the serial connection, they are sending sequences that are compatible with the terminal emulator being used on the host. Of course the TUI libraries would need to support this to some extent but that needs further research.

Deep dive into TUI libraries is probably not our focus, but we could use one if it can provide a proper usability on both serial and graphic terminals. It is a matter of running some examples on both, I recall starting one from https://github.com/charmbracelet/bubbletea over serial and it looked OK-ish.

It might be possible within the software to detect whether it is operating in a TTY environment or a serial connection, allowing it to automatically switch between different interfaces accordingly

Yes, it is possible, but implementing two UI paths is asking for troubles. We could not incorporate simple improvements to existing one over last year, so need to be realistic here.

TLDR: I am in favor of using TUI library as long as it will produce a single application, with a single UI, no matter if we use serial connection or graphical output - it must be optimized for serial.

The obvious benefit is we could also use a "proper" programming language for the rest of the logic as well, instead of bash.

Things to consider:

How much more effort than integrating this we need for this TUI path.
What would be the other benefits.
Which TUI to use, can we implement the rest of the required features with the same language, what are the other side effects (e.g. image size increase).

tym2k1 commented 7 months ago

Messing around with framebuffer and this guide I managed to insert raw pixel data straight into /dev/fb0 thus inserting and image into the basic linux VT. So it's possible to insert a better DTS logo (or other stuff) into the interface while not getting in conflict with the serial communication (as nothing is actually written to the stdout).

macpijan commented 4 months ago

Given our current situation, we must focus more on testing and features working correctly, not UI and new technologies.

Dasharo / dasharo-issues