Rich UI and graphics with notcurses

[olanod]

Describe the solution you'd like

Notcurses is a library facilitating complex TUIs on modern terminal emulators, supporting vivid colors, multimedia, threads, and Unicode to the maximum degree possible.

It would be interesting to use notcurses to support rich graphics, i.e. plotting data or open multimedia. Overtime the library could be used in more areas of nushell's UI like in menus, prompts, etc.

[rgwood]

Notcurses is, indeed, extremely cool. I was looking into writing .NET bindings for it a while back (unrelated to Nushell). I'd recommend that anyone not familiar with it check out this demo video: https://www.youtube.com/watch?v=dcjkezf1ARY

I think we might be open to PRs for this, but we'd need answers to a few questions first:

• how hard is it to statically link Notcurses into a Rust app like Nushell (and does it work on all 3 major OSs)?
• what kind of compile time and binary size penalty would we pay for embedding Notcurses?

It's also possible that Notcurses might make more sense in Nushell plugins than in Nushell's core.

[fdncred]

We used to have some plugins that used tui-rs I believe. I always fixed we'd return to tui-rs.

[olanod]

I think we might be open to PRs for this, but we'd need answers to a few questions first:

* how hard is it to statically link Notcurses into a Rust app like Nushell (and does it work on all 3 major OSs)?

* what kind of compile time and binary size penalty would we pay for embedding Notcurses?

I haven't tried it yet but there is a notcurses crate that has most features and re-exports libnotcurses-sys to access whatever is not implemented yet. The creator of notcurses also seems interested in Rust, I take that as a positive sign that support will continue to improve. If static linking is not already the default for the sys crate it should be simple to add support for it.
About the size I don't know what would be acceptable, what is mentioned in the readme is,

Notcurses can be built without its multimedia functionality, requiring a significantly lesser set of dependencies.

So if opening something like videos is not a priority then some bytes can be spared.

If PRs are welcomed, what kind of initial integration would be nice to see? perhaps rendering nushell on a notcurses plane without adding much/any new functionality? some kind of integration with reedline? I'm not very familiar with nushell's architecture

[fdncred]

LOL, the author in that video made me laugh by saying, "The rust community is a little huggy feely for my taste".

If PRs are welcomed...

That's a big yes, please submit a PR or 2 or 10 with this stuff. 😆

what kind of initial integration would be nice to see? perhaps rendering nushell on a notcurses plane ...

yes, that sounds interesting.

I was also thinking about something hopefully simple like open cat.png and it displays the image in the terminal for real, kind of like with sixel or other terminal rendering technologies. I can do it in iterm2 with a script but it would be cool if it was native.

reedline + nushell

I wonder if we could render some cool completion combo boxes or some cool UI listbox? It would be neat to see an experiment.

[rgwood]

I was also thinking about something hopefully simple like open cat.png and it displays the image in the terminal for real kind of like with sixel or other terminal rendering technologies.

Yeah, I think a command that displays an image would be a good first experiment. That seems like one of Notcurses's strong suits, and it should (🤞) involve way fewer architectural changes than overhauling Nushell/Reedline's rendering.

[Fl1tzi]

I'm currently trying on implementing picture viewing. It looks like it would work.

[fdncred]

@Fl1tzi cool. is there a repo where i can look at the code. i've been interested in drawing pics for a while. i've successfully been able to use the kitty and iterm2 protocols to draw pics but i'm guessing this may be something independent of that?

[Fl1tzi]

use image::EncodableLayout;
use notcurses::*;

fn main() {
    let img = image::open("test-image.png")
        .unwrap()
        .into_rgba8();
    let height = img.height();
    let width = img.width();
    /* print!("height: {} width: {}", height, width);*/

    let buf = img
        .as_bytes();

    let mut nc = Notcurses::new_cli().unwrap();

    let mut visual = Visual::from_rgba(buf, (width, height)).unwrap();
    // visual.set_blitter_pixel();
    visual.set_blitter(Blitter::Half)

    let mut pl = Plane::new(&mut nc).unwrap();
    visual.blit_plane(&mut nc, &mut pl).unwrap();
    pl.render().unwrap();

}

(You can change the blitter to get another type of quality)

This is actually the only thing needed to show a picture anywhere on the screen. It is more of a proof of concept because it does not scale the image to the screen size currently and all that things.

[fdncred]

very interesting. It would be cool to have a nuplugin that displays images as an experimental nushell command.

[Fl1tzi]

So I got further and we should really do it as an experimental plugin. I would say we could provide it as an extra plugin at first and later maybe point open to it when it's stable.

For now I was able to set the whitespaces right so it fits in the output without laying on text. Next thing is scaling/resizing it so it fills the whole output. Only thing that currently concerns me is that libnotcurses must be installed on the system.

So the next step is probably really implementing it in the source code.

However this is the current state where I'm at:

---

• picture is 6020x4024
• rendered with default blitter

nearest algorithm for scaling execution time: 582ms

Triangle algorithm for scaling execution time: 802ms

The pixelblitter (the high quality one) is currently not working on my system and I still need to figure out why.

[fdncred]

oh, wow. Those look pretty pixelated but cool!

[Fl1tzi]

Yeah it's because I'm using Alacritty. In Kitty you have the "native quality" image viewing.

[fdncred]

ya, we'll want to test a variety of terminals on Windows, MacOS, and Linux eventually. That's why i was thinking that a experimental plugin could probably be ideal for this type of thing so we can get a minimum viable candidate in and the people an test it and contribute to it if they're interested.

[Fl1tzi]

Notcurses does provide image viewing for Iterm2 so mac would also get the great viewing. Every other terminal that does not provide that type of support for pixels is able to just run the most compatible blitter. But yes we should test it at first.

[tryggvil]

Viu (https://github.com/atanunq/viu) or more specifically its underlying library (https://github.com/atanunq/viuer) could also be a more minimal option for displaying images. It supports the Kitty/iTerm2 image protocol as well as optionally Sixel

[olanod]

Viu looks nice, I'd say if we only end up using notcurses for pictures(and video?) then sure, a lighter weight library is better. But the point of notcurses is to eventually use more of its capabilities to draw more parts of the interface.