Implement `urwid` image widget

[AnonymouX47]

Resolves #71

• Adds term_image.widget subpackage and term_image.widget.urwid submodule.
• Adds the following widgets and related classes for the displaying images with the urwid TUI framework.
  • UrwidImage
  • UrwidImageCanvas
  • UrwidImageJanitor
  • UrwidImageScreen

One major feature left out in this PR is horizontal trimming of graphics-based images because it requires the implementation a non-trivial feature at the core of this library. This will happen in a few releases to come (tentatively planned for 0.8.0).

Credit: Major thanks to @danschwarz for his consistent suggestions and help with testing things out.

[AnonymouX47]

Hello @danschwarz! 😃

Please, I'll appreciate if you can help test this out.

[danschwarz]

Thanks! I pulled the urwid-widget branch and did a make install locally into the venv of my test app.

import term_image seems to work

mywidget = UrwidImage(img,format='RGB',upscale=False)

fails with NameError: name 'UrwidImage' is not defined.

I'm not a Python expert so this is probably an environment config issue on my side, but in case I'm just not importing the right library, let me know and that'll be an easy fix.

[danschwarz]

also tried from term_image import widget then attempted to print out all functions in widget.. it prints []. So probably an environmental issue?

[AnonymouX47]

First of all, I apologize for not giving any directions earlier. 🤦🏾‍♂️

Did you install with make install?

If so, then term_image.widget shouldn't be empty, it should contain two classes, UrwidImage and UrwidImageCanvas. I just made sure of this by testing with a fresh virtual environment.

The key thing is the urwid package must also be installed (in the same environment) for the widgets to be available. You can do that by install urwid normally, though running make install does that.

As for importing the required parts of the library, this should be sufficient:

from term_image.image import AutoImage
from term_image.widget import UrwidImage

To use the widget, the docs should be of help.

You might need to familiarize yourself with the library first, the tutorial should be quick and sufficient.

Finally, I'll try to give a minimal working sample soon.

[AnonymouX47]

mywidget = UrwidImage(img,format='RGB',upscale=False)

This won't work even if UrwidImage were properly imported because the argument types/values are wrong. The pages I linked in my previous comment should help with that.

Basically,

• the first argument must be an instance of an image class defined in term_image.image.
• The second is an image format specifier, which is shown in use here in the tutorial. For now, you may omit this argument, the default should be fine.
• The third argument determines if the source image may be upscaled when rendering.

[AnonymouX47]

urwid_sample.py:

import sys

import urwid
from PIL import Image

from term_image.image import AutoImage
from term_image.widget import UrwidImage

class MyLoop(urwid.MainLoop):
    def process_input(self, keys):
        if "window resize" in keys:
            UrwidImage.clear()
        return super().process_input(keys)

def handle_input(key):
    if key == "q":
        UrwidImage.clear()
        raise urwid.ExitMainLoop
    return False

img = Image.open(sys.argv[1])
image = AutoImage(img)
image_w = UrwidImage(image)

base_widget = urwid.LineBox(
    urwid.Pile(
        [
            urwid.Columns(
                [urwid.LineBox(image_w, "Image")] * 2
            )
        ] * 2 
    ),
    "Images",
)

loop = MyLoop(base_widget, unhandled_input=handle_input)
loop.run()

python urwid_sample.py logo.png

produces

on Kitty and

on Terminator (VTE-based).

EDIT: Updated the sample code to clear graphics-based images properly.

[danschwarz]

Thanks, I'll try it when I have some free time. I read your comment in the code:

`

If using an image widget with a :ref:graphics-based <graphics-based> render style as or within a flow widget, make sure to use a render method that splits images across lines such as the LINES render method for kitty and iterm2 styles. ` This is an important point, and makes future sixel support in the widget seem unlikely. I have a widget that renders images using sixel but haven't figured out a way to get it to output a row at a time without creating a sixel image-per-row, which is likely to slow down or break Xterm.

On Sat, Feb 4, 2023 at 1:28 PM Toluwaleke Ogundipe @.***> wrote:

urwid_sample.py:

import sys import urwidfrom PIL import Image from term_image.image import AutoImagefrom term_image.widget import UrwidImage def process_input(key): if key == "q": raise urwid.ExitMainLoop UrwidImage.clear()

return True

img = Image.open(sys.argv[1])image = AutoImage(img)image_w = UrwidImage(image) base_widget = urwid.LineBox( urwid.Pile( [ urwid.Columns( [urwid.LineBox(image_w, "Image")] 2 ) ] 2 ), "Images", ) loop = urwid.MainLoop(base_widget, unhandled_input=process_input)loop.run()

python urwid_sample.py logo.png

produces

[image: Screenshot_2023-02-04_19-26-29] https://user-images.githubusercontent.com/61663146/216783518-29ac8788-f69c-43ad-9e50-80642a600abf.png

on Kitty and

[image: Screenshot_2023-02-04_19-27-44] https://user-images.githubusercontent.com/61663146/216783545-ff083b1e-a8fb-46f2-8e22-7d9b524cb85b.png

on Terminator (VTE-based).

— Reply to this email directly, view it on GitHub https://github.com/AnonymouX47/term-image/pull/73#issuecomment-1416819765, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAY4FJWLBMZVGQS5JF4CRKDWV2NVXANCNFSM6AAAAAAURC4P4I . You are receiving this because you were mentioned.Message ID: @.***>

[AnonymouX47]

without creating a sixel image-per-row, which is likely to slow down or break Xterm.

I see. Unfortunately, that's the only way to achieve that :(

[AnonymouX47]

Please note that I just updated the sample code above to clear graphics-based images properly.

When images need to be cleared will majorly depend on your use case but one common case is a window resize.

[danschwarz]

Thanks for the extra documentation. With that, I was able to get the widget working properly in my test app. ANSI and Kitty images look good.

One anomaly I've found with the ANSI images - I have one that entirely filled my screen down to the lower right corner. You can see the [ character printed over the image. I don't know why that is appearing. This is an UrwidImage enclosed in an urwid.Filler container, nothing more. Displaying on Windows Terminal/Ubuntu 22.04.

This is repeatable with any image displayed as ANSI that is as large, or larger than the terminal width/height.

Another example, this is a NASA image

[danschwarz]

I know there's some weirdness with VT100 terminal emulation and the last cell in the lower right position - urwid raw_display.py _last_row(..) method does some extra work to set the last row, rightmost cell. Perhaps something there is interacting with the ANSI graphics output? Just a guess.

[AnonymouX47]

I looked into it and as you mentioned, it's due to urwid.raw_display.Screen._last_row().

I have implemented a workaround in https://github.com/AnonymouX47/term-image/commit/e8f89b26c8cbf0ed99f83e639e5292a699aa7ee4. Also, includes some comments explaining the problem.

The issue still affects, at least, the iterm2 style on Wezterm where the last line of the image is not drawn because technically, the image escape sequence is truncated/modified. The workaround resulted in a different problem (scrolling the screen) which seems to be worse IMO. So, I have excluded this case from the workaround. The best option is to avoid having an image fall at the bottom right cell.

Please, let me know how this works on your end, particularly on Windows Terminal.

Thanks for all your feedback. 🙏🏾

[AnonymouX47]

The major thing keeping this from being merged is unit tests, such a necessary evil 😡 but I'm almost done with them 😃.

[danschwarz]

What is the minimum python version you’re targeting for the widget? The app I’m contributing to has a min version of 3.6 but that might get bumped up in future as 3.6 is EOL.

On Sun, Feb 5, 2023 at 8:44 AM Toluwaleke Ogundipe @.***> wrote:

The major thing keeping this from being merged is unit tests, such a necessary evil 😡 but I'm almost done with them 😃.

— Reply to this email directly, view it on GitHub https://github.com/AnonymouX47/term-image/pull/73#issuecomment-1417881731, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAY4FJX4MXO3K5KDH26EMETWV6VC5ANCNFSM6AAAAAAURC4P4I . You are receiving this because you were mentioned.Message ID: @.***>

[AnonymouX47]

Python 3.7 is the minimum supported version for the entire library.

[AnonymouX47]

I think this might've been brought to a wrap at this point.

Please, let me know how this works on your end, particularly on Windows Terminal.

Any feedback on this?

Finally, before I merge, is there anything else you observed or would like to suggest?

Thanks.

[danschwarz]

I've done some more testing. I've integrated your widget into a custom version of the toot app.

Results:

1. The ] defect reported above is fixed (tested that against a separate app)
2. In Windows Terminal, ANSI graphics display is good. Up/down scrolling works correctly, which means you're honoring trim_top values correctly in your canvas.

3. Unfortunately it doesn't look like you're honoring trim_left values, so overlays on top of ANSI images cause the image to get corrupted.

I have an example of how to do this in my code, see the truncate_ansi_safe method in ansiwidget.py

4. In Kitty, images initially load correctly

But when you scroll down, images get corrupted when they hit the top of the screen. See the author avatar (compare to above image). Looks like trim_top isn't being honored correctly.

[danschwarz]

Kitty images don't respect trim_left either.

Original:

With overlay:

[danschwarz]

I have some code that deals with trim_top and trim_left for pixel images, see sixelwidget.py. I'm not sure if it would be useful for kitty or iterm images. The sixel code is mostly working - the big problem with it is that the image is rendered all at once, which means it can only honor a single trim_left value across all rows.

[danschwarz]

OK, for completeness, here's iTerm2. This looks like the best of all the pixel formats right now. Up/down scrolling is perfect.

Unfortunately we see the same problems with trim_left when overlays are displayed

iTerm2 does a good job of erasing images when no longer needed, i.e., when you quit the app and restart, images from the previous run are not visible onscreen. Contrast to kitty which requires explicit clearing of images. For this app, I think there may need to be some hook when the UrwidImage widget is destroyed, all images associated with it should be cleared. Putting that logic in the widget will be.a great convenience for app developers :)

[danschwarz]

And finally, this is not a bug, but it might be a feature request? You made no claim to support less than true color ANSI rendering, and that's fine. Unfortunately the default terminal app on Mac only supports 256 colors. Most savvy Mac owners have upgraded, but in order to support the default app, I've added explicit support for 256 color rendering in my builds of toot, controlled by a --256 flag (although in theory it should be possible to detect true color vs 256 color vs lesser support). If you're interested in adding that support, what I did was to use Pillow to create.a reference image using the Xterm 256-color palette, then quantized the displayable image to use that palette. Works fine to get usable images on the Apple Terminal.

[AnonymouX47]

The ] defect reported above is fixed (tested that against a separate app)

Good

In Windows Terminal, ANSI graphics display is good.

Good

---

Unfortunately it doesn't look like you're honoring trim_left

Yeah... Sorry, I was aware of that and should've mentioned it.

I've thought about this and here are some results:

• I have conceived a way to implement this functionality generally for all render styles but it will require a whole new feature addition to the library's core i.e rendering trimmed images, a feature I had no plans for before now.
• With a few modifications to the block (ANSI) render style, i can implement trim_left but I wouldn't want to do this now and scrap it later. Also, the approach feels very "hacky" and it will only solve the issue for just one render style, so I'll have to create a special case for that and I really don't like doing such.

With all that said... IMO overlaying widgets on images is not a good idea because trimming is really costly. The reason being that it results in [potentially multiple but at least one] re-render(s) of [different] parts an image at draw-time (i.e when canvases are being drawn to the screen), as opposed to render-time.

This renders canvas caching almost completely useless. If caching were to be implemented, it would be at the level of each canvas, which will also be really complicated because a single screen draw may require multiple pieces of the image canvas. All the pieces have to be cached and a single shift in the overlaying widget or the image widget invalidates at least one of the pieces.

Without caching, it would be an even bigger problem IMO. Imagine a widget laid over and within an image widget i.e concentric, that would require 4 separate pieces or 4 separate trimmed renders of the image... all at draw-time.

All these might seem inconsequential when displaying only a single image at time (though, that even depends on the machine on which the program is running and the nature the image) but with a couple more images, the difference will be substantial, even on a decently performing machine.

---

In Kitty, images initially load correctly... but when you scroll down, images get corrupted when they hit the top of the screen

That actually not what's happening. One key property of the kitty graphics protocol is that image pixels don't get overwritten/erased by text, as opposed to ANSI images (which are themselves composed of text) or the iTerm2 inline image protocol (except on Konsole).

For this reason, the UrwidImage class has a clear() method. Where and when this method should be called depends on the flow of the UI, typically whenever any image on the screen shifts but there is always one common case, a window resize.

Not to worry, the method also includes a workaround to ensure urwid redraws images or lines of an image that didn't change.

Beware though, clearing images on every redraw can result in some sort of perceived flickering. So, one has to try as much as possible to clear only when neccessary.

NB: Sorry my reply took a while. I started typing this about 13 hours ago but due to the length and other things I had to attend to... :\

[AnonymouX47]

Concerning honouring trim_left, there are two options IMO:

1. Modify to render blanks (spaces) when trim_left != 0 and/or cols != None, add a notice about that, merge and include in next release.
2. Do not merge, make the next release, implement image trimming, update the widget & canvas and include in the release after the next. (my choice)

Which would you prefer?

[AnonymouX47]

I have an example of how to do this in my code, see the truncate_ansi_safe method in ansiwidget.py

Checked it out... but it won't be of much help. If you look into BlockImage._render_image(), you can see my approach is quite more complex, it handles transparency amongst other things. Though, with some modification, I can make, horizontal trimming work.

Please. note that I mentioned this earlier but more importantly, my comments about it.

I have some code that deals with trim_top and trim_left for pixel images, see sixelwidget.py.

Yeah, quite similar to what I envisioned for the trimmed image rendering feature I mentioned earlier. It should also work for trim_top and both combined.

the big problem with it is that the image is rendered all at once, which means it can only honor a single trim_left value across all rows.

I don't quite understand this though.

[AnonymouX47]

iTerm2 does a good job of erasing images when no longer needed

Yeah, it's how the protocol was designed to work. Though, I should note that Konsole's implementation doesn't work the same way, it works like kitty's, due the the fact that both protocols were implemented using the same backend (see here).

when you quit the app and restart, images from the previous run are not visible onscreen

Calling UrwidImage.clear() upon exit of the TUI fixes that.

I think there may need to be some hook when the UrwidImage widget is destroyed, all images associated with it should be cleared.

This might be achievable for kitty (but not for iterm2 on Konsole) using IDs but I have to say, it'll be pretty tasking to accomplish. Also, I'm yet to incorporate the use of kitty's id parameter but it's in the plan (see #40).

[AnonymouX47]

You made no claim to support less than true color ANSI rendering, and that's fine.

Yeah, this amongst a number of others (i.e "render styles and methods" in particular) is part of my major focus for version 0.7. Take a look at #61 😃.

Actually, if not your request for the widget, I should've released 0.6 by now and probably started working towards this feature :grinning:.

what I did was to use Pillow to create.a reference image using the Xterm 256-color palette, then quantized the displayable image to use that palette

I see. Thanks for the tip. What I've had in mind (as seen in #61) is quite similar to this but just uses math :) as I believe the standard Xterm 265 palette has a pattern/algorithm to convert colors from RGB being that the palette basically defines a 6x6x6 cube (i.e aside the ANSI 16 color palette and the 24 color greyscale)... I'm yet to deeply investigate this but the last time I checked, my calculations seemed to be correct.

I'll keep your approach in mind, as it might be more efficient.

Side note: The amount of features I have conceived for this project is way more than I can implement in a rush and being sort of a perfectionist make things even more interesting. What's shown in the list of planned features and in the few issues on here is not up to a tenth of what I have put down in my notes and they keep getting more everyday. 😓

---

I really appreciate all your feedback and suggestions. I'm more than grateful. 🙏🏾 🙇🏾‍♂️

[danschwarz]

Thank you for the detailed responses. Lots to respond to.

Re: left-trim and overlapping images.

My use case at the moment is adding image support to the existing toot TUI. The TUI relies on overlays for key functionality (Compose, Reply, show Author details...). No one is really looking at the image while they have an overlay up, so it could disappear, but it'd be more like a traditional GUI for it to get clipped and show in the background.

I think that in the case of a generic Urwid widget, designed for reuse, robustness > performance. Rendering blanks when trim_left > 0 is better than rendering improperly.

I'm in no rush for the widget so any additional features can wait.

re: 256 color palette support - you're right, math will work; this code claims to do it. Performance? Unknown.

Thanks!

[AnonymouX47]

You're welcome.

No one is really looking at the image while they have an overlay up, so it could disappear

Yeah, that's exactly what I did in my TUI image viewer.

in the case of a generic Urwid widget, designed for reuse, robustness > performance

Very true. I intend to implement as many features as reasonably possible and allow users to decide on the balance between functionality and performance.

Rendering blanks when trim_left > 0 is better than rendering improperly.

I will do this for now but only for the graphics-based styles.

I reconsidered what I said about implementing this for ANSI images only. Since the approach is applicable to any text-based style and does not require re-rendering, it'll even be a more performant approach. Even when I implement trimmed rendering, I intend to still use this for text-based render styles. Though it won't be exposed in the public API or documented publicly, it'll only be used internally.

I'm in no rush for the widget so any additional features can wait.

To be sure I get what you mean here, are you saying the release of the widget can wait for the additional features (particularly trim_left) or the widget can be released now and additional features come later?

Performance? Unknown.

Noted. I will do some profiling/comparison when the time comes.

Thanks so much.

[danschwarz]

Re:

To be sure I get what you mean here, are you saying the release of the widget can wait for the additional >features (particularly trim_left) or the widget can be released now and additional features come later?

I think it's good to release a working widget with image blanking on trim_left > 0 , then add any extra functionality in a future release. But do what you think is best - I can't be sure that we'll be able to integrate the widget into the toot app right away - I'm a contributor, but @ihabunek makes the final decision, and there are python min version differences and other things to sort out. But ultimately I do want to get this widget integrated in toot (and elsewhere).

Right now we have iterm and kitty and other terminal emulators implementing some form of image support, but there are few apps that make use of those features, other than imgcat and notcurses and of course your term-image. More useful apps will drive demand, which will push terminal emulator authors to implement support.

Thanks again for all your work on this.

[AnonymouX47]

Hi!

I actually read you reply yesterday but I had already started working on trim_left (for the ANSI images). So, I though I would just reply along with the results but it the implementation has turned to not be as trivial as I had imagined... got a couple of edge cases to take care of.

I'll go with your opinion i.e releasing it now with blanking if trim_left != 0 or cols is not None (but only for the graphics-based styles).

As soon as I'm done clearing out the edge cases, I'll let you know.

More useful apps will drive demand, which will push terminal emulator authors to implement support.

Yeah, really hoping for this and I have some other plans underway to foster creation of apps to drive demand.

Thanks for your all suggestions and help.

[AnonymouX47]

Below are a few images showing various cases of vertical and horizontal trimming...

For text-based styles, horizontal trimming is fully (at least, for all the cases I could think of) implemented:

---

For graphics-based images, blanks are emitted whenever horizontal trimming is required:

---

Writing tests for these might take a while though. 🥲

[danschwarz]

Thanks - those test cases eem comprehensive. I've pulled the latest version and integrated with the toot app. No code changes on my side. ANSI images scroll and crop perfectly. Kitty images don't clear - as you noted, there's an UrwidImage.clear() method. I can keep track of my UrwidImages and call clear() before program exit, but it's the scrolling use case I'm concerned with right now. When an UrwidImage widget scrolls up off screen one row at a time, who should be responsible for calling UrwidImage.clear() and when? I suggest that the UrwidImage should be responsible for calling clear() itself when needed - your concern about flicker is warranted - perhaps it should be a flag sent at UrwidImage creation - auto-clear image before returning content when trim_top > 0, or not?

[AnonymouX47]

I can keep track of my UrwidImages and call clear() before program exit

I should mention that, at least for now, the clear() method doesn't work per image, it's simply a static method (i.e has no implicit reference to the class or any instance) that clears all images (only for kitty images, on any terminal and iterm2 images, on Konsole) currently on-screen.

but it's the scrolling use case I'm concerned with right now

Yes, this is actually the most complex case that requires clearing images.

When an UrwidImage widget scrolls up off screen one row at a time

The problem is, this is not the only case of scrolling. An image can be scrolled while still remaining fully in view, consider a ListBox([Pile([UrwidImage, ...])]), even a ListBox([UrwidImage, ...]) can show this but not as much.

Also, an image can remain in the same position and still require clearing. For instance, given a Columns widget containing an image amongst other widgets. If any line of any of the other widget changes, that entire screen line is redrawn, not just the columns of the widget that changed. On the kitty terminal emulator, this causes the new image to be drawn over the old image (i.e blends the images) and this poses two major problems:

• Transparent images are messed up, since alpha will be combined.
• Over time, the terminal emulator begins to slow down in performance as more and more images are overlaid. I have tested and experienced this while developing the TUI image viewer.

Now, the major problem with an image widget clearing its images is that it has no way to know when the image should be cleared in most cases. In the case of scrolling (considering the image may not be trimmed), the widget is not aware of its position on-screen. Only a widget that contain the image widgets can know definitively when images within it should be cleared (e.g See here in GridListBox).

The way I approached this issue in the image viewer was to find every possible case in which images required clearing and called the appropriate method one by one for every case.

A one-size-fits-all solution would be to clear all images on every screen redraw but that is outside the scope of a widget or canvas and as I mentioned earlier, would cause flickering.

---

As for clearing images per widget, I've given it some further thought and realized the "image ID" parameter i mentioned earlier can't work. One workable approach I've thought of will be using kitty's "z-index" parameter such that each widget draws all of it's own images on a separate z-index but this would imply that:

• a clear would affect all occurrences of that same widget and any canvas it rendered i.e if the widget is visible in multiple places on the screen, though this isn't much of a problem.
• users of the widget will not be allowed to control the z-index parameter which is normally allowed for a kitty-style image (i.e with KittyImage).
• it introduces the problem of allocating and deallocating z-indexes efficiently.

Also, there is a limited number of z-indexes, specifically 2 ** 32. Even though it's really wide range, it's still finite. I might just raise an exception if no z-index is available.

Finally, this is not applicable to iterm2 images on Konsole, which also require clearing but I guess that can be avoided simply by using kitty images on Konsole instead.

---

I will really appreciate any workable ideas you have about the "auto clear" (i.e clearing being handled by the image widgets themselves). I would also like to have this feature but I just don't see it feasible yet.

I really hope I didn't omit anything I wanted to write cos I just wrote the paragraphs as they popped into my head, not necessarily in order of appearance. Also, I'm sorry if my comments are usually too long.

Thanks

[danschwarz]

Hmm.

Now, the major problem with an image widget clearing its images is that it has no way to know when the image should be cleared in most cases. In the case of scrolling (considering the image may not be trimmed), the widget is not aware of its position on-screen. Only a widget that contain the image widgets can know definitively when images within it should be cleared (e.g See here in GridListBox).

You have a deeper understanding of the problem than I do. I read the code you linked - I'll need to read a lot more code to get the context, I'll do that when I have more time.

re: cleanup of images at program exit time. Perrhaps there needs to be some kind of ImageManager class that tracks the images and clears them when needed. some kind of ImageManager.clear_all() method could be invoked just before an app raises urwid.ExitMainLoop(). Just a thought.

[AnonymouX47]

I'll need to read a lot more code to get the context, I'll do that when I have more time.

Ok, cool.

Perrhaps there needs to be some kind of ImageManager class that tracks the images and clears them when needed.

Even though I don't yet see how this could help with "clearing images at program exit", it might actually work for "clearing images when needed" and I'm currently investigating it. Along the line of a wrapper widget that inspects the canvas (potentially a CompositeCanvas) rendered by the wrapped widget to detect changes in size, position and trim of image canvases.

some kind of ImageManager.clear_all() method

Umm... UrwidImage.clear() already does that, though I think I'll be renaming it to clear_all() since I plan to implement per-image clearing.

I'll prefer to make the UrwidImageManager (tentative name) widget perform only implicit operations.

[AnonymouX47]

Perrhaps there needs to be some kind of ImageManager class that tracks the images and clears them when needed.

Good news!

I'm done implementing this and it seems to be working just fine on my end. Just some cleanup/organization to do... I'll commit the changes later today and you can test it.

Thanks so much for your suggestions.

[AnonymouX47]

By the way, what do you think about the class being named UrwidImageClearer?

I don't want to use UrwidImageManager as I feel that's too "broad", at least for now when all it does is clear images.

I'm really not good at naming things :man_facepalming:

[danschwarz]

I'll check it out. The name is fine, or you could consider "UrwidImageJanitor" or something like that.

[AnonymouX47]

Sorry it's coming in later than I said 🥲, even though it's still the same day (at least, on my end) 😁.

[danschwarz]

Terrific! I'm integrating the widget into the latest toot build.

After I wrapped the main app widget class with UrwidImageJanitor, kitty images scroll properly.

Clipping works as expected with overlays

Image display downgrades to ANSI automatically on Windows Terminal

Can't test with iTerm2 until I get in front of my Mac next week.

[danschwarz]

Brilliant work.

[ihabunek]

Sorry it's coming in later than I said smiling_face_with_tear, even though it's still the same day (at least, on my end) grin.

You are so much more responsive than I usually am. :D Thanks a lot for your work!

[AnonymouX47]

Can't test with iTerm2 until I get in front of my Mac next week.

Ok, good. If I can get my hands on one, I'll test it out. It's actually been a really long while since I tested anything on iTerm2.

Though, I should note that the janitor is not required for iTerm2 inline images as they're are overwritten by text, except on Konsole because it uses the same backend as it does for the kitty graphics protocol. I'm not even sure if I should report this as a bug to Konsole developers.

Anyways, the Janitor also takes care of the case of iterm2 on Konsole but it can't clear images per widget, it has to clear all images if a single one changes position/trim. So, it flickers a lot more with multiple images on-screen. All-in-all, one is better-off using kitty images on Konsole instead (made some changes here concerning this).

Brilliant work.

Most of these wouldn't even have happened (at least, not anytime soon) without your suggestions, help and persistence.

Thanks so much 🙏🏾.

[AnonymouX47]

@ihabunek You're welcome 🤗.

[danschwarz]

Wezterm is handling images well with the widget. It advertises itself as iterm2 image format compatible on its website, but the kitty site says it's kitty compatible, so I'm not sure which protocol is actually being used?

Image scrolling works but is just as flickery as kitty image scrolling, so I suspect it's getting kitty protocol?

[danschwarz]

My image11 branch of toot integrates the latest term-image widget and was used to generate the latest screenshot in this thread.

Question: What's the right way to know what render style is currently in use? Interrogate AutoImage.auto_style()? The reason I need to know this is that I want to use more rows to display images rendered as Unicode half-blocks. They're more legible at larger sizes. But with any other rendering style, I can use fewer rows, for smaller images.

[danschwarz]

oh, one more question:

The following code is my attempt to clean up all remaining images onscreen before exiting the app ('Q' key)

UrwidImage.clear_all() works fine elsewhere in the code, but in this case, the image is not cleared before the program exits. If a kitty image was onscreen, the next time the Alternate Screen is displayed, the image reappears. Any idea what's going on there?

[AnonymouX47]

It advertises itself as iterm2 image format compatible on its website, but the kitty site says it's kitty compatible, so I'm not sure which protocol is actually being used?

It's using the iterm2 protocol. The current state of support for the Kitty graphics protocol is still quite unusable (see https://github.com/wez/wezterm/issues/986#issuecomment-1120622597)

As for flickering while scrolling, I don't think it should be as much as kitty's 🤔 based on the fact that images get overwritten by text and from my experience testing on Wezterm. Images are not entirely cleared before redraw, it should clear and redraw line by line.

Interrogate AutoImage.auto_style()?

Exactly. Though, it's term_image.image.auto_style().

UrwidImage.clear_all() works fine elsewhere in the code, but in this case, the image is not cleared before the program exits.

Seems as though the stdout buffer is not getting flushed before exiting the alternate buffer. Though, that's highly unexpected (or impossible) as exiting the alternate buffer itself requires flushing the output after writing the required escape sequence 🤔. I have used this with multiple test TUIs and it has always worked properly.

Anyways, to handle such situations, I'll incorporate the now parameter which I just recently added in https://github.com/AnonymouX47/term-image/commit/97eceab77e7448a18281aa6edb3fa8ec9e6564c5 (for just about the same reason) into UrwidImage.clear_all().

[AnonymouX47]

Another thing I should note at this point is that I haven't figured out a way to clear images upon a crash i.e when an unhandled exception is raised within MainLoop.run(). The closest solution to this is to clear images upon startup of the TUI i.e after switching to alternate buffer. I've tried a few ideas:

1. The user overrides Screen.start() and clear all images after calling super().start().
   • This requires either flushing the output buffer before clearing the images OR manually switching to the alternate buffer and clearing the images (and possibly switching back to the normal buffer) all by writing directly to the TTY.
2. Let it be handled by UrwidImagejanitor by clearing all images upon its first render.
   • I would prefer this in order not to bother users with No. 1 but there are a couple of issues that depend entirely on the user.
   • What if, due to the user's actions, the first render of widget does not occur immediately after the urwid screen startup i.e either before Screen.start() is called or later after urwid has switched to the alternate buffer and drawn the UI?
   • One way to solve the problem above is to make the clear at first render optional and disabled by default.

What do you see to this?

[AnonymouX47]

My image11 branch of toot integrates the latest term-image widget.

As a reminder, when adding term_image as a dependency, please remember to pin it to a specific minor version, until version 1.0 is released. See here.