Support for tmux?

[Mickychen00]

Hi, I am using tmux. The left pane is my vim pane writing python, the right is ipython pane. When I use term-image's image.show() funtion. A very vague image printed in the right pane. how to make it more sharp? thx

[Mickychen00]

Hi when I use term-image command to open a image file in iTerm2, a high-resolution image appears. But when I type tmux into a tmux session, the image turns into vague. Why? Is there any solution? thx

[AnonymouX47]

Hello! Thanks for the suggestion.

A very vague image

Well... that's because the image is represented using Unicode blocks with ANSI color escape sequences. In other words, what you're seeing is basically colored text.

This is called the "block" render style within the context of this project.

how to make it more sharp?

Well... Tmux itself doesn't support any graphics protocol, cos it just technically can't but there might be a way forward, I'll get to that in a moment.

But when I type tmux into a tmux session, the image turns into vague. Why?

The iTerm2 inline image protocol is implemented by the iTerm2 terminal emulator (the GUI program whose icon shows on your dock).

Tmux is a terminal multiplexer... basically, that means a program that provides and manages multiple terminal sessions within another terminal emulator.

Tmux intercepts escape sequences interpretes them and decides if and how they should be passed on to the actual terminal emulator.

Now, the problem is Tmux doesn't implement/support the image protocol i.e it can not understand the escape sequences used by the image protocol.

By default, term_image automatically detects the best image render style (or protocol) supported within the environment it is running. So, since the iTerm2 protocol is not supported within Tmux, it uses the block render style which is supported within Tmux.

Is there any solution?

Well... more of a workaround. Tmux provides a means by which escape sequences can be passed directly to the terminal emulator... called the Tmux passthrough escape sequence.

I've known about this for a while and had it in mind to add support for this but I've been reluctant to do so because it's a "workaround" meaning many things won't work as expected... e.g images won't scroll along with text as they normally would within the actual terminal emulator, amongst other issues.

The best option you have (and my advice) is to use use the terminal emulator directly and I'll recommend Kitty, it probably has most of the features of Tmux you need.

---

Please note that I'm not promising support for Tmux passthrough, I'm still evaluating the situation and at the moment, the cons outweigh the pros.

Thank you!

[Mickychen00]

Hi when I use term-image command to open a image file in iTerm2, a high-resolution image appears. But when I type tmux into a tmux session, the image turns into vague. Why? Is there any solution? thx

Hi! Thanks a lot for your very detailed reply and instruction. After one day's work, I turn from tmux to kitty. And indeed the image appears very clear (high-resolution)! Thx!!! However, I found term-image is the only tool which can show image in tmux pane to my best knowledge despite its vague feature. But if the support is available in future version of term-image. I think it's very promising tool for wide python coders using tmux.

[AnonymouX47]

You're welcome :smiley:

I'll look more into and actually test it out soon i.e support for Tmux passthrough... I'll keep this issue open till then.

If I do add support at some point in the future, I'll simply note the quirks and maybe add some tips on how to best use it without much issues.

Once again, thanks for the suggestion.

[AnonymouX47]

By the way, I'll add support for matplotlib backend and IPython magic soon.

I just thought I should let you know cos they might be useful to you. :smiley:

[Mickychen00]

Hi thanks for your sincere help! I am very delighting to see these new features! I really like term-image, an awesome tool! BTW, is there a solution for term-image to draw the object of matplotlib figure? like

from term-image.image import from_mpl # ideal function
fig = plt.figure()
ax = fig.subplot()
ax.plot()

image = from_mpl(fig)
image.draw()

I know I can use fig.savefig() to a file, a use from file() to read it back and then draw this image() But I think this feature may be more convenient for users.

[AnonymouX47]

I personally don't use matplotlib but I'll look around and get back to you.

In the mean time, you can simply save the image to a buffer instead of a file i.e

import io
from PIL import Image
from term_image.image import AutoImage

buf = io.BytesIO()
fig.savefig(buf)
image = AutoImage(Image.open(buf))

I don't think there'll be any significant performance difference but it'll help avoid writing to disk.

[Mickychen00]

from term_image.image import AutoImage
fig.canvas.draw() 
img = Image.frombytes('RGB',fig.canvas.get_width_height(),fig.canvas.tostring_rgb())
image = AutoImage(img)
img.draw()

Hi thank your for your reply. I found a solution that seems to be much faster than savefig as the answer said (2ms vs 35~40ms)
See: This link

[AnonymouX47]

Yeah, thanks.

Beware though... only the AGG backend(s) implement(s) the tostring_rgb() method, so you might need to create a new canvas first i.e

from matplotlib.backends.backend_agg import FigureCanvasAgg
canvas = fig.canvas if isinstance(fig.canvas, FigureCanvasAgg) else FigureCanvasAgg(fig)

After spending some time going through the package's docs (and a little through the source code) yesterday... I found FigureCanvasAgg.buffer_rgba() to be the best option (based on the implementation).

I had not yet mentioned it because I'm yet to test it but I plan to use this in implementing a from_figure() helper method later.

[AnonymouX47]

Now, back to the issue at hand...

After reading through https://gitlab.com/gnachman/iterm2/issues/3898 with some experimentation and thought... I've come to conclude that tmux support for any graphics-based style just isn't workable.

One could make the image display, no much problem with that... but due to the fact that tmux is in no anyway aware of the image or it's size, the whole thing becomes literally useless and in some cases even problematic.

See also: https://github.com/kovidgoyal/kitty/issues/2457

A few amongst the problems I've realised are:

1. For graphics protocols wherein image cells are overwritten by text (e.g iTerm2's), any screen redraw or update by tmux will make the image disappear.
2. For graphics protocols wherein image cells are not overwritten by text (e.g Kitty's), more output or scrolling in tmux will either make an image cover up text or make text overlay an image.
3. Images will never scroll along with tmux' output.
4. In a multi-window setup, images wider or taller than the window in which they were drawn can extend into neighbouring windows.

---

In short... Terminal multiplexers are a pain in the ass... if you need terminal graphics, simply use the terminal emulators that support them! :(

Thank you! :smiley:

[Mickychen00]

Thanks a lot for your valuable advice! I will listen to you!