Feature Request: sixel graphics mode

BrainDamage commented 8 years ago

https://en.wikipedia.org/wiki/Sixel is a raster protocol format part of VT200 and VT300 serial terminals it recently started gaining traction on command line tools to output images directly to the terminal because it works trought ssh, works reasonably well, etc for an implementation detail to copy the encoder/decoder, one can look at https://github.com/saitoha/libsixel , the url documents also a series of terminals and programs that support the standard I realize it's a low priority feature, but it'd allow rather nice semi graphical workflow under termux's terminal, currently it's already possible to view sixel graphics from launching a ssh daemon and connecting externally with a sixel capable terminal emulator

EralpCelebi commented 6 years ago

İs there any development ? I would really like to use this thing.

ghost commented 6 years ago

This should be good. But can you provide an example of software that supports sixel graphics ? I know a termplay program (https://github.com/jD91mZM2/termplay) that can play image/video on terminal but what else ?

its-pointless commented 6 years ago

Gnuplot does I think w3m can or one textmode brower can display sixel images

beiriannydd commented 6 years ago

https://devhub.io/repos/saitoha-libsixel says it better than I can. I would really appreciate this feature.

petrmachcz commented 6 years ago

Is it supported now?

Momiro commented 5 years ago

Want.

hackerb9 commented 5 years ago

I'm the author of lsix which is like ls but shows images in the terminal using SIXEL. I have received a request for Termux support. I believe that my program should work fine with Termux once it supports SIXEL graphics.

ghost commented 5 years ago

Neofetch supports sixel, instead of distro art, you can use an image.

https://github.com/dylanaraps/neofetch/wiki/Image-Backends#sixel

cogburnd02 commented 4 years ago

I would love for this to be implemented.

Perhaps some code can be ported over from yaft-android?? (MIT licensed)

https://github.com/uobikiemukot/yaft-android

Efreak commented 2 years ago

I'm quite interested in using this with neofetch, lsix. This could also be used for previewing formatted documents directly in the terminal.

cogburnd02 commented 2 years ago

@Efreak RIGHT? A version of fbgs (https://github.com/kraxel/fbida/blob/master/fbgs) modified to use img2sixel instead of fbi would be great for viewing ps and pdf files such as those produced by groff or TeX and wa (https://developer.run/37) could make use of img2sixel to show results from Wolfram Alpha right in the command line!

evert-mouw commented 2 years ago

Sixel support would be very nice.

nonetrix commented 2 years ago

Would like the idea of being able to use w3m browser with images since it uses sixel if you run it with -sixel

evert-mouw commented 2 years ago

Konsole now supports Sixel. Increasing support in many places.

https://invent.kde.org/utilities/konsole/-/merge_requests/594

cogburnd02 commented 2 years ago

Meant to comment this before, but I suppose 'better late than never,' right?

I put up a bounty for this on bountysource: https://app.bountysource.com/issues/36956941-feature-request-sixel-graphics-mode

FoxieFlakey commented 2 years ago

how the progress of sixel support?

MatanZ commented 2 years ago

A brach with Sixel support is here: https://github.com/MatanZ/termux-app/commits/sixel

I will not post a pull request for this, as I use Termux on Android 5, so I use an older branch.

This works, but should still be considered more proof of concept. Some known problems:

limited to 16 images up to 1024x1024 each.
Inefficient in both memory and CPU.
Little error handling, so a malformed sixel sequence may leave the terminal in an unknown state.
There may be memory leaks - I do not know java, so it is likely that I did something wrong with allocations.

I do believe that someone with a little experience in Android programming can get this code to the quality needed for merging without a lot of difficulties.

agnostic-apollo commented 2 years ago

It can be looked into in future. Also master branch supports android 5/6.

https://github.com/termux/termux-app/wiki/Termux-on-android-5-or-6

MatanZ commented 2 years ago

I missed that. Anyway, here's a branch of the current master. A few problems fixed, but still needs work: https://github.com/MatanZ/termux-app/commits/sixel2

Grimler91 commented 2 years ago

@MatanZ this is awesome, thanks for working on this!

Sneakpeek for everyone:

sixel2

I had to apply a small patch to build the sixel2 branch:

diff --git a/terminal-emulator/src/main/java/com/termux/terminal/TerminalEmulator.java b/terminal-emulator/src/main/java/com/termux/terminal/TerminalEmulator.java
index 9eea39655828..01180b95237d 100644
--- a/terminal-emulator/src/main/java/com/termux/terminal/TerminalEmulator.java
+++ b/terminal-emulator/src/main/java/com/termux/terminal/TerminalEmulator.java
@@ -3,6 +3,7 @@ package com.termux.terminal;
 import android.util.Base64;

 import java.nio.charset.StandardCharsets;
+import java.util.ArrayList;
 import java.util.Arrays;
 import java.util.Locale;
 import java.util.Objects;
@@ -2229,7 +2230,7 @@ public final class TerminalEmulator {
                         mOSCOrDeviceControlArgs.setLength(ESC_OSC_colon);
                     }
                     if (ESC_OSC_colon >= 0) {
-                        Log.e("1337", "len = " + ESC_OSC_data.size());
+                        logError("OSC1337 len = " + ESC_OSC_data.size());
                         byte[] result = new byte[ESC_OSC_data.size()];
                         for(int i = 0; i < ESC_OSC_data.size(); i++) {
                             result[i] = ESC_OSC_data.get(i).byteValue();
@@ -2240,7 +2241,7 @@ public final class TerminalEmulator {
                         }
                         ESC_OSC_data.clear();
                     } else {
-                        Log.e("1337", "no data");
+                        logError("OSC1337: no data");
                     }
                 } else if (textParameter.startsWith("ReportCellSize")) {
                     mSession.write(String.format(Locale.US, "\0331337;ReportCellSize=%d;%d\007", cellH, cellW));

FoxieFlakey commented 2 years ago

Screenshot_20220903-231638

maybe dont try play gif yet :3 (this is that one gif in libsixel repo)

agnostic-apollo commented 2 years ago

Wow, that looks cool. Once bugs are sorted out, let me know and can try merging it with rest of the pull requests.

MatanZ commented 2 years ago

So, I learned a bit of Java, and I am a bit closer to submitting a PR. An ew branch, due to some playing with git an gradle: https://github.com/MatanZ/termux-app/tree/sixel4

I see that that specific GIF does not work in Konsole (or xterm, for that matter), and neither with libsixel or ImageMagick converter. Looking at the sixels themselves, I do not see why.

MatanZ commented 2 years ago

The problem is actually in the file. I corrected it using something like this: https://gist.github.com/BigglesZX/4016539?permalink_comment_id=3202586 and it works.

FoxieFlakey commented 2 years ago

offtopic, why creating new branch everytime instead commit to existing one?

Grimler91 commented 2 years ago

This gif from wikipedia renders without issues in Konsole (as far as I can tell), but has similar artifacts in termux. Fixing it with python code as suggested above:

import numpy as np
from PIL import Image, ImageSequence
from array2gif import write_gif

img = Image.open('Loituma.gif')
frames = np.array([np.array(frame.copy().convert('RGB').getdata(),dtype=np.uint8).reshape(frame.size[1],frame.size[0],3) for frame in ImageSequence.Iterator(img)]) 

write_gif(frames, 'Loituma-fixed.gif', fps=5)

makes it render fine in termux as well, compare the two gifs shown at the beginning (img2sixel Loituma.gif) and end (img2sixel Loituma-fixed.gif) of this video:

https://user-images.githubusercontent.com/18900601/188305141-921d40fe-a947-490d-b778-1f8d2f96628d.mp4

edit: tested on the sixel4 branch

MatanZ commented 2 years ago

This gif does not work for me in konsole (with both img2sixel and convert), showing similar artefacts, but less severe.

Are you sure you used the exact same sixel encoder in konsole and termux? It might depend on the version of the encoder or any of its libraries. It is best to convert the image to a sixel file and cat the same file in both emulators, or ssh from termux to the system that runs konsole.

Grimler91 commented 2 years ago

Are you sure you used the exact same sixel encoder in konsole and termux? It might depend on the version of the encoder or any of its libraries.

Probably not, I just tested to img2sixel the gif in archlinux's vanilla Konsole package, and with its builtin encoder it seems to work

defencedog commented 2 years ago

I want to use matplotlib.use('module://sixel') https://github.com/jonathf/matplotlib-sixel

cogburnd02 commented 2 years ago

I want to use matplotlib.use('module://sixel') https://github.com/jonathf/matplotlib-sixel

If you're interested in math stuff with sixel, there's a version of gnuplot that can do sixel graphics here: https://github.com/csdvrx/sixel-gnuplot and a script to pretty-print equations in latex format here: https://github.com/nilqed/sixelSPAD/blob/master/bin/latex2sixel and a preview/screenshot of it here: https://raw.githubusercontent.com/saitoha/libsixel/data/data/latex2sixel.jpg

defencedog commented 2 years ago

@cogburnd02 thanks, but insofar this isnt implemented in termux?

cogburnd02 commented 2 years ago

@cogburnd02 thanks, but insofar this isnt implemented in termux?

Not as of yet, but MatanZ is working on a branch (I think that's what it's called) that does support it. There's a pull request here: https://github.com/termux/termux-app/pull/2973

ashley-graves commented 1 year ago

7 years later, any updates on if/when this will get added?

sylirre commented 1 year ago

Next release probably.

agnostic-apollo commented 1 year ago

Yeah, planned for next release.

AnonymouX47 commented 1 year ago

Any ETA for that?

agnostic-apollo commented 1 year ago

My ETAs end up being wrong mostly, but mainly 2-3 items left on todo list before I push my changes, will merge pulls after that, so hopefully end of month for release, but then again, could be delayed.

AnonymouX47 commented 1 year ago

Okay, thanks. The uncertainty is totally understandable by the way.

FoxieFlakey commented 1 year ago

curious about the bounty, has @MatanZ claimed it?

AnonymouX47 commented 1 year ago

Please, does Termux currently support any standard/common means of identification and version introspection?... Such as:

CSI > q (preferably)
TERM_PROGRAM, TERM_PROGRAM_VERSION environment variables.
- Note that these are rendered useless when running a terminal multiplexer like tmux which overrides them.

If not, this will be required for the iTerm2 protocol as it provides no means for detecting support.

Thanks.

MatanZ commented 1 year ago

I don't like this method for feature detection. It means each application needs to maintain a database of terminals and features they support, and this will, generally, lag actual support.

The way to detect iterm2 graphics support should be using ReportCellSize sequence here https://iterm2.com/documentation-escape-codes.html . A terminal emulator that supports iterm2 image protocol should support this. konsole and termux (with my PR) do.

See example here: https://github.com/dotcs/xkcd-cli/blob/919e8cc4c41f9bab29934cbc6b9756c1315a0d59/xkcd_cli/iv.py#L410

AnonymouX47 commented 1 year ago

It means each application needs to maintain a database of terminals and features they support, and this will, generally, lag actual support.

True.

A terminal emulator that supports iterm2 image protocol should support this.

Yes, but the problem with this is the "should". The cell size escape sequence is separate from the image protocol itself, which also doesn't specify that the cell size query be implemented by a terminal emulator implementing the image protocol (which could happen since the terminal could already have other means of querying cell size).

konsole and termux (with my PR) do.

I see, I guess I never even bothered to check... I just always assumed it was only iTerm2 that supported it. Wezterm also does (just tried it now)... Never tried it on mintty or hterm (which both support the image protocol, at least to an extent) though 🤔

All in all, you're right about this. I think I'll change my detection method and hope no terminal emulator goes astray 🥲 (if so, I'll probably use the name/version method as a fallback for such).

Anyways, I believe the name/version introspection is still useful is other scenarios... particularly where the implementation of a common feature differs from other terminal emulators (and is not a bug) e.g Termux may not support some image formats or animated images (I should've asked about this earlier), or behave differently by design/platform limitation, etc... and special handling may be needed for such differences.

This applies beyond just the iTerm2 protocol by the way.

Thanks.

AnonymouX47 commented 1 year ago

This just randomly came to mind... concerning my previous comment (didn't wanna edit it for the sake of those who may have read it), version introspection is useful (maybe even crucial) in the following cases:

a bug (for which a workaround was applied before) got fixed
- e.g see https://github.com/kovidgoyal/kitty/issues/5081 and https://github.com/wez/wezterm/issues/3250
an existing feature got broken (and there is a possible workaround)
- e.g see https://github.com/wez/wezterm/issues/3529 (and other issues mentioned in there)
a new feature (of an existing protocol) got added/implemented and there's no way to detect support for that feature specifically.
the behavior of an existing feature was intentionally changed, for whatever reason.

Kreijstal commented 8 months ago

This should be good. But can you provide an example of software that supports sixel graphics ? I know a termplay program (https://github.com/jD91mZM2/termplay) that can play image/video on terminal but what else ?

vv, vidxel, w3m with sixel support, ranger

demure commented 7 months ago

This should be good. But can you provide an example of software that supports sixel graphics ? I know a termplay program (https://github.com/jD91mZM2/termplay) that can play image/video on terminal but what else ?

vv, vidxel, w3m with sixel support, ranger

And now tmux 3.4 has added support, making termux sixel even more desireable :)

Flexximilian commented 3 months ago

This should be good. But can you provide an example of software that supports sixel graphics ? I know a termplay program (https://github.com/jD91mZM2/termplay) that can play image/video on terminal but what else ?

vv, vidxel, w3m with sixel support, ranger

And now tmux 3.4 has added support, making termux sixel even more desireable :)

Don't forget chafa, caca and graphviz. In combination with chafa basically anything that emits graphics to STDOUT can be used on the command line, e. g.:

curl 'https://avatars.githubusercontent.com/u/2194786?s=80&v=4' | chafa

Can this be merged and released? What's still needed to bring this feature?

yeetssite commented 3 months ago

This is still open? Is anybody actually working on this? I would very much like to see this implemented

TomJo2000 commented 3 months ago

It is still open. As indicated by its status being Open.

But no it is not being worked on anymore, seeing as the PR linked to this issue:

2973

is complete and awaiting merger, unfortunately GitHub has no status to indicate that.

Kreijstal commented 3 months ago

This is still open? Is anybody actually working on this? I would very much like to see this implemented

just use termux monet no?

TomJo2000 commented 3 months ago

just use termux monet no?

That is a 3rd party fork we offer no support or compatibility guarantees for. Use according to your own judgement.

termux / termux-app

Feature Request: sixel graphics mode #142

2973