Support Kitty Image Protocol / MIME Types / Rendering

zachlloyd commented 3 years ago

https://sw.kovidgoyal.net/kitty/graphics-protocol/

elviskahoro commented 2 years ago

iTerm2 is able to display images (and animated gifs) within the terminal, see: https://iterm2.com/documentation-images.html.

It would be cool if this was supported within warp as well.

ghost commented 2 years ago

@elviskahoro

Additional resources:

I know you are aware of notcurses. Aim to survive the notcurses-demo, it is a very good test of performance, Unicode support, 24-bit RGB, memory pressure, and images (both kitty and sixel).
If you choose to support sixel (perhaps from merging ayosec's graphics PR branch from alacritty), then you should also be aware of chafa . Among its encoders is an extremely fast high-quality sixel encoder. notcurses and chafa are the fastest encoders I know of, and have different performance characteristics on the high end. If you can keep up with both of them, then you will be in great shape.
If you choose either sixel or iTerm2, then you should also test against jexer . When I wrote Jexer's image backend, only 3 terminals survived its lots-and-lots-of-small-images approach. Now most terminals are working OK (though some still crash). If you retrofit ayosec's graphics PR from alacritty, it should work fine with Jexer-type output, but will not work fine for notcurses output due to how it treats partially-damaged images.

On a different topic, just so I don't spam you with junk:

Since you started from alacritty, you should be good with vttest now as they stopped crashing on it 5-ish years ago. But their coverage of vttest/VT100/xterm is still far from complete. If very good VT100/xterm coverage including VT52 submode is one of your goals, then you will need to go a bit past the Paul Flo Williams parser. (Specifically it does not have a state for the VT52 direct cursor address sequence, which isn't terminated like ECMA-48/ANSI X3.64 sequences are supposed to be.)

Looking forward to your success. :)

elviskahoro commented 2 years ago

@klamonte thanks for sharing!

ghost commented 2 years ago

You're welcome. I'm rooting for you!

elviskahoro commented 2 years ago

From elna via Discord:

given that the UI framework can render arbitrary object, and like jupyter you have both a) a client server architecture and b) the notion of "cells", do you think you'll ever implement rendering of non-text based mime-types, e.g. svgs, pngs, ... for interactive plots like plotly?

https://discord.com/channels/851854972600451112/870145535993077780/885727936433500160

yes definitely although these two in my eyes are well solved by programs like https://github.com/antonmedv/fx and obviously nushell or https://www.visidata.org/, so I don't need warp to provide this - vs. e.g. a remote control API and support for a graphics rendering protocol which I need the terminal emulator to provide.

https://discord.com/channels/851854972600451112/870145535993077780/885728473463808020

https://github.com/joouha/euporie this is also a good inspiration which already works with kitty

https://discord.com/channels/851854972600451112/870145535993077780/885728712799166474

as is this https://github.com/dccsillag/magma-nvim

https://discord.com/channels/851854972600451112/870145535993077780/919330371647180832

https://github.com/tauri-apps/wry - how cool would it be if we could display html etc in warp? terminal based notebooks a la jupyter with rendering things like plotly plots would be possible

elviskahoro commented 2 years ago

https://gitlab.com/gnachman/iterm2/-/issues/6436

japanese-goblinn commented 2 years ago

Any updates on this? Really cool feature 😅

elviskahoro commented 2 years ago

@japanese-goblinn No updates. I'll share info in this github issue when there is. My guess is that this wouldn't be soon though.

elviskahoro commented 2 years ago

@japanese-goblinn No updates. I'll share info in this github issue when there is. My guess is that this wouldn't be soon though.

japanese-goblinn commented 2 years ago

@elviskahoro okey, thanks!

elviskahoro commented 2 years ago

From @wellcomez via #1754

Protocol iTerm2 extends the xterm protocol with a set of proprietary escape sequences. In general, the pattern is:

ESC ] 1337 ; key = value ^G Whitespace is shown here for ease of reading: in practice, no spaces should be used.

For file transfer and inline images, the code is:

ESC ] 1337 ; File = [arguments] : base-64 encoded file contents ^G The arguments are formatted as key=value with a semicolon between each key-value pair. They are described below:

mikeal commented 2 years ago

any updates on this? love this project but can't switch until this works :)

warpdotdev-devx commented 1 year ago

No updates. This wouldn't be for awhile it's very high engineering lift!

amjad1233 commented 1 year ago

Is this one why emoji's are not rendering or should I open a new issue for it. Bascially Emojis are coming as wierd character please see bleow screenshot, where emoji should render as Aussie flag. CleanShot 2022-11-28 at 08 57 25

p-i-c-o commented 1 year ago

@warpdotdev-dx any updates??

dannyneira commented 1 year ago

@p-i-c-o As far as I'm aware, there is some work being done on gif support, but nothing for kitty image support yet.

scottbea commented 1 year ago

It's honestly hard to take Warp serious if it does not offer the basics like rendering inline images, animated gifs, and images. I'd expect rendering to support pixel-based and character-based rendering both, with scaling, opacity control, caching for animations, etc. Even more, I'd expect pixel level R/W access to the terminal window (with plenty of security controls which makes this non-trivial). Also, plan now on how to support voxel rendering in the terminal because 3D matters.

There are some great comments above about the specifics around ensuring the terminal can perform very well with many hundreds of embedded / inline images. Yes, let's make it very fast please.

Think what we did with the C64 in the 80s... Let's see some radical stuff here!

RobbieMcKinstry commented 1 year ago

Ratatui, the Rust TUI framework, added experimental Sixel support as described here. I'm salivating at how great this would look in Warp! :D

t0msk commented 7 months ago

+1 I wish this feature

sneazy-ibo commented 7 months ago

any updates on this for warp?

t0msk commented 7 months ago

hmm its 3 years old feature request :( I mean modern terminal should support graphics rendering

sneazy-ibo commented 7 months ago

fr I got so many cli tools that use images, not having them visible on warp is really a pain

Cryolitia commented 7 months ago

need it, too

ALEZ-DEV commented 7 months ago

I'm really surprised that warp does not already support this

Ast3risk-ops commented 7 months ago

I'm really surprised that warp does not already support this

It should be a feature for a terminal that brands itself as "your terminal reimagined."

MilkTicc commented 4 months ago

Any update on this? Really need it

bgyss commented 4 months ago

Would definitely love to be able to use a tool like this in Warp:

https://github.com/hackerb9/lsix

ZhangHanDong commented 3 months ago

any updates on this for warp?

neuromuse commented 3 months ago

I love Warp but having image support is necessary for it to become my daily driver.

Can we please get some kind of developer response here? This request has been sitting here for 3 years.

danscheer commented 3 months ago

yeah, would love to use warp but no image support, no chance. When i do Frontend Development i'm constantly in Yazi for blaszingly fast codebase searching with previews, will switch to warp the day image preview is available, without it its just not worth the tradeoff.