Consider replacing anstyle-lossy with prettypretty

[apparebit]

Hi,

nice work! I happen to agree on much of the premise of this project and am also building a library, called prettypretty, to improve color output of terminal applications. But where this project seems to have focused on integration and interoperability with the ecosystem, I focused on the color science. I believe that both projects have complementary strengths and would benefit from each other. Concretely, I am suggesting that you replace the anstyle-lossy crate ("Lossy") with prettypretty because you'll get much better color conversions. Before I share results and on a more personal level, it was really cool discovering that we independently came up with the same color conversion algorithm based on a brute force search for the closest color.

I spent some time earlier today trying to understand the source for Lossy's color math and couldn't quite discern whether the author is confused about color science or just using language and symbols in ways that are unfamiliar to me. I do know of at least one expert who similarly abuses the term "perceptually uniform," so I can't decide. But in either case, basing one's color matching experiments on an RGB cube without defining either base color space or the new primaries and without accounting for test subjects' differences in color vision is a recipe for meaningless results. Assuming that the color space really is sRGB, those 64 colors are not even remotely close to being distributed in a perceptually uniform manner—as you can see in the plots of chroma/hue pairs and lightness values after conversion to Oklab, a color space that comes closer to being perceptually uniform than most. The rainbow line is the boundary of the sRGB gamut and the chart is labelled 60+4 because 4 colors are grays and they all sit on the origin.

Suffice to say, that this is not a space where Euclidian distance is an accurate reflection of color distance. But maybe the weights make all the difference. I performed some experiments:

• Inputs: all colors in the 6x6x6 RGB cube embedded in 8-bit terminal colors
• Color processing: Convert embedded RGB to 24-bit color and then convert result to ANSI colors using:
  • Lossy's brute force search for the closest match in sRGB with Lossy's distance metric
  • Prettypretty's independent implementation of the same algorithm but using prettypretty's high-resolution colors with floating point coordinates, the Oklrab color space (a slight improvement on the above mentioned Oklab), and an unweighted Euclidian distance metric
  • My own conversion algorithm: I developed it after noticing some pathological results from closest match search. It exploits the semantics of ANSI colors, which come in pairs of regular and bright colors, and uses hue to select the closest pair and then lightness to select the closest matching color, all in Oklrch, the cylindrical version of Oklrab.
• Contextual state: Prettypretty uses the terminal's theme colors, which were the "Basic" theme for Apple's Terminal.app, a saturated and bright theme, with colors having the same hues as the sRGB primaries and secondaries.

Before I share the three color grids for the three algorithms, here is the color grid without downsampling but showing the input colors:

Here are the results of Lossy's algorithm:

The color ramps on the right show that the algorithm is sensitive to luminance but its coordinate space and metrics push it towards the grays.

Here are the results for the Oklrab version with high-resolution colors:

The ramps are more fractured and there are fewer grays, both improvements in my mind.

Here is my own algorithm:

Since it matches colors and grays to colors and grays only, the grays are limited to single cells on the diagonal from upper-left to lower-right corner, and the colors dominate. There also are more graduations. Arguably, it's a bit heavy on the bright reds and might benefit from a weight to bias red selection towards the darker tone.

Still, prettypretty clearly outperforms Lossy and also has more options for other conversions as well as color manipulation in general. So please do consider replacing anstyle-lossy with prettypretty. Or, since you are really good at interfacing with the ecosystem, do offer an option that enables prettypretty as the color engine.

Cheers!

Robert

[epage]

At the moment, anstyle-lossy only has two dependents on crates.io

• anstyle-roff for lossless conversion from upper-256 colors to RGB
• anstyle-svg for lossy conversion from 16-colors to RGB

I had considered anstream providing automatic downgrade from RGB to 16-colors but decided to pass on that out of concern for quality and performance (it would require parsing, slowing things down when likely nothing would come).

It seems your focus is on RGB to 16-colors conversion. Seeing as we have no use case for that atm and your description sounds heavier weight than anstyle-lossy, I would be inclined to keep using it but I would be willing to link out to prettypretty.

[apparebit]

but decided to pass on that out of concern for quality and performance

I believe that prettypretty addresses the quality concern quite nicely. As to performance: I am not convinced that the performance of color conversions matters, especially when it comes to interactive use. Some OS terminal implementations are notoriously slow to begin with. If humans are on the critical path, we tend to be operating at much slower speeds as well.

More importantly, converting styles on the critical output path seems like the wrong usage model anyways. I cover this topic in the documentation, see the progress bar deep dive. I strongly believe that the right way forward is to define terminal application styles separately from the output routines and to adjust them to terminal capabilities, runtime context, and user preferences at application startup. If an application does that, the performance overhead won't matter.

Finally, my subjective impression, now that I ported all critical color routines to Rust, is that native prettypretty is blazing fast. But it probably is worth doing some benchmarking just so I have some idea what the cost of floating point operations is in Rust. (It's been quite a ride, in part because Rust's support for floating point math is really uneven. Currently, we can't do floating point operations in const functions. But we can do them in const expressions. So the universal duct tape of Rust, macros, can save the day yet again....)

It seems your focus is on RGB to 16-colors conversion

Actually, the focus is on conversion any which way, between the terminal color representations, to high-resolution color, and back again to 24-bit, 8-bit, and ANSI colors. They are all implemented and work well enough. But the conversion from any of the other colors to ANSI colors probably is the hardest because there are so few candidates and those candidates have rather unusual semantics, i.e., they don't have intrinsic color values and hence are abstract colors.

I would be willing to link out to prettypretty.

That would be fantastic. If you get started on that, please do keep notes on what still are rough edges and let me know, so that I can improve the developer experience.

Two more things:

First, when you say "lossy conversion from 16-colors to RGB," that's not quite lossy. It's perfectly accurate and color-preserving if the conversion uses the current color theme, which prettypretty encourages applications to do. Though, the result is very much context-sensitive and depends on the current terminal and color theme.

Second, I noticed that you kind of duct-taped the two default foreground and background colors into anstyle's output routines. I started out in a similar way, then added an explicit sentinel for the default color (prettypretty was still Python-only at the time, so using -1 was trivial). I originally used an explicit variant in the Rust implementation. But about two weeks ago, I realized that this approach caused friction because it's impossible to tell the context (foreground or background) from a single default color value. So I switched to a DefaultColor enum with two entires. It makes for a cleaner and more uniform model and better developer experience, even if down-conversion never targets the two. Though they are eminently useful for restoring the terminal to a known good color state, which simplifies the automatic deprivation of "undo" codes that revert a style. Do you want me to create a separate issue for that?

[epage]

I believe that prettypretty addresses the quality concern quite nicely.

While its an improvement, I think there is still enough loss in differentiation that someone should probably theme for 16-color independent of truecolor.

I am not convinced that the performance of color conversions matters, especially when it comes to interactive use. Some OS terminal implementations are notoriously slow to begin with. If humans are on the critical path, we tend to be operating at much slower speeds as well.

Its not just the cost of color conversions but the parsing of ANSI escape codes.

More importantly, converting styles on the critical output path seems like the wrong usage model anyways. I cover this topic in the documentation, see the progress bar deep dive. I strongly believe that the right way forward is to define terminal application styles separately from the output routines and to adjust them to terminal capabilities, runtime context, and user preferences at application startup. If an application does that, the performance overhead won't matter.

That is the model I used for a while and found to be pretty broken. For TUIs and some CLIs, I think it can still work.

In this model, you can do all of the work to pick color palettes but in the end, you are still left with two palettes, stdout and stderr. The code formatting output needs to know which of those two it's writing to. For TUIs, everything just goes to the screen and this distinction is moot, so that works. anstream is meant to help with that by allowing code formatting not have to know what it is writing to. For example, this works well with clap and env_logger where the user can provide colored output but where that gets written is determined at runtime.

I would be willing to link out to prettypretty.

That would be fantastic. If you get started on that, please do keep notes on what still are rough edges and let me know, so that I can improve the developer experience.

In response to something you said, I went digging into the docs and I think it would be helpful for them to be polished up a bit first. I opened https://docs.rs/prettypretty/latest/prettypretty/ and had no idea where something you mentioned was or how to use it.

Some thoughts

• Don't generate the docs with pyffi (as Rust users aren't a target audience for that) or at least add #![cfg_attr(docsrs, feature(doc_auto_cfg))] to your src/lib.rs so that the API is automatically tagged with pyffi so people know to ignore those. Or consider moving that into a separate package.
• Add an example on landing page at https://docs.rs/prettypretty/latest/prettypretty/
• Through examples, API organization, or documentation summaries, provide hints to help people find the parts of the API that are relevant
  • Its not clear why assert_close_enough or to_eq_bits is even in the API
  • Docs for assert_same_color focuses on the implementation and I have no clue what it actually considers the "same". I assume there is a level of "close enough" with it and that is also unclear
  • Its hard to tell what are helper types (like ThemeEntry) and what are types that I need to look at for meaningful logic (like Color)
  • The summary of Fidelity gives me no context. The second paragraph starts with a repetition of the first. It then tells me what I need to know. This a "Terminal Color Capabilities"

First, when you say "lossy conversion from 16-colors to RGB," that's not quite lossy.

Its a weird middle ground. You can strictly convert back losslessly if you have the same theme as generated it.

It's perfectly accurate and color-preserving if the conversion uses the current color theme, which prettypretty encourages applications to do

What do you mean by "current color theme". Are you detecting what color theme the terminal is set to?

btw in the case where I use this, there is no terminal. I also don't really see much point to this when there is a terminal unless you are manipulating the colors.

Second, I noticed that you kind of duct-taped the two default foreground and background colors into anstyle's output routines. I started out in a similar way, then added an explicit sentinel for the default color (prettypretty was still Python-only at the time, so using -1 was trivial). I originally used an explicit variant in the Rust implementation. But about two weeks ago, I realized that this approach caused friction because it's impossible to tell the context (foreground or background) from a single default color value. So I switched to a DefaultColor enum with two entires. It makes for a cleaner and more uniform model and better developer experience, even if down-conversion never targets the two. Though they are eminently useful for restoring the terminal to a known good color state, which simplifies the automatic deprivation of "undo" codes that revert a style. Do you want me to create a separate issue for that?

Frankly, I don't even know what you are talking about. I don't see how a DefaultColor would fit into anstyle. Also, anstyles primary responsibility is for describing colors, not outputting, so it intentionally does not do any optimizations of the output.

[apparebit]

I'm writing this in the cab to the airport, so my apologies for the lack of quotes for context. The GitHub iPhone app doesn't have good support for commenting.

Thanks for the feedback. I did make some documentation changes, including restoring the overview of main types in the API docs, adding text in the item summaries for helpers, and adding a light dusting of links.

Also great point about docs.rs. The version displayed there now is Rust-only. (I actually have color badges for Python- and Rust-only items but docs.rs doesn't pick up my CSS.)

I'll add more links and example over the coming few weeks but am traveling in Europe for 10 days, so won't have much bandwidth. Still, the latest release is out and much improved thanks to your feedback.

As to your comments about needing to parse before output, that seems to be a result of your decision to use strings as the only representation. I can see that that necessitates parsing, but that also is a strong reason to use a different in-memory representation. However, I don't see how can get away with not parsing before output and also have a model where apps don't need to adjust their terminal styles. If you don't dynamically adjust styles to terminal capabilities and user preferences at some point before output, then you may just end up with ANSI escapes in the output that weren't consumed by the terminal or colors where none are wanted. How do you plan on achieving both?

Prettypretty does indeed query the terminal for its current color theme. It's part of adjusting to the runtime context. The code currently lives in Python only but I plan on porting to Rust soon. Though I'm a bit weary of its interaction with async. Since I need non-blocking single character reads (well, with a short timeout), I currently use select on Linux + macOS (and punt on Windows). That works because there's only one file descriptor that needs selecting. Since I don't want to hardcode that into the Rust version, I may just try the sans-IO approach and provide the good ol' select implementation as the lightweight default.

Finally, the two default colors for foreground and background correspond to SGR parameters 39 and 49, respectively. Many terminals make them configurable as well and they may not be the same as any of the 16 ANSI colors. While their use is limited by them only affecting foreground and background, they actually are a great solution to restoring a terminal to a known good state. By modeling them, prettypretty can automatically compute the style that undoes another or that is the difference from another style for incremental updates.

[epage]

As to your comments about needing to parse before output, that seems to be a result of your decision to use strings as the only representation. I can see that that necessitates parsing, but that also is a strong reason to use a different in-memory representation. However, I don't see how can get away with not parsing before output and also have a model where apps don't need to adjust their terminal styles. If you don't dynamically adjust styles to terminal capabilities and user preferences at some point before output, then you may just end up with ANSI escapes in the output that weren't consumed by the terminal or colors where none are wanted. How do you plan on achieving both?

This is making it sounds like its theoretical. I am actively using this strategy today. env_logger used to have a bespoke styling API in its API, kind of like yansi and similar packages. That has a lot of policy and people have their own styling packages, choice for when to disable colors, etc. Rather than coupling all of that together and trying to maintain API compatibility, what using String does is it means the caller can use whatever styling package they want, passing a String to use using a very stable API (SGR). We then do more processing and then write to a stream and the user again has a choice as to what they do with the output. This keeps concepts decoupled and APIs minimal.

EDIT: there are also cases involved here that involve JSON APIs (rust to cargo) where it adds even more complexity to design a bespoke styling "API" (json schema).

The only time we parse is to strip ANSI escape codes and that is a diferent level of parsing than needing to translate truecolor escape codes to 16-color escape codes.

Finally, the two default colors for foreground and background correspond to SGR parameters 39 and 49, respectively. Many terminals make them configurable as well and they may not be the same as any of the 16 ANSI colors. While their use is limited by them only affecting foreground and background, they actually are a great solution to restoring a terminal to a known good state. By modeling them, prettypretty can automatically compute the style that undoes another or that is the difference from another style for incremental updates.

Calculating undoing or incremental updates is out of scope of anstyle the package. The package's primary role is to be a minimal, stable API for showing up in other package's API (clap, env_logger, etc).