ASCII color codes are displayed inline for for editors

[lindhe]

Sometimes, the error shown in Vim (and apparently VS Code too) is printed with ASCII color codes showing in the string from rust_analyzer. It makes the error very hard to read. See https://github.com/neovim/nvim-lspconfig/issues/2760.

It seems to me like rust-analyzer does not properly distinguish between being executed in a terminal environment or not, and decide from that whether to print ASCII color codes or skip them.

In terminal:

In editor:

Steps to reproduce

1. Create the following crate:

cargo new -q foo
cd foo
cargo add -q rocket@0.5.0-rc.3
cat <<EOF > src/main.rs
use rocket::UriDisplayPath;

fn main() {
    println!("Hello, world!");
}

#[derive(UriDisplayPath)]
pub enum Foo {
    Bar,
}
EOF

2. Optionally run cargo check.
3. Open src/main.rs with Vim.

Other info

rust-analyzer version: (eg. output of "rust-analyzer: Show RA Version" command, accessible in VSCode via Ctrl/⌘+Shift+P)

Mason reports rust-analyzer 2023-08-07.

rustc version: (eg. output of rustc -V)

$ rustc -V
rustc 1.71.1 (eb26296b5 2023-08-03)

relevant settings: (eg. client settings, or environment variables like CARGO, RUSTC, RUSTUP_HOME or CARGO_HOME)

Not sure, nothing I've set explicitly at least. I run NeoVim with rust-lang/rust.vim and rust_analyzer for lspconfig installed via Mason.

[Veykril]

Your error message for some reason contains a green [note] part, which is what you are seeing inline. That should not be there as that is not the normal diagnostic output of the compiler. This is what I am getting here I don't know what could cause this for you but something seems to fiddle with your compiler output?

[tadeokondrak]

I think the derive macro is emitting escape codes (when using a stable compiler): https://github.com/SergioBenitez/proc-macro2-diagnostics/blob/master/src/line.rs#L79

[Veykril]

Ah good catch, that's indeed gonna be the issue. There isn't really anything we can do here then, r-a just spits out the error messages we get in json verbatim as the expectation is for compiler diagnostics to be just text. This might be something to request as a feature for the proc-macro api, since the compiler itself has a flag that toggles whether the json output has colors in it or not which this should hook into.

[lindhe]

Ah, there it is! Thanks so much for identifying the real culprit for me! I'll continue my hunt over there.

[lindhe]

Interestingly, it seems like cargo build and proc-macro properly toggles color output sometimes!

Here's what it looks like in my terminal, and the color codes are not printed!

It seems like proc-macro do attempt to use different logic depending on whether or not color is set:

#[cfg(all(feature = "colors", not(nightly_diagnostics)))]

@Veykril, you mentioned "the error messages we get in json". It seems like cargo build --message-format has a few different options that might be of relevance here: json, json-diagnostic-short, json-diagnostic-rendered-ansi and json-render-diagnostics.

What field do you render? Is it message.message?

The full JSON object

```json { "reason": "compiler-message", "package_id": "foo 0.1.0 (path+file:///home/andreas/tmp/foo)", "manifest_path": "/home/andreas/tmp/foo/Cargo.toml", "target": { "kind": [ "bin" ], "crate_types": [ "bin" ], "name": "foo", "src_path": "/home/andreas/tmp/foo/src/main.rs", "edition": "2021", "doc": true, "doctest": false, "test": true }, "message": { "rendered": "error: [note] error occurred while deriving `UriDisplay`\n --> src/main.rs:7:10\n |\n7 | #[derive(UriDisplayPath)]\n | ^^^^^^^^^^^^^^\n |\n = note: this error originates in the derive macro `UriDisplayPath` (in Nightly builds, run with -Z macro-backtrace for more info)\n\n", "children": [], "code": null, "level": "error", "message": "\u001b[1;34m[\u001b[0m\u001b[1;32mnote\u001b[0m\u001b[1;34m] \u001b[0m\u001b[1;39merror occurred while deriving `UriDisplay`\u001b[0m", "spans": [ { "byte_end": 98, "byte_start": 84, "column_end": 24, "column_start": 10, "expansion": { "def_site_span": { "byte_end": 36109, "byte_start": 36044, "column_end": 66, "column_start": 1, "expansion": null, "file_name": "/home/andreas/.cargo/registry/src/index.crates.io-6f17d22bba15001f/rocket_codegen-0.5.0-rc.3/src/lib.rs", "is_primary": false, "label": null, "line_end": 1060, "line_start": 1060, "suggested_replacement": null, "suggestion_applicability": null, "text": [ { "highlight_end": 66, "highlight_start": 1, "text": "pub fn derive_uri_display_path(input: TokenStream) -> TokenStream {" } ] }, "macro_decl_name": "#[derive(UriDisplayPath)]", "span": { "byte_end": 98, "byte_start": 84, "column_end": 24, "column_start": 10, "expansion": null, "file_name": "src/main.rs", "is_primary": false, "label": null, "line_end": 7, "line_start": 7, "suggested_replacement": null, "suggestion_applicability": null, "text": [ { "highlight_end": 24, "highlight_start": 10, "text": "#[derive(UriDisplayPath)]" } ] } }, "file_name": "src/main.rs", "is_primary": true, "label": null, "line_end": 7, "line_start": 7, "suggested_replacement": null, "suggestion_applicability": null, "text": [ { "highlight_end": 24, "highlight_start": 10, "text": "#[derive(UriDisplayPath)]" } ] } ] } } ```

BTW, I notice that cargo build prints the [note] while rustc does not. What do rust-analyzer use internally to build/check?

EDIT 1: To my surprise, json and json-diagnostic-rendered-ansi has the same behavior:

[lindhe]

Rats! I think this might be a bug in Cargo. Even with --color=never, the color feature appears to be set when building using --json:

[Veykril]

No, there is no bug here. The colored [note] is created by the proc-macro itself, no tool has control over that proc-macro emitting color codes or not. Hence why I said

This might be something to request as a feature for the proc-macro api, since the compiler itself has a flag that toggles whether the json output has colors in it or not which this should hook into.

earlier

[lindhe]

the compiler itself has a flag that toggles whether the json output has colors in it or not

What flag is that, if not --color?

[Veykril]

That flag controls the color of the compilers output colors for diagnostic, the color you are seeing here (that is the [note] part) is created by the proc-macro though, not by the compiler. The compiler merely emits the message that the proc-macro tells it to.

[lindhe]

Yes, but proc-macro emits that conditionally on whether or not the color feature is set:

https://github.com/SergioBenitez/proc-macro2-diagnostics/blob/45fa2d68669d10daeaf0f9b664a96576626fae16/src/line.rs#L78C1-L78C1

[Veykril]

Yes, that is a feature flag. Passing --color or not to the compiler is rrelevant to that though, as that is not a flag that sets the feature. In fact, crates cannot observe --color being passed or not whatsoever

[lindhe]

Oh, that's interesting. Any idea how on earth proc-macro is able to distinguish between colorized and plain output then? Because, as you can see in my screenshots, it does not colorize "[note]" when I pipe it to cat.

[lindhe]

the compiler itself has a flag that toggles whether the json output has colors in it or not which this should hook into.

Do you have any pointers to how one checks that flag?

[lindhe]

That flag controls the color of the compilers output colors for diagnostic, the color you are seeing here (that is the [note] part) is created by the proc-macro though, not by the compiler. The compiler merely emits the message that the proc-macro tells it to.

Passing --color or not to the compiler is rrelevant to that though, as that is not a flag that sets the feature. In fact, crates cannot observe --color being passed or not whatsoever

I am sure you are right, I just don't understand. What trips me up here is that [note] is printed in color when setting --color=always but printed not in color when setting --color=never (see screenshot below). Unless Cargo or the compiler does a text replacement to strip color codes from the output, which I very much doubt, I fail to see how it can be the case that --color=always is not picked up by the proc-macro.

[SergioBenitez]

All, copying my comments from https://github.com/SergioBenitez/proc-macro2-diagnostics/issues/3#issuecomment-1712314620 here for discourse:

The issue [...] is that as far as I can tell, there's no way for a proc-macro to know if it should emit colors or not. The proxy is typically to check whether we're connected to a TTY, but Cargo captures all output, so we're never connected to a TTY, and detecting via this manner would mean never emitting colors, which would be a shame.

Ideally, there would be some way to query 1) whether cargo itself will be emitting colors and/or 2) what message-format is expected. Alternatively, if we could inquire about how the compiler is invoked, we might be able to patch this minimally for rust-analyzer. It doesn't seem like 1) or 2) are viable options now, but the alternative might be, given that rust-analyzer appears to set RUSTC_WRAPPER to rust-analyzer by default. If we can detect this, we can stop emitting colors when rust-analyzer invokes rustc.

[...] the above didn't work. First, the wrapper is only set for build scripts, which is fine-ish as we can forward the env-var along via Cargo. Second, and more importantly, the wrapper is only set the first time the build script is built/run, so the solution only really works once in a given workspace, if it works at all.

Until there's some reliable mechanism to detect whether we should color, I think the "correct" solution here is for rust-analyzer to strip ANSI escapes from any output before it re-emits it. Cargo itself uses https://crates.io/crates/strip-ansi-escapes, which is why using --color=never works as expected.

[Veykril]

Interesting, so cargo already handles this specially itself as well? In that case you are right, we should do the same as cargo here when processing compile_error! invocations (and also we should pass --color=never to cargo check, I don't think we are doing that currently)

[DriedYellowPeach]

Is there any solution I can avoid this right now? I tried add color=never in my .toml, but it only avoid color output in cargo build, not in diagnostics. Can I setup rust-analyzer or something?

[lindhe]

I know of no work-around until they fix this.

[SergioBenitez]

One "work-around" is to use a nightly toolchain, at least for rust-analyzer.