Compiler unexpectedly panicked when formatting code

[gianzellweger]

I recently looked through all the features of Rustfmt and picked a few options where I preferred some option over the default. When trying this new mix, however, I got an error. I have attached both the log and my configuration. rustc-ice-2023-08-02T21:08:46.715244Z-75321.txt rustfmt.toml

[ytmimi]

@gianzellweger when you have a moment can you please provide a minimal code snippet inline that we can run rustfmt on with your configuration to reproduce the issue. something like:

fn main() {
    // code that reproduces the issue
}

Without an example code snippet there's very little we can do to help.

[gianzellweger]

@ytmimi Is there a way to see which code it was trying to format? The entire project has about 2000 LOC and I do not think that satisfies the "minimal code snippet" requirement. I also tried it with another project and there it worked like intended.

[ytmimi]

@gianzellweger first, I want to say thanks for bringing this to our attention. If you can try to narrow it down that would be great. Just looking at the logs you've provided I think this might be a duplicate of #4968

[gianzellweger]

@ytmimi I have now narrowed it down to a single file (which is still 1000 lines, so not a helpful progression) and I have tried deactivating every single configuration on its own. The panic only goes away if I remove error_on_unformatted, everything else still produces the panic. Is there anything else I should try?

[ytmimi]

Normally I'd recommend running rustfmt with RUSTFMT_LOG=debug. The extra info might help you narrow it down, but I think there might be an issue with logging with the most recent released version that won't be resolved until we sync this repo back up with the upstream rust-lang/rust repo. I was also going to say that you could have run rustfmt with -v to figure out what file, but you already narrowed it down.

Any chance you can share a link to the file?

[gianzellweger]

@ytmimi How about I instead narrow it down to the specific piece of code? There's only about 10-20 functions which I can comment out separately until the error goes away.

[gianzellweger]

@ytmimi The cause either appears multiple times in this one file or the cause is something else entirely. I will link the code below. If you need anything else, feel free to comment. Thank you and good luck. main.rs

[ytmimi]

Alright, I had a look at this and I think I've got it. In your case the error is happening in the component_tab function on the line that starts with format!("{:name_width$} {:temperature_width$.2}°C {:critical_width$}",. The issue is that there's a trailing space at the end of the line and you're using a °C unicode character. removing the whitespace will prevent the panic.

[gianzellweger]

@ytmimi That fixed it, thank you so much. Can we agree that this isn't optimal behavior from rustfmt and could be improved? Not trying to put any pressure on anybody.

[ytmimi]

@gianzellweger this particular issue will almost certainly be resolved once we add support for let-chains. The underlying issue here has more to do with not properly handling the size of the unicode char correctly. Similar issues have been reported before.

Here's a bit of a deep dive explanation into what's happening in this case:

fn fail() {
    if let Some(x) = x && y {
        format!("{:name_width$}  {:temperature_width$.2}°C  {:critical_width$}") 
        //                                                                      ^
        //                                                                      |
        //                                                                      |
        //                                                               trailing whitespace
    }
}

and here's a screenshot that highlights the whitespace at the end:

The issue is that rustfmt doesn't currently support formatting let-chains so when rustfmt reaches any let-chain in the code the best it can do is copy the entire Span of the AST node into the output buffer and move on with formatting.

At the end of the formatting process rustfmt does a pass over that output buffer to see if there are instances of trailing whitespace or lines that exceed the max_width, and in this case the panic is happening during this reporting process. The panic in annotate-snippets is a direct result of not handling the unicode char length properly.

[gianzellweger]

@ytmimi That makes a lot of sense, as my code uses let-chains. Is there any issue that tracks the progress of this or has this implementation not started at all? Thank you once more.

[ytmimi]

The example input from https://github.com/rust-lang/rustfmt/issues/5888 is also really helpful at highlighting this issue:

pub(crate) fn sanity_needs_ocr<T: HasWord + HasRect>(collection: &[T]) -> bool {
    let needs_ocr = collection.is_empty()
    // the presence of 'UNICODE REPLACEMENT CHARACTER'
    || collection.iter().any(|c| c.text().contains('�'))  
    // (yes, empty pages will also be OCR'd)
    || collection_is_void_of_text(collection)
    || is_gibberish(&collection.to_rendered_text());

    #[cfg(debug_assertions)]
    if needs_ocr {
        tracing::trace!(
            "sanity check failed, with text: {:?}",
            collection.to_rendered_text()
        );
    }

    needs_ocr
}

and here's my explanation of the problem: https://github.com/rust-lang/rustfmt/issues/5888#issuecomment-1682925447

[danielhuang]

I found a much smaller example that triggers the same error:

#![allow(incomplete_features)]
#![feature(
    try_blocks,
    never_type,
    min_specialization,
    exclusive_range_pattern,
    lazy_cell
)]

Putting that as input into rustfmt --edition 2021 --verbose --config max_width=10,use_small_heuristics=Off,error_on_line_overflow=true will create this error:

``` thread 'main' panicked at /rust/deps/annotate-snippets-0.9.1/src/display_list/from_snippet.rs:286:9: SourceAnnotation range `(10, 15)` is bigger than source length `12` stack backtrace: 0: 0x7f9a99b8b6f6 - std::backtrace_rs::backtrace::libunwind::trace::h5aa40918bcc5ae39 at /rustc/bf8716f1cd6416266807706bcae0ecb2e51c9d4a/library/std/src/../../backtrace/src/backtrace/libunwind.rs:104:5 1: 0x7f9a99b8b6f6 - std::backtrace_rs::backtrace::trace_unsynchronized::h59dfde9356d5c892 at /rustc/bf8716f1cd6416266807706bcae0ecb2e51c9d4a/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5 2: 0x7f9a99b8b6f6 - std::sys_common::backtrace::_print_fmt::hb48360d84c9c1fb4 at /rustc/bf8716f1cd6416266807706bcae0ecb2e51c9d4a/library/std/src/sys_common/backtrace.rs:68:5 3: 0x7f9a99b8b6f6 - ::fmt::hf56e8f54db192c87 at /rustc/bf8716f1cd6416266807706bcae0ecb2e51c9d4a/library/std/src/sys_common/backtrace.rs:44:22 4: 0x7f9a99bddf40 - core::fmt::rt::Argument::fmt::h19a989fd1ae4ebac at /rustc/bf8716f1cd6416266807706bcae0ecb2e51c9d4a/library/core/src/fmt/rt.rs:142:9 5: 0x7f9a99bddf40 - core::fmt::write::h01e0450c53f8a565 at /rustc/bf8716f1cd6416266807706bcae0ecb2e51c9d4a/library/core/src/fmt/mod.rs:1120:17 6: 0x7f9a99b7f53f - std::io::Write::write_fmt::hf0c44899be398691 at /rustc/bf8716f1cd6416266807706bcae0ecb2e51c9d4a/library/std/src/io/mod.rs:1810:15 7: 0x7f9a99b8b4d4 - std::sys_common::backtrace::_print::hb411883b66e31079 at /rustc/bf8716f1cd6416266807706bcae0ecb2e51c9d4a/library/std/src/sys_common/backtrace.rs:47:5 8: 0x7f9a99b8b4d4 - std::sys_common::backtrace::print::h957a27a025206bbf at /rustc/bf8716f1cd6416266807706bcae0ecb2e51c9d4a/library/std/src/sys_common/backtrace.rs:34:9 9: 0x7f9a99b8e267 - std::panicking::default_hook::{{closure}}::h959c1d4b0d8aaee7 10: 0x7f9a99b8dfc9 - std::panicking::default_hook::h0e46bc398d89a2e3 at /rustc/bf8716f1cd6416266807706bcae0ecb2e51c9d4a/library/std/src/panicking.rs:292:9 11: 0x7f9a9c959d2c - std[cf610168ecc2edd7]::panicking::update_hook::>::{closure#0} 12: 0x7f9a99b8e9b6 - as core::ops::function::Fn>::call::ha6a8ea08f3b30cdb at /rustc/bf8716f1cd6416266807706bcae0ecb2e51c9d4a/library/alloc/src/boxed.rs:2029:9 13: 0x7f9a99b8e9b6 - std::panicking::rust_panic_with_hook::hf3b1917b74ec8507 at /rustc/bf8716f1cd6416266807706bcae0ecb2e51c9d4a/library/std/src/panicking.rs:783:13 14: 0x7f9a99b8e702 - std::panicking::begin_panic_handler::{{closure}}::hf89e73c1529c6185 at /rustc/bf8716f1cd6416266807706bcae0ecb2e51c9d4a/library/std/src/panicking.rs:657:13 15: 0x7f9a99b8bbf6 - std::sys_common::backtrace::__rust_end_short_backtrace::hae46ca4bf38118e9 at /rustc/bf8716f1cd6416266807706bcae0ecb2e51c9d4a/library/std/src/sys_common/backtrace.rs:171:18 16: 0x7f9a99b8e460 - rust_begin_unwind at /rustc/bf8716f1cd6416266807706bcae0ecb2e51c9d4a/library/std/src/panicking.rs:645:5 17: 0x7f9a99bda645 - core::panicking::panic_fmt::hfa54bdafbcaf5e1d at /rustc/bf8716f1cd6416266807706bcae0ecb2e51c9d4a/library/core/src/panicking.rs:72:14 18: 0x5631664627c5 - >::from 19: 0x56316632d742 - ::fmt 20: 0x7f9a99bddf40 - core::fmt::rt::Argument::fmt::h19a989fd1ae4ebac at /rustc/bf8716f1cd6416266807706bcae0ecb2e51c9d4a/library/core/src/fmt/rt.rs:142:9 21: 0x7f9a99bddf40 - core::fmt::write::h01e0450c53f8a565 at /rustc/bf8716f1cd6416266807706bcae0ecb2e51c9d4a/library/core/src/fmt/mod.rs:1120:17 22: 0x7f9a99b7d57b - std::io::Write::write_fmt::h01243e7359d59acd at /rustc/bf8716f1cd6416266807706bcae0ecb2e51c9d4a/library/std/src/io/mod.rs:1810:15 23: 0x7f9a99b7d57b - <&std::io::stdio::Stderr as std::io::Write>::write_fmt::hf2788b8a958b4d5d at /rustc/bf8716f1cd6416266807706bcae0ecb2e51c9d4a/library/std/src/io/stdio.rs:947:9 24: 0x7f9a99b7de7a - ::write_fmt::hc9b4d235a188910c at /rustc/bf8716f1cd6416266807706bcae0ecb2e51c9d4a/library/std/src/io/stdio.rs:921:9 25: 0x7f9a99b7de7a - std::io::stdio::print_to::h25daf93803720e91 at /rustc/bf8716f1cd6416266807706bcae0ecb2e51c9d4a/library/std/src/io/stdio.rs:1020:21 26: 0x7f9a99b7de7a - std::io::stdio::_eprint::h8471efde70d82013 at /rustc/bf8716f1cd6416266807706bcae0ecb2e51c9d4a/library/std/src/io/stdio.rs:1108:5 27: 0x56316622bb0b - rustfmt[b8de71484191ea15]::format_and_emit_report:: 28: 0x563166229a26 - rustfmt[b8de71484191ea15]::execute 29: 0x5631662251d4 - rustfmt[b8de71484191ea15]::main 30: 0x56316620f373 - std[cf610168ecc2edd7]::sys_common::backtrace::__rust_begin_short_backtrace:: 31: 0x563166211569 - std[cf610168ecc2edd7]::rt::lang_start::<()>::{closure#0} 32: 0x7f9a99b6fcb1 - core::ops::function::impls:: for &F>::call_once::h020f5b91b80d06f2 at /rustc/bf8716f1cd6416266807706bcae0ecb2e51c9d4a/library/core/src/ops/function.rs:284:13 33: 0x7f9a99b6fcb1 - std::panicking::try::do_call::h74687c611156f5dd at /rustc/bf8716f1cd6416266807706bcae0ecb2e51c9d4a/library/std/src/panicking.rs:552:40 34: 0x7f9a99b6fcb1 - std::panicking::try::h124d6897d80260a0 at /rustc/bf8716f1cd6416266807706bcae0ecb2e51c9d4a/library/std/src/panicking.rs:516:19 35: 0x7f9a99b6fcb1 - std::panic::catch_unwind::ha7750cdf1bab93d0 at /rustc/bf8716f1cd6416266807706bcae0ecb2e51c9d4a/library/std/src/panic.rs:142:14 36: 0x7f9a99b6fcb1 - std::rt::lang_start_internal::{{closure}}::hbe641ae1f78ee023 at /rustc/bf8716f1cd6416266807706bcae0ecb2e51c9d4a/library/std/src/rt.rs:148:48 37: 0x7f9a99b6fcb1 - std::panicking::try::do_call::hfecfeb9f26c80cb7 at /rustc/bf8716f1cd6416266807706bcae0ecb2e51c9d4a/library/std/src/panicking.rs:552:40 38: 0x7f9a99b6fcb1 - std::panicking::try::hf9fca15f209cea9a at /rustc/bf8716f1cd6416266807706bcae0ecb2e51c9d4a/library/std/src/panicking.rs:516:19 39: 0x7f9a99b6fcb1 - std::panic::catch_unwind::hc5e2d1c6c479e094 at /rustc/bf8716f1cd6416266807706bcae0ecb2e51c9d4a/library/std/src/panic.rs:142:14 40: 0x7f9a99b6fcb1 - std::rt::lang_start_internal::hf290d4b3f3777ee0 at /rustc/bf8716f1cd6416266807706bcae0ecb2e51c9d4a/library/std/src/rt.rs:148:20 41: 0x56316622cb35 - main 42: 0x7f9a997d3cd0 - 43: 0x7f9a997d3d8a - __libc_start_main 44: 0x5631661fecb9 - 45: 0x0 - ```

There's no let-chains or non-ascii characters in this snippet.

[ytmimi]

@danielhuang thanks! The tab characters are what's causing the panic. Similar issues have been reported before. An explanation of what's going on can be found at https://github.com/rust-lang/rustfmt/issues/4968#issuecomment-949461340, and there are PRs already open to resolve your case.