'rustc' panicked at 'called `Result::unwrap()` on an `Err` value: TryFromIntError(())

[hzargar2]

I am having compiler errors when compiling my project and there is no specific code that causes the issue. The issue goes away when the size of the file decreases. The file is very long (~85 000 lines) and it is generated by a script. When the first half of the file is removed, it compiles, when the last half is removed it also compiles. But when both are present it does not compile which leads me to believe it is an issue with the length of the file or the number of repeated structs/enums. I have included a code example of a minimum reproducible example. The functions in the file are all similar in structure except with different struct and enum names and extra conditional branches depending on the attributes present in the structs. They represent resolvers in my GraphQL API for the async-graphql lib.

I also have 96GB of ram in my machine.

The issue still persists when the --jobs is set to 1 and when cargo clean if run before compile. Also persists when compiling in --release mode

Code

#[derive(Default)]
pub struct FormObjectASubscriptionRootV1;

#[Subscription(rename_fields="lowercase", rename_args="lowercase")]
impl FormObjectASubscriptionRootV1 {

    async fn form_object_a_program_assigned_metadata<'a>(
        &'a self,
        ctx: &'a Context<'_>,
        filters: Option<InputV1FormObjectAProgramAssignedMetadata>,
    ) -> Result<impl Stream<Item = Result<OutputV1FormObjectAProgramAssignedMetadata, Error>> + 'a, Error> {
        let credentials = ctx
            .data::<Credentials>()
            .map_err(|x| Error::new("Error getting credentials."))?;
        let server_uid = ctx
            .data::<ServerUID>()
            .map_err(|x| Error::new("Internal service error."))?.0.clone();
        let client_uid = Uuid::new_v4().to_string();
        let broadcaster_maps = ctx
            .data::<HashMap<String, FormObjectABroadcasterEnum>>()
            .map_err(|x| Error::new("Internal service error."))?;
        let broadcaster_enum = broadcaster_maps
            .get("form_object_a_program_assigned_metadata")
            .ok_or_else(|| Error::new("Internal service error."))?;
        let redis_pool = ctx
            .data::<bb8_redis::bb8::Pool<RedisConnectionManager>>()
            .map_err(|x| Error::new("Internal service error."))?;
        let resource = "sec";

        if let Some(document_version) = filters.as_ref().and_then(|x| x.document_version.as_ref()){
            let regex_res = regex::Regex::new(document_version.as_str());
            if let Err(e) = regex_res{
                return Err(Error::new(e.to_string().as_str()));
            }
        }

        if let FormObjectABroadcasterEnum::ReceivableInsertedFormObjectAProgramAssignedMetadataValueBroadcaster(broadcaster) = broadcaster_enum{
            let mut receiver = broadcaster.subscribe();
            match redis_rate_limit_lib::subscribe::subscribe(credentials.api_key.as_str(), resource, server_uid.as_str(), client_uid.as_str(), &600, redis_pool).await {
                Ok(headers)=> {

                    return Ok(async_stream::stream! {

                        // not used, want Drop to be called on this when stream goes out of scope so it send an unsubscribe call to our server.
                        let subscription_connection = SubscriptionConnection{
                            client_uid: client_uid.clone(),
                            server_uid: server_uid.clone(),
                            resource: resource.to_string(),
                            api_key: credentials.api_key.clone(),
                            redis_pool: redis_pool.clone()
                        };

                        let mut st = time::Instant::now();
                        let mut consecutive_failed_subs = 0;

                        loop{

                            let val_res = receiver.try_recv();

                            match val_res{
                                Ok(received_value) => {
                                    let output_object = OutputV1FormObjectAProgramAssignedMetadata::from(received_value);

                                    // filter messages from channel based on supplied filters
                                    if let Some(input_filters) = &filters{

                                        let mut filing_number_filter_set = false;
                                        let mut filing_number_match = false;

                                        if let Some(filing_number_filterable_i64) = input_filters.filing_number.as_ref(){
                                            filing_number_filter_set = true;
                                            filing_number_match = filterable_inputs::filterable_i64::filter(&filing_number_filterable_i64, &output_object.filing_number);
                                        }

                                        let mut document_version_filter_set = false;
                                        let mut document_version_match = false;

                                        if let Some(document_version_regex) = input_filters.document_version.as_ref(){
                                            document_version_filter_set = true;
                                            document_version_match = filterable_inputs::string_match_regex::regex_is_match(&document_version_regex, output_object.document_version.as_str());
                                        }

                                        let mut filter_set_and_match = Vec::new();
                                        filter_set_and_match.push((filing_number_filter_set, filing_number_match));
                                        filter_set_and_match.push((document_version_filter_set, document_version_match));

                                        // if all filters that have been set have a match found for them then yield the object
                                        if !filter_set_and_match.iter().any(|(filter, m)| *filter && !m){
                                            yield Ok(output_object);
                                        }

                                    }else{ // no filters so just return all objects
                                        yield Ok(output_object);
                                    }

                                    // reset counter since successfully received a value from the channel
                                    consecutive_failed_subs = 0;
                                }
                                Err(TryRecvError::Closed) => {
                                    if consecutive_failed_subs < 3{
                                        yield Err(Error::new(format!("Subscription stream closed due to internal server error. Retrying to resubscribe in 5 seconds. Attempts left: {}", 3 - consecutive_failed_subs)));
                                        tokio::time::sleep(Duration::from_secs(5)).await;
                                        receiver.resubscribe();
                                        consecutive_failed_subs += 1;
                                    }
                                    // break out loop, failed to resubscribe after 3 times
                                    else{
                                        yield Err(Error::new("Closing connection due to internal server error."));
                                        // breaking out loop exits stream as it terminates after loop
                                        break;
                                    }
                                }
                                _ => {}
                            }

                            // random await needed so that it can check whether the stream has
                            // been dropped or not. After client attempts to
                            // close connection by sending stop request, Stream does not drop until awaited so client doesn't get a closed/complete response
                            // (preventing client from reconnecting in graphql). Also, since stream does not drop until awaited,
                            // the unsubscribe call to remove the entry in redis never happens. See if better way to do this later. Might need to
                            // use a stream object instead of a macro. This await is needed because we are using try_recv()
                            // instead of recv().await so our refresh code for our key can execute without anything being sent in the channel
                            time::sleep(Duration::from_nanos(1)).await;

                            if st.elapsed().as_secs() >= 30{

                                // terminate subscription if got response but it has an error status code
                                if let Err(rate_limit_error) = redis_rate_limit_lib::refresh::refresh_key(credentials.api_key.as_str(), resource, server_uid.as_str(), client_uid.as_str(), &600, redis_pool).await {
                                    yield Err(Error::new("Closing connection due to internal server error."));
                                    // breaking out loop exits stream as it terminates after loop
                                    break;
                                }
                                // keep subscription open if can't get response until service comes back online, better user experience.

                                st = time::Instant::now();
                            }
                        }
                    });
                }
                _ => {
                    return Err(Error::new("Unable to subscribe."));
                }
            }
        }else{
            return Err(Error::new("Unable to subscribe to channel."));
        }
    }
}

Meta

rustc --version --verbose:

rustc 1.70.0 (90c541806 2023-05-31) running on x86_64-unknown-linux-gnu

Error also persists on nightly (1.72) and in both debug and release mode for compilation in all versions and also in 1.69.

Error output

thread 'rustc' panicked at 'called `Result::unwrap()` on an `Err` value: TryFromIntError(())', compiler/rustc_metadata/src/rmeta/table.rs:317:57

Backtrace

``` stack backtrace: 0: 0x7f32cf564cca - std::backtrace_rs::backtrace::libunwind::trace::h9a6b80bbf328ba5d at /rustc/90c541806f23a127002de5b4038be731ba1458ca/library/std/src/../../backtrace/src/backtrace/libunwind.rs:93:5 1: 0x7f32cf564cca - std::backtrace_rs::backtrace::trace_unsynchronized::hd162ec543a11886b at /rustc/90c541806f23a127002de5b4038be731ba1458ca/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5 2: 0x7f32cf564cca - std::sys_common::backtrace::_print_fmt::h78a5099be12f51a6 at /rustc/90c541806f23a127002de5b4038be731ba1458ca/library/std/src/sys_common/backtrace.rs:65:5 3: 0x7f32cf564cca - ::fmt::ha1c5390454d74f71 at /rustc/90c541806f23a127002de5b4038be731ba1458ca/library/std/src/sys_common/backtrace.rs:44:22 4: 0x7f32cf5c90cf - core::fmt::write::h9ffde816c577717b at /rustc/90c541806f23a127002de5b4038be731ba1458ca/library/core/src/fmt/mod.rs:1254:17 5: 0x7f32cf557805 - std::io::Write::write_fmt::h88186074961638e4 at /rustc/90c541806f23a127002de5b4038be731ba1458ca/library/std/src/io/mod.rs:1698:15 6: 0x7f32cf564a95 - std::sys_common::backtrace::_print::h184198273ed08d59 at /rustc/90c541806f23a127002de5b4038be731ba1458ca/library/std/src/sys_common/backtrace.rs:47:5 7: 0x7f32cf564a95 - std::sys_common::backtrace::print::h1b4d8e7add699453 at /rustc/90c541806f23a127002de5b4038be731ba1458ca/library/std/src/sys_common/backtrace.rs:34:9 8: 0x7f32cf56773e - std::panicking::default_hook::{{closure}}::h393bcea75423915a at /rustc/90c541806f23a127002de5b4038be731ba1458ca/library/std/src/panicking.rs:269:22 9: 0x7f32cf5674e5 - std::panicking::default_hook::h48c64f31d8b3fd03 at /rustc/90c541806f23a127002de5b4038be731ba1458ca/library/std/src/panicking.rs:288:9 10: 0x7f32d2823995 - >::call_once::{shim:vtable#0} 11: 0x7f32cf567f34 - as core::ops::function::Fn>::call::hb9b860f5a1175bda at /rustc/90c541806f23a127002de5b4038be731ba1458ca/library/alloc/src/boxed.rs:1987:9 12: 0x7f32cf567f34 - std::panicking::rust_panic_with_hook::hafdc493a79370062 at /rustc/90c541806f23a127002de5b4038be731ba1458ca/library/std/src/panicking.rs:695:13 13: 0x7f32cf567ca9 - std::panicking::begin_panic_handler::{{closure}}::h0a64bc82e36bedc7 at /rustc/90c541806f23a127002de5b4038be731ba1458ca/library/std/src/panicking.rs:582:13 14: 0x7f32cf565136 - std::sys_common::backtrace::__rust_end_short_backtrace::hc203444fb7416a16 at /rustc/90c541806f23a127002de5b4038be731ba1458ca/library/std/src/sys_common/backtrace.rs:150:18 15: 0x7f32cf567a02 - rust_begin_unwind at /rustc/90c541806f23a127002de5b4038be731ba1458ca/library/std/src/panicking.rs:578:5 16: 0x7f32cf5c5443 - core::panicking::panic_fmt::h0f6ef0178afce4f2 at /rustc/90c541806f23a127002de5b4038be731ba1458ca/library/core/src/panicking.rs:67:14 17: 0x7f32cf5c5a73 - core::result::unwrap_failed::h8090202169109f9c at /rustc/90c541806f23a127002de5b4038be731ba1458ca/library/core/src/result.rs:1687:5 18: 0x7f32d1b98308 - ::encode_crate_root 19: 0x7f32d1b28af9 - rustc_metadata[af2d1f1b6c1f5533]::rmeta::encoder::encode_metadata_impl 20: 0x7f32d1b1c541 - rustc_data_structures[fea82b52c2e47d17]::sync::join:: 21: 0x7f32d1b1c25f - rustc_metadata[af2d1f1b6c1f5533]::rmeta::encoder::encode_metadata 22: 0x7f32d1b1b5f1 - rustc_metadata[af2d1f1b6c1f5533]::fs::encode_and_write_metadata 23: 0x7f32d1b14d80 - rustc_interface[7aa24cde61830128]::passes::start_codegen 24: 0x7f32d1b101b4 - ::enter::<::ongoing_codegen::{closure#0}::{closure#0}, core[da82827a87f140f9]::result::Result, rustc_span[2f805118d893a80f]::ErrorGuaranteed>> 25: 0x7f32d1b0e858 - ::ongoing_codegen 26: 0x7f32d1b0e061 - ::enter::, rustc_span[2f805118d893a80f]::ErrorGuaranteed>> 27: 0x7f32d1b09281 - rustc_span[2f805118d893a80f]::set_source_map::, rustc_interface[7aa24cde61830128]::interface::run_compiler, rustc_driver_impl[fe6df70259db788d]::run_compiler::{closure#1}>::{closure#0}::{closure#0}> 28: 0x7f32d1b0882f - std[b70fedfd8b77e9]::sys_common::backtrace::__rust_begin_short_backtrace::, rustc_driver_impl[fe6df70259db788d]::run_compiler::{closure#1}>::{closure#0}, core[da82827a87f140f9]::result::Result<(), rustc_span[2f805118d893a80f]::ErrorGuaranteed>>::{closure#0}::{closure#0}, core[da82827a87f140f9]::result::Result<(), rustc_span[2f805118d893a80f]::ErrorGuaranteed>> 29: 0x7f32d20ec661 - <::spawn_unchecked_, rustc_driver_impl[fe6df70259db788d]::run_compiler::{closure#1}>::{closure#0}, core[da82827a87f140f9]::result::Result<(), rustc_span[2f805118d893a80f]::ErrorGuaranteed>>::{closure#0}::{closure#0}, core[da82827a87f140f9]::result::Result<(), rustc_span[2f805118d893a80f]::ErrorGuaranteed>>::{closure#1} as core[da82827a87f140f9]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0} 30: 0x7f32cf5723b5 - as core::ops::function::FnOnce>::call_once::ha1f2224656a778fb at /rustc/90c541806f23a127002de5b4038be731ba1458ca/library/alloc/src/boxed.rs:1973:9 31: 0x7f32cf5723b5 - as core::ops::function::FnOnce>::call_once::haa29ed9703f354b7 at /rustc/90c541806f23a127002de5b4038be731ba1458ca/library/alloc/src/boxed.rs:1973:9 32: 0x7f32cf5723b5 - std::sys::unix::thread::Thread::new::thread_start::h33b6dae3e3692197 at /rustc/90c541806f23a127002de5b4038be731ba1458ca/library/std/src/sys/unix/thread.rs:108:17 33: 0x7f32cf31bc57 - start_thread 34: 0x7f32cf3a1a70 - __clone3 35: 0x0 - error: the compiler unexpectedly panicked. this is a bug. note: we would appreciate a bug report: https://github.com/rust-lang/rust/issues/new?labels=C-bug%2C+I-ICE%2C+T-compiler&template=ice.md note: rustc 1.70.0 (90c541806 2023-05-31) running on x86_64-unknown-linux-gnu note: compiler flags: --crate-type lib -C embed-bitcode=no -C debuginfo=2 -C incremental=[REDACTED] note: some of the compiler flags provided by cargo are hidden query stack during panic: end of query stack ```

[saethlin]

The problem is that rustc encodes a lot of metadata into tables with u32 indices. I am surprised this hasn't bitten us sooner.

Splitting your project into multiple crates might help you dodge this issue until it is fixed.

[hzargar2]

The problem is that rustc encodes a lot of metadata into tables with u32 indices. I am surprised this hasn't bitten us sooner.

Splitting your project into multiple crates might help you dodge this issue until it is fixed.

Splitting the project into multiple crates fixed the issue but it is from from ideal.

Is this an easy and quick fix or does this require extensive knowledge about the compiler? I may consider attempting it if not even though I have no idea where to start. Also, in the event I am unsuccessful, would you happen to know average turn around times for issues like this?

Thanks again!

[saethlin]

I expect that fixing this will be somewhat challenging. Upgrading any one code path to use a larger index is not that difficult but there are many. And it will probably impose a slowdown on all compiles, which is tough to swallow.

If you want to try to tackle this, the Rust Zulip has a t-compiler/help stream where a number of people could pitch in and help you through trying to fix this. https://forge.rust-lang.org/platforms/zulip.html

This is not a regression so it will not be automatically prioritized but I will nominate the general topic of using 32-bit indices for discussion at the next compiler team weekly meeting. Running out of 32-bit indices like this is quite rare but I have seen it before.

[estebank]

@saethlin could we, as a stop gap, avoid the ICE and produce a hard error asking the user to split their project into multiple crates? There are already errors for recursion limits in Rust and other languages, so it would seem reasonable to have a better DX for an implementation limitation. I could also see us using a new-type index that could be u32 or u64 based on a compiler flag in the future so that people with larger projects could opt-into the more expensive behavior.

[saethlin]

Adding a hard error for this code path sounds like a worthy experiment.

It's not clear to me if adding a compiler flag to toggle the index size would be any better than just switching to u64. If it is runtime-configurable, the compiler needs to contain all the code paths for handling both sizes and we lose any optimizations from constant-propagating the size. I expect we'd still have some wins in u32 mode because of the smaller encoded size, but really this is starting to sound like it has the performance characteristics of a specialized varint encoding. Probably a series of good experiments.

[apiraino]

T-compiler discussed during triage meeting on Zulip (notes).

We agree on the suggestion to turn the ICE into an error and to explore the idea of increasing the index size to u64, run a perf bench and then see the results (maybe discuss them in an MCP). @saethlin volunteered to drive this exploration. Thanks!

@rustbot label -I-compiler-nominated

[novafacing]

I'm seeing this on nightly when building a crate with multiple (64) large (30-80MB) include_bytes! uses. I know there was some recent effort to make include_bytes! not explode in memory usage, and that seems to have made this problem reveal itself more.

[saethlin]

The real solution to your trouble is for my PR linked above to be merged, but FWIW it seems very unlikely to me that my changes to improve include_bytes! performance are responsible for the change you've seen; those changes aren't in the latest nightly.

If you want to know which PR caused the change you should use https://github.com/rust-lang/cargo-bisect-rustc (just make sure to install from git, the published version is broken).