search

rust-lang / rust

Empowering everyone to build reliable and efficient software.
https://www.rust-lang.org
Other
97.97k stars 12.69k forks source link

ICE: unwrap none error in compiler\rustc_mir\src\monomorphize\collector.rs #85447

Closed kocsis1david closed 2 years ago

kocsis1david commented 3 years ago

Code

#![crate_type="lib"]
use std::{collections::HashMap, hash::Hash};

struct C;

trait Ctx {
    type BindGroupId;
}

impl Ctx for C {
    type BindGroupId = u32;
}

pub struct BindGroup {
    _id: <C as Ctx>::BindGroupId,
}

#[derive(Copy, Clone)]
pub struct BindGroupRef<'a>(&'a BindGroup);

impl<'a> PartialEq for BindGroupRef<'a> {
    fn eq(&self, other: &Self) -> bool {
        &*self.0 as *const BindGroup == &*other.0 as *const BindGroup
    }
}

impl<'a> Eq for BindGroupRef<'a> {}

impl<'a> Hash for BindGroupRef<'a> {
    fn hash<H: std::hash::Hasher>(&self, state: &mut H) {
        (&*self.0 as *const BindGroup).hash(state);
    }
}

pub struct Context<'a> {
    map: HashMap<BindGroupRef<'a>, u32>,
}

impl<'a> Context<'a> {
    pub fn insert(&mut self, bind_group: &'a BindGroup) {
        self.map.insert(BindGroupRef(bind_group), 1);
    }
}

Meta

rustc --version --verbose:

rustc 1.52.1 (9bc8c42bb 2021-05-09)
binary: rustc
commit-hash: 9bc8c42bb2f19e745a63f3445f1ac248fb015e53
commit-date: 2021-05-09
host: x86_64-pc-windows-msvc
release: 1.52.1
LLVM version: 12.0.0

This error also occurs on nightly, that last version that it worked on was 1.51. It seems that this only occurs if the crate-type is lib. Even if I remove a pub, it can compile successfully.

Error output

   Compiling ice_test v0.1.0 (C:\Users\David\Documents\GitHub\ice_test)
thread 'rustc' panicked at 'called `Option::unwrap()` on a `None` value', compiler\rustc_mir\src\monomorphize\collector.rs:746:93
stack backtrace:
   0:     0x7ff97f0681fe - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::hbc74f7e0bfeacc3c
   1:     0x7ff97f0932fc - core::fmt::write::h21444a3971da01ea
   2:     0x7ff97f05be68 - <std::io::IoSliceMut as core::fmt::Debug>::fmt::h826464ad0b49271e
   3:     0x7ff97f06c39d - std::panicking::take_hook::h3044e321765c5bc0
   4:     0x7ff97f06be69 - std::panicking::take_hook::h3044e321765c5bc0
   5:     0x7ff95bb6cd67 - rustc_driver::report_ice::h2924aece1396e3b2
   6:     0x7ff97f06cbf5 - std::panicking::rust_panic_with_hook::h9bc2ba6fb9e8afc8
   7:     0x7ff97f06c733 - rust_begin_unwind
   8:     0x7ff97f068b1f - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::hbc74f7e0bfeacc3c
   9:     0x7ff97f06c6b9 - rust_begin_unwind
  10:     0x7ff97f0c5630 - core::panicking::panic_fmt::h4afad618263d181d
  11:     0x7ff97f0c557c - core::panicking::panic::h9d4ec9844559842b
  12:     0x7ff95ece36ad - <rustc_mir::monomorphize::collector::MirNeighborCollector as rustc_middle::mir::visit::Visitor>::visit_terminator::hcdde1de3787208af
  13:     0x7ff95ece6dc2 - <rustc_mir::monomorphize::collector::RootCollector as rustc_hir::itemlikevisit::ItemLikeVisitor>::visit_impl_item::h5839b6f0482da285
  14:     0x7ff95ecdf603 - rustc_mir::monomorphize::collector::collect_crate_mono_items::hc869d40906e46d1f
  15:     0x7ff95ecdf832 - rustc_mir::monomorphize::collector::collect_crate_mono_items::hc869d40906e46d1f
  16:     0x7ff95ecdf832 - rustc_mir::monomorphize::collector::collect_crate_mono_items::hc869d40906e46d1f
  17:     0x7ff95ecdf832 - rustc_mir::monomorphize::collector::collect_crate_mono_items::hc869d40906e46d1f
  18:     0x7ff95ecdf832 - rustc_mir::monomorphize::collector::collect_crate_mono_items::hc869d40906e46d1f
  19:     0x7ff95ecdf832 - rustc_mir::monomorphize::collector::collect_crate_mono_items::hc869d40906e46d1f
  20:     0x7ff95ecddf88 - rustc_mir::monomorphize::collector::collect_crate_mono_items::hc869d40906e46d1f
  21:     0x7ff95efd8029 - rustc_mir::monomorphize::partitioning::partition::h3f2686721f839ef6        
  22:     0x7ff95e7b71ea - <rustc_ast_lowering::item::ItemLowerer as rustc_ast::visit::Visitor>::visit_foreign_item::hd54389a61e1546bf
  23:     0x7ff95e92e623 - <rustc_query_impl::Queries as rustc_middle::ty::query::QueryEngine>::try_print_query_stack::hd396f5d73fd6bafc
  24:     0x7ff95e9efc3b - <rustc_query_impl::plumbing::QueryCtxt as rustc_query_system::query::QueryContext>::store_diagnostics_for_anon_node::hef4094f9d7d4b7c1
  25:     0x7ff95e6da77b - <rustc_ast_lowering::item::ItemLowerer as rustc_ast::visit::Visitor>::visit_foreign_item::hd54389a61e1546bf
  26:     0x7ff95e68702f - <rustc_ast_lowering::item::ItemLowerer as rustc_ast::visit::Visitor>::visit_foreign_item::hd54389a61e1546bf
  27:     0x7ff95e8f7918 - <rustc_query_impl::Queries as rustc_middle::ty::query::QueryEngine>::try_print_query_stack::hd396f5d73fd6bafc
  28:     0x7ff95f12b06c - rustc_codegen_ssa::back::symbol_export::crates_export_threshold::h241c297691609457
  29:     0x7ff95e9239ae - <rustc_query_impl::Queries as rustc_middle::ty::query::QueryEngine>::try_print_query_stack::hd396f5d73fd6bafc
  30:     0x7ff95e9e6a05 - <rustc_query_impl::plumbing::QueryCtxt as rustc_query_system::query::QueryContext>::store_diagnostics_for_anon_node::hef4094f9d7d4b7c1
  31:     0x7ff95e6d45bd - <rustc_ast_lowering::item::ItemLowerer as rustc_ast::visit::Visitor>::visit_foreign_item::hd54389a61e1546bf
  32:     0x7ff95e5f1d6f - <rustc_ast_lowering::item::ItemLowerer as rustc_ast::visit::Visitor>::visit_foreign_item::hd54389a61e1546bf
  33:     0x7ff95e8f7842 - <rustc_query_impl::Queries as rustc_middle::ty::query::QueryEngine>::try_print_query_stack::hd396f5d73fd6bafc
  34:     0x7ff95f26cc95 - <rustc_metadata::rmeta::encoder::EncodeContext as rustc_middle::ty::codec::TyEncoder>::encode_alloc_id::h7a5710125ac2350a
  35:     0x7ff95f283767 - <rustc_metadata::rmeta::encoder::ImplVisitor as rustc_hir::itemlikevisit::ItemLikeVisitor>::visit_item::he60760398867c589
  36:     0x7ff95f2c99b5 - ZN14rustc_metadata5rmeta69_DERIVE_rustc_serialize_Encodable_EncodeContext_a_tcx_FOR_AssocFnData155_$LT$impl$u20$rustc_serialize..serialize..Encodable$LT$rustc_metadata..rmeta..encoder..EncodeContext$GT$$u20$for$u20$rustc_metadata..rmeta..AssocFnData$GT$6e
  37:     0x7ff95f292a62 - rustc_metadata::rmeta::decoder::cstore_impl::<impl rustc_middle::middle::cstore::CrateStore for rustc_metadata::creader::CStore>::encode_metadata::h1ed0b65c27604724
  38:     0x7ff95fa7c8a9 - rustc_middle::ty::context::TyCtxt::encode_metadata::ha5e017944f117061      
  39:     0x7ff95bcbea03 - rustc_interface::passes::BoxedResolver::to_resolver_outputs::h189f623607001da9
  40:     0x7ff95bccb4a6 - rustc_interface::queries::Queries::ongoing_codegen::h79b7cd4881f0c623      
  41:     0x7ff95bb85be7 - <rustc_middle::ty::SymbolName as core::fmt::Debug>::fmt::h7caf896a35c7e6af 
  42:     0x7ff95bb6eeec - <rustc_driver::Compilation as core::fmt::Debug>::fmt::h8df3c0fee143d8f3    
  43:     0x7ff95bb88177 - <rustc_middle::ty::SymbolName as core::fmt::Debug>::fmt::h7caf896a35c7e6af 
  44:     0x7ff95bb77364 - rustc_driver::pretty::print_after_hir_lowering::hcb64343913f571c9
  45:     0x7ff95bb88b8f - <rustc_middle::ty::SymbolName as core::fmt::Debug>::fmt::h7caf896a35c7e6af 
  46:     0x7ff95bba864d - <rustc_middle::ty::SymbolName as core::fmt::Debug>::fmt::h7caf896a35c7e6af 
  47:     0x7ff97f07bc9a - std::sys::windows::thread::Thread::new::he7957b99123a49eb
  48:     0x7ff9c5787034 - BaseThreadInitThunk
  49:     0x7ff9c7162651 - RtlUserThreadStart

error: internal compiler error: unexpected panic

note: the compiler unexpectedly panicked. this is a bug.

note: we would appreciate a bug report: https://github.com/rust-lang/rust/issues/new?labels=C-bug%2C+I-ICE%2C+T-compiler&template=ice.md

note: rustc 1.52.1 (9bc8c42bb 2021-05-09) running on x86_64-pc-windows-msvc

note: compiler flags: -C embed-bitcode=no -C debuginfo=2 -C incremental --crate-type lib

note: some of the compiler flags provided by cargo are hidden

query stack during panic:
#0 [collect_and_partition_mono_items] collect_and_partition_mono_items
#1 [exported_symbols] exported_symbols
end of query stack
error: could not compile `ice_test`

To learn more, run the command again with --verbose.

LingMan commented 3 years ago

Bisecting points to https://github.com/rust-lang/rust/pull/81172 as the culprit. cc @SimonSapin

searched nightlies: from nightly-2021-02-14 to nightly-2021-05-03 regressed nightly: nightly-2021-02-19 searched commits: from https://github.com/rust-lang/rust/commit/152f6609246558be5e2582e67376194815e6ba0d to https://github.com/rust-lang/rust/commit/0148b971c921a0831fbf3357e5936eec724e3566 regressed commit: https://github.com/rust-lang/rust/commit/d1462d8558cf4551608457f63d9b999185ebf3bf

bisected with cargo-bisect-rustc v0.6.0 Host triple: x86_64-unknown-linux-gnu Reproduce with: ```bash cargo bisect-rustc --start=2021-02-14 --end=2021-05-03 ```
SimonSapin commented 3 years ago

Reducing a bit more:

#![crate_type = "lib"]

use std::{collections::HashMap, hash::Hash};

struct C;

trait Ctx {
    type BindGroupId;
}

impl Ctx for C {
    type BindGroupId = u32;
}

pub struct BindGroup {
    _id: <C as Ctx>::BindGroupId,
}

pub fn insert(map: &mut HashMap<*const BindGroup, u32>, bind_group: *const BindGroup) {
    map.insert(bind_group, 1);
}

However, if I remove insert and replace the HashMap with only calling hash() I don’t reproduce the ICE anymore:

pub fn hash(bind_group: *const BindGroup, state: &mut impl std::hash::Hasher) {
    bind_group.hash(state)
}

The panic message points to the second unwrap in:

https://github.com/rust-lang/rust/blob/9bc8c42bb2f19e745a63f3445f1ac248fb015e53/compiler/rustc_mir/src/monomorphize/collector.rs#L746

… which means that rustc_middle::ty::instance::Instance::resolve returns Ok(None).

Because https://github.com/rust-lang/rust/pull/81172 and a HashMap are involved, I suspected that it might be <*const BindGroup as Hash>::hash that fails to be resolved to a concrete function because something fails to resolve <BindGroup as Pointee>::Metadata through the <C as Ctx>::BindGroupId associated type. https://github.com/rust-lang/rust/pull/81172/commits/696b239f72350ce2a647ede1a330039d0e0ecfa9#diff-4864ab6ed9e5cec3c6d0d561211554aafbd0094842869d2a711b5451151bc534R2139 is the part of #81172 relevant to that.

https://github.com/rust-lang/rust/blob/9bc8c42bb2f19e745a63f3445f1ac248fb015e53/library/core/src/hash/mod.rs#L691-L698

However my failed reduction says that might not be it. I’m not sure what to try next. Maybe someone more familiar with compiler internals can comment on how Instance::resolve returning Ok(None).

apiraino commented 3 years ago

Nominating for discussion during T-compiler meeting to get more eyes on this (cc: @SimonSapin feel free to jump in)

steffahn commented 3 years ago

Some more reduced code examples can be found in this related/duplicate issue. #86469

There’s also a case of hitting a slightly different ICE

error: internal compiler error: compiler/rustc_codegen_llvm/src/debuginfo/metadata.rs:704:14: debuginfo: unexpected type in type_metadata: <Struct as std::ptr::Pointee>::Metadata

thread 'rustc' panicked at 'Box<dyn Any>', compiler/rustc_errors/src/lib.rs:1007:9

here: https://github.com/rust-lang/rust/issues/86469#issuecomment-864674629

SimonSapin commented 3 years ago

Ouch, I forgot about this and the regression has reached Stable :/ However I still don’t know how to proceed as I’m not at all familiar with these compiler internals. Any help would be appreciated.

The slightly different ICE points to:

https://github.com/rust-lang/rust/blob/a85f584aebd9b08314bf30b9adc17b4a752143e5/compiler/rustc_codegen_llvm/src/debuginfo/metadata.rs#L704

I think this file is about some other kind of metadata, not https://github.com/rust-lang/rust/issues/81513 pointer metadata? But the underlying issue seems to be similar: some code receives a Ty that represents an unresolved <Foo as Pointee>::Metadata associated type projection but expected that to be resolved earlier to a concrete type like () or usize.

pnkfelix commented 3 years ago

Hey @SimonSapin , I self-assigned this and then didn't follow up. I'm looking at it more carefully now.

pnkfelix commented 3 years ago

Looking at the reduction from https://github.com/rust-lang/rust/issues/85447#issuecomment-843879967, I added a bit of instrumentation to the compiler to see which ty is causing resolution to return Ok(None). Answer:

variable debug output
ty for<'r, 's> fn(&'r <BindGroup as std::ptr::Pointee>::Metadata, &'s mut std::collections::hash_map::DefaultHasher) {<<BindGroup as std::ptr::Pointee>::Metadata as std::hash::Hash>::hash::<std::collections::hash_map::DefaultHasher>}
def_id DefId(2:9096 ~ core[3e18]::hash::Hash::hash)
substs [<BindGroup as std::ptr::Pointee>::Metadata, std::collections::hash_map::DefaultHasher]

If nothing else, this lends evidence to @SimonSapin 's notes above: we're trying to resolve a trait method, Hash::hash, via a substitution where the Self type is an associated item.

So my current guess is that we're missing some extra resolution path somewhere to convert the <BindGroup as std::ptr::Pointee>::Metadata into a more concrete type that can be used for the necessary method resolution.

Update: I mostly added this note because of something that I didn't mention in the text above: The presence of the higher-kinded type for<'r, 's> fn(...) could be part of the reason that we are unexpectedly encountering an unresolved type here.

pnkfelix commented 3 years ago

Update: I mostly added this note because of something that I didn't mention in the text above: The presence of the higher-kinded type for<'r, 's> fn(...) could be part of the reason that we are unexpectedly encountering an unresolved type here.

Hmm, no, this doesn't seem like a sufficient explanation, at least not in this trivialized form. Here's why: I took a whirl at running the same test case on a build immediately prior to PR #81172, and in the debug output from that run, I see successful visit_fn_use direct resolution calls on higher-kinded types, including the exact an analogous one of interest here: 

DEBUG rustc_mir::monomorphize::collector visit_fn_use resolve direct 
ty: for<'r, 's> fn(&'r BindGroupRef, &'s mut std::collections::hash_map::DefaultHasher) {<BindGroupRef as std::hash::Hash>::hash::<std::collections::hash_map::DefaultHasher>} 
= def_id: DefId(2:7145 ~ core[1f15]::hash::Hash::hash) 
with substs: [BindGroupRef, std::collections::hash_map::DefaultHasher]

Update: Of course the ty is not "the exact" one of interest. The whole point is that after this PR, we are seeing attempts to resolve like: visit_fn_use resolve direct ty: for<'r, 's> fn(&'r <BindGroup as std::ptr::Pointee>::Metadata, &'s mut std::collections::hash_map::DefaultHasher), while before PR #81172, it was visit_fn_use resolve direct ty: for<'r, 's> fn(&'r BindGroupRef, &'s mut std::collections::hash_map::DefaultHasher)

I'm just restating that the HKT alone isn't the reason for the resolution .unwrap().unwrap() failure. (Its entirely possible that the HKT is an issue somewhere upstream. Or maybe the construction from PR #81172 is just not quite right.)

nikomatsakis commented 3 years ago

@pnkfelix and I dug into this. The problem is exactly this FIXME here:

https://github.com/rust-lang/rust/blob/ee86f96ba176f598d64dc9f3bb7e074d5b8b86b6/compiler/rustc_trait_selection/src/traits/project.rs#L1058

In particular, the BindGroup struct has a single field, and it is a (normalizable) projection:

pub struct BindGroup {
    _id: <C as Ctx>::BindGroupId,
}

The Instance::resolve code expect that, by the time we get to monomorphization, we are able to resolve all associated types -- however, in this case, we are not, because project.rs sees the tail field as <C as Ctx>::BindGroupId, doesn't attempt to normalize, and then decides there is no candidate for projection.

Fixing this is not entirely trivial, I am debating the best approach. The problem is that the "assemble candidates" logic isn't really meant to normalize, which is kind of a side-effecting operation that produces subobligations. But it should be possible to do -- I suppose we could look at what select.rs does here. I think maybe it uses a "probe" to do the normalization and then throw away the result?

(Egads I would like to convert this code to use chalk!)

SimonSapin commented 3 years ago

The problem is exactly this FIXME here:

When writing this comment as part of https://github.com/rust-lang/rust/pull/81172 I remember feeling very much feeling like the "no idea what I’m doing" meme. I mostly imitated what the compiler already did for the DiscriminantKind trait, which is also automagically implemented for every type and also has an associated type.

Does DiscriminantKind not have a similar problem?

apiraino commented 3 years ago

I think I can remove the I-nominate, was discussed on Zulip a while ago, T-compiler followed up on this

@rustbot label -I-nominated

Alexendoo commented 2 years ago

Fixed by #92248