Incremental compilation error with evaluate_obligation: Ok(EvaluatedToOkModuloRegions)

Imberflur commented 3 years ago

I was hoping this would be fixed by https://github.com/rust-lang/rust/pull/85186 but we are still experiencing this issue on the 2021-05-15 nightly

Code

Original case: https://gitlab.com/veloren/veloren/-/tree/imbris/update-toolchain

Reduced case: https://github.com/Imberflur/incremental-ice-with-specs

// lib.rs

use specs::{
    shred::ResourceId, Component, ParJoin, ReadStorage, System, SystemData, World,
};

pub struct Pos;

impl Component for Pos {
    type Storage = specs::VecStorage<Self>;
}

pub struct InterpBuffer<T> {
    _t: T,
}

impl<T: 'static + Send + Sync> Component for InterpBuffer<T> {
    type Storage = specs::VecStorage<Self>;
}

#[derive(SystemData)]
pub struct ReadData<'a> {
    pos_interpdata: ReadStorage<'a, InterpBuffer<Pos>>,
}

#[derive(Default)]
pub struct Sys;

impl<'a> System<'a> for Sys {
    type SystemData = (ReadData<'a>, ReadStorage<'a, Pos>);

    fn run(&mut self, (data, pos): Self::SystemData) {
        (&pos, &data.pos_interpdata).par_join();
    }
}

# Cargo.toml

[dependencies.specs]
version = "0.16.1"
git = "https://github.com/amethyst/specs"
rev = "1a113a85fb6283d23294a58c2ef749d41cad5deb"
features = ["derive"]

Error output (original case)

   Compiling veloren-common-systems v0.9.0 (/home/projects/veloren2/common/systems)
   Compiling veloren-world v0.9.0 (/home/projects/veloren2/world)
   Compiling veloren-client v0.9.0 (/home/projects/veloren2/client)
error: internal compiler error: encountered incremental compilation error with evaluate_obligation(d2f6f8c07ccd5d49-bd504c819d17ce51)
  |
  = help: This is a known issue with the compiler. Run `cargo clean -p veloren_common_systems` or `cargo clean` to allow your project to compile
  = note: Please follow the instructions below to create a bug report with the provided information
  = note: See <https://github.com/rust-lang/rust/issues/84970> for more information

thread 'rustc' panicked at 'Found unstable fingerprints for evaluate_obligation(d2f6f8c07ccd5d49-bd504c819d17ce51): Ok(EvaluatedToOkModuloRegions)', /rustc/8cf990c9b5c59f25c806fad9f4466f9d6509bbea/compiler/rustc_query_system/src/query/plumbing.rs:619:9

note: compiler flags: -C opt-level=2 -C panic=abort -C embed-bitcode=no -C codegen-units=8 -C debug-assertions=on -C incremental -C link-arg=-fuse-ld=gold --crate-type lib

note: some of the compiler flags provided by cargo are hidden

Backtrace

query stack during panic

``` #0 [evaluate_obligation] evaluating trait selection obligation `specs::storage::MaskedStorage>: std::marker::Sized` #1 [normalize_projection_ty] normalizing `Canonical { max_universe: U0, variables: [], value: ParamEnvAnd { param_env: ParamEnv { caller_bounds: [], reveal: All }, value: ProjectionTy { substs: [rayon::iter::MapInit, &mut specs::Storage>>, &specs::Storage, specs::shred::Fetch>>>, &specs::Storage, specs::shred::Fetch>>>)>, [closure@common/systems/src/interpolation.rs:49:21: 49:62]>, [closure@common/systems/src/interpolation.rs:51:17: 54:18], [closure@common/systems/src/interpolation.rs:55:17: 57:18]>], item_def_id: DefId(83:4492 ~ rayon[db88]::iter::ParallelIterator::Item) } } }` #2 [normalize_generic_arg_after_erasing_regions] normalizing `fn(rayon::iter::MapInit, &mut specs::Storage>>, &specs::Storage, specs::shred::Fetch>>>, &specs::Storage, specs::shred::Fetch>>>)>, [closure@common/systems/src/interpolation.rs:49:21: 49:62]>, [closure@common/systems/src/interpolation.rs:51:17: 54:18], [closure@common/systems/src/interpolation.rs:55:17: 57:18]>) {<() as rayon::iter::FromParallelIterator<, &mut specs::Storage>>, &specs::Storage, specs::shred::Fetch>>>, &specs::Storage, specs::shred::Fetch>>>)>, [closure@common/systems/src/interpolation.rs:49:21: 49:62]>, [closure@common/systems/src/interpolation.rs:51:17: 54:18], [closure@common/systems/src/interpolation.rs:55:17: 57:18]> as rayon::iter::ParallelIterator>::Item>>::from_par_iter::, &mut specs::Storage>>, &specs::Storage, specs::shred::Fetch>>>, &specs::Storage, specs::shred::Fetch>>>)>, [closure@common/systems/src/interpolation.rs:49:21: 49:62]>, [closure@common/systems/src/interpolation.rs:51:17: 54:18], [closure@common/systems/src/interpolation.rs:55:17: 57:18]>>}` #3 [collect_and_partition_mono_items] collect_and_partition_mono_items ```

Error output (reduced case)

error: internal compiler error: encountered incremental compilation error with evaluate_obligation(cf1bd6537d51543b-849a581f5e8ac13b)
  |
  = help: This is a known issue with the compiler. Run `cargo clean -p ice_test` or `cargo clean` to allow your project to compile
  = note: Please follow the instructions below to create a bug report with the provided information
  = note: See <https://github.com/rust-lang/rust/issues/84970> for more information

thread 'rustc' panicked at 'Found unstable fingerprints for evaluate_obligation(cf1bd6537d51543b-849a581f5e8ac13b): Ok(EvaluatedToOkModuloRegions)', /rustc/8cf990c9b5c59f25c806fad9f4466f9d6509bbea/compiler/rustc_query_system/src/query/plumbing.rs:619:9

Backtrace

``` 0: rust_begin_unwind at /rustc/8cf990c9b5c59f25c806fad9f4466f9d6509bbea/library/std/src/panicking.rs:493:5 1: std::panicking::begin_panic_fmt at /rustc/8cf990c9b5c59f25c806fad9f4466f9d6509bbea/library/std/src/panicking.rs:435:5 2: rustc_query_system::query::plumbing::incremental_verify_ich 3: rustc_query_system::query::plumbing::load_from_disk_and_cache_in_memory 4: rustc_data_structures::stack::ensure_sufficient_stack 5: rustc_query_system::query::plumbing::get_query_impl 6: ::evaluate_obligation 7: ::evaluate_obligation 8: ::evaluate_obligation_no_overflow 9: rustc_trait_selection::traits::fulfill::FulfillProcessor::process_trait_obligation 10: rustc_trait_selection::traits::fulfill::FulfillProcessor::progress_changed_obligations 11: rustc_data_structures::obligation_forest::ObligationForest::process_obligations 12: ::select_where_possible 13: ::select_all_or_error 14: rustc_trait_selection::traits::codegen::drain_fulfillment_cx_or_panic 15: rustc_infer::infer::InferCtxtBuilder::enter 16: rustc_trait_selection::traits::codegen::codegen_fulfill_obligation 17: rustc_query_impl:: for rustc_query_impl::queries::codegen_fulfill_obligation>::compute 18: rustc_middle::dep_graph::::with_deps 19: rustc_query_system::dep_graph::graph::DepGraph::with_task_impl 20: rustc_data_structures::stack::ensure_sufficient_stack 21: rustc_query_system::query::plumbing::force_query_with_job 22: rustc_query_system::query::plumbing::get_query_impl 23: ::codegen_fulfill_obligation 24: rustc_ty_utils::instance::inner_resolve_instance 25: rustc_ty_utils::instance::resolve_instance 26: rustc_query_impl:: for rustc_query_impl::queries::resolve_instance>::compute 27: rustc_middle::dep_graph::::with_deps 28: rustc_query_system::dep_graph::graph::DepGraph::with_task_impl 29: rustc_data_structures::stack::ensure_sufficient_stack 30: rustc_query_system::query::plumbing::force_query_with_job 31: rustc_query_system::query::plumbing::get_query_impl 32: ::resolve_instance 33: rustc_middle::ty::instance::Instance::resolve_opt_const_arg 34: rustc_middle::ty::instance::Instance::resolve 35: ::visit_terminator 36: rustc_mir::monomorphize::collector::collect_neighbours 37: rustc_mir::monomorphize::collector::collect_items_rec 38: rustc_mir::monomorphize::collector::collect_items_rec 39: rustc_session::utils::::time 40: rustc_mir::monomorphize::collector::collect_crate_mono_items 41: rustc_mir::monomorphize::partitioning::collect_and_partition_mono_items 42: rustc_query_impl:: for rustc_query_impl::queries::collect_and_partition_mono_items>::compute 43: rustc_middle::dep_graph::::with_deps 44: rustc_query_system::dep_graph::graph::DepGraph::with_task_impl 45: rustc_data_structures::stack::ensure_sufficient_stack 46: rustc_query_system::query::plumbing::force_query_with_job 47: rustc_query_system::query::plumbing::get_query_impl 48: ::collect_and_partition_mono_items 49: rustc_codegen_ssa::back::symbol_export::exported_symbols_provider_local 50: rustc_middle::dep_graph::::with_deps 51: rustc_query_system::dep_graph::graph::DepGraph::with_task_impl 52: rustc_data_structures::stack::ensure_sufficient_stack 53: rustc_query_system::query::plumbing::force_query_with_job 54: rustc_query_system::query::plumbing::get_query_impl 55: ::exported_symbols 56: rustc_metadata::rmeta::encoder::EncodeContext::encode_crate_root 57: rustc_metadata::rmeta::encoder::encode_metadata_impl 58: rustc_data_structures::sync::join 59: rustc_metadata::rmeta::decoder::cstore_impl::::encode_metadata 60: rustc_middle::ty::context::TyCtxt::encode_metadata 61: rustc_interface::passes::QueryContext::enter 62: rustc_interface::queries::Queries::ongoing_codegen 63: rustc_interface::queries::::enter 64: rustc_span::with_source_map 65: rustc_interface::interface::create_compiler_and_run 66: scoped_tls::ScopedKey::set ```

query stack during panic

``` #0 [evaluate_obligation] evaluating trait selection obligation `specs::storage::MaskedStorage>: std::marker::Sized` #1 [codegen_fulfill_obligation] checking if `std::ops::Deref` fulfills its obligations #2 [resolve_instance] resolving instance `>> as std::ops::Deref>::deref` #3 [collect_and_partition_mono_items] collect_and_partition_mono_items #4 [exported_symbols] exported_symbols ```

Aaron1011 commented 3 years ago

This is caused by the way that we cache projection obligations. During candidate confirmation, we call collect_predicates_for_types, which causes us to register obligations arising from trying to normalize types:

https://github.com/rust-lang/rust/blob/7dc9ff5c629753b6930ecfe9a0446538b8e25fb7/compiler/rustc_trait_selection/src/traits/select/mod.rs#L1785-L1794

The first time we normalize a type, opt_normalize_projection_type will produce all required sub-obligations, which we then try to select. However, once we've cached a normalization result, we may skip returning the projection sub-obligations:

https://github.com/rust-lang/rust/blob/7dc9ff5c629753b6930ecfe9a0446538b8e25fb7/compiler/rustc_trait_selection/src/traits/project.rs#L533-L537

In the reproducer for this issue, the projection of <InterpBuffer<Pos> as specs::Component>::Storage ends up generating the sub-obligation Pos: 'static, due to the T: 'static bound on the impl. This obligation will generate EvaluatedToOkModulo regions if we try to select it, which will end up 'infecting' the root evaluate_obligation(specs::storage::MaskedStorage<InterpBuffer<Pos>>: std::marker::Sized) call. However, if we've cached the result of this projection before we reach our evaluate_obligation call (which happens the first time we compile the reproducer) then we will not try to process the sub-obligation Pos: 'static, giving us a result of EvaluatedToOk.

Aaron1011 commented 3 years ago

This issue has now been fixed. However, we still need a minimized regression test.

Imberflur commented 3 years ago

@Aaron1011 I can still produce an ICE with this on the latest nightly: rustc 1.54.0-nightly (5dc8789e3 2021-05-21) running on x86_64-unknown-linux-gnu

thread 'rustc' panicked at 'Found unstable fingerprints for evaluate_obligation(cf1bd6537d51543b-849a581f5e8ac13b): Ok(EvaluatedToOkModuloRegions)', /rustc/5dc8789e300930751a78996da0fa906be5a344a2/compiler/rustc_query_system/src/query/plumbing.rs:619:9

although perhaps the query stack has changed a bit:

query stack during panic:
#0 [evaluate_obligation] evaluating trait selection obligation `specs::storage::MaskedStorage<InterpBuffer<Pos>>: std::marker::Sized`
#1 [is_sized_raw] computing whether `specs::storage::MaskedStorage<InterpBuffer<Pos>>` is `Sized`
#2 [collect_and_partition_mono_items] collect_and_partition_mono_items
#3 [exported_symbols] exported_symbols

backtrace

``` stack backtrace: 0: rust_begin_unwind at /rustc/5dc8789e300930751a78996da0fa906be5a344a2/library/std/src/panicking.rs:515:5 1: std::panicking::begin_panic_fmt at /rustc/5dc8789e300930751a78996da0fa906be5a344a2/library/std/src/panicking.rs:457:5 2: rustc_query_system::query::plumbing::incremental_verify_ich 3: rustc_query_system::query::plumbing::load_from_disk_and_cache_in_memory 4: rustc_data_structures::stack::ensure_sufficient_stack 5: rustc_query_system::query::plumbing::get_query_impl 6: ::evaluate_obligation 7: ::evaluate_obligation 8: ::evaluate_obligation_no_overflow 9: rustc_trait_selection::traits::type_known_to_meet_bound_modulo_regions 10: rustc_infer::infer::InferCtxtBuilder::enter 11: rustc_ty_utils::common_traits::is_sized_raw 12: rustc_middle::dep_graph::::with_deps 13: rustc_query_system::query::plumbing::load_from_disk_and_cache_in_memory 14: rustc_data_structures::stack::ensure_sufficient_stack 15: rustc_query_system::query::plumbing::get_query_impl 16: ::is_sized_raw 17: rustc_middle::ty::util::::is_sized 18: rustc_mir::monomorphize::collector::find_vtable_types_for_unsizing::{{closure}} 19: rustc_mir::monomorphize::collector::find_vtable_types_for_unsizing 20: ::visit_rvalue 21: rustc_mir::monomorphize::collector::collect_neighbours 22: rustc_mir::monomorphize::collector::collect_items_rec 23: rustc_mir::monomorphize::collector::collect_items_rec 24: rustc_mir::monomorphize::collector::collect_items_rec 25: rustc_mir::monomorphize::collector::collect_items_rec 26: rustc_mir::monomorphize::collector::collect_items_rec 27: rustc_session::utils::::time 28: rustc_mir::monomorphize::collector::collect_crate_mono_items 29: rustc_mir::monomorphize::partitioning::collect_and_partition_mono_items 30: rustc_query_impl:: for rustc_query_impl::queries::collect_and_partition_mono_items>::compute 31: rustc_middle::dep_graph::::with_deps 32: rustc_query_system::dep_graph::graph::DepGraph::with_task_impl 33: rustc_data_structures::stack::ensure_sufficient_stack 34: rustc_query_system::query::plumbing::force_query_with_job 35: rustc_query_system::query::plumbing::force_query_impl 36: rustc_query_system::query::plumbing::force_query 37: rustc_query_impl::query_callbacks::collect_and_partition_mono_items::force_from_dep_node 38: rustc_query_system::dep_graph::graph::DepGraph::try_mark_previous_green 39: rustc_query_system::dep_graph::graph::DepGraph::try_mark_green_and_read 40: rustc_data_structures::stack::ensure_sufficient_stack 41: rustc_query_system::query::plumbing::get_query_impl 42: ::exported_symbols 43: rustc_metadata::rmeta::encoder::EncodeContext::encode_crate_root 44: rustc_metadata::rmeta::encoder::encode_metadata_impl 45: rustc_data_structures::sync::join 46: rustc_metadata::rmeta::decoder::cstore_impl::::encode_metadata 47: rustc_middle::ty::context::TyCtxt::encode_metadata 48: rustc_interface::passes::QueryContext::enter 49: rustc_interface::queries::Queries::ongoing_codegen 50: rustc_interface::queries::::enter 51: rustc_span::with_source_map 52: scoped_tls::ScopedKey::set ```

Aaron1011 commented 3 years ago

You're right - it looks like I didn't fully test that fix. I'll continue to investigate.

Imberflur commented 3 years ago

I managed to remove the specs dependency! https://github.com/Imberflur/incremental-ice-with-specs/blob/93829d0f72eb1296bf31fe37c486a9155e1ec4ec/src/lib.rs

use core::any::Any;
use core::marker::PhantomData;

struct DerefWrap<T>(T);

impl<T> core::ops::Deref for DerefWrap<T> {
    type Target = T;
    fn deref(&self) -> &Self::Target {
        &self.0
    }
}

struct Storage<T, D> {
    phantom: PhantomData<(T, D)>,
}

type ReadStorage<T> = Storage<T, DerefWrap<MaskedStorage<T>>>;

pub trait Component {
    type Storage;
}

struct VecStorage;

struct Pos;

impl Component for Pos {
    type Storage = VecStorage;
}

struct GenericComp<T> {
    _t: T,
}

impl<T: 'static> Component for GenericComp<T> {
    type Storage = VecStorage;
}
struct ReadData {
    pos_interpdata: ReadStorage<GenericComp<Pos>>,
}

trait System {
    type SystemData;

    fn run(data: Self::SystemData, any: Box<dyn Any>);
}

struct Sys;

impl System for Sys {
    type SystemData = (ReadData, ReadStorage<Pos>);

    fn run((data, pos): Self::SystemData, any: Box<dyn Any>) {
        <ReadStorage<GenericComp<Pos>> as SystemData>::setup(any);

        ParJoin::par_join((&pos, &data.pos_interpdata));
    }
}

trait ParJoin {
    fn par_join(self)
    where
        Self: Sized,
    {
    }
}

impl<'a, T, D> ParJoin for &'a Storage<T, D>
where
    T: Component,
    D: core::ops::Deref<Target = MaskedStorage<T>>,
    T::Storage: Sync,
{
}

impl<A, B> ParJoin for (A, B)
where
    A: ParJoin,
    B: ParJoin,
{
}

pub trait SystemData {
    fn setup(any: Box<dyn Any>);
}

impl<T: 'static> SystemData for ReadStorage<T>
where
    T: Component,
{
    fn setup(any: Box<dyn Any>) {
        let storage: &MaskedStorage<T> = any.downcast_ref().unwrap();

        <dyn Any as CastFrom<MaskedStorage<T>>>::cast(&storage);
    }
}

pub struct MaskedStorage<T: Component> {
    _inner: T::Storage,
}

pub unsafe trait CastFrom<T> {
    fn cast(t: &T) -> &Self;
}

unsafe impl<T> CastFrom<T> for dyn Any
where
    T: Any + 'static,
{
    fn cast(t: &T) -> &Self {
        t
    }
}

Edit: removed MetaTable

Aaron1011 commented 3 years ago

See https://github.com/rust-lang/rust/pull/85868 and https://rust-lang.zulipchat.com/#narrow/stream/144729-wg-traits/topic/projection.20cache.20and.20subobligations.20.2377325/near/241147624 for discussion about a fix.

Imberflur commented 3 years ago

@Aaron1011 is this blocked on finding a solution to the conflict you mention in the linked discussion?

Aaron1011 commented 3 years ago

@Imberflur - that's right. I'm hoping that we'll be able to get to this soon.

pnkfelix commented 3 years ago

cc #84970

vi commented 3 years ago

Is it the same issue or a separate?

error: internal compiler error: encountered incremental compilation error with evaluate_obligation(da800f1097592f6e-98d6b04e2aecd25a)
  |
  = help: This is a known issue with the compiler. Run `cargo clean -p rtcexperiment` or `cargo clean` to allow your project to compile
  = note: Please follow the instructions below to create a bug report with the provided information
  = note: See <https://github.com/rust-lang/rust/issues/84970> for more information

thread 'rustc' panicked at 'Found unstable fingerprints for evaluate_obligation(da800f1097592f6e-98d6b04e2aecd25a): Ok(EvaluatedToOk)', /rustc/08095fc1f875c89e507f17cf6c6a780c8ffa4c01/compiler/rustc_query_system/src/query/plumbing.rs:624:9
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

error: internal compiler error: unexpected panic

note: the compiler unexpectedly panicked. this is a bug.

note: we would appreciate a bug report: https://github.com/bjorn3/rustc_codegen_cranelift/issues/new

note: rustc 1.56.0-nightly (08095fc1f 2021-07-26) running on x86_64-unknown-linux-gnu

note: compiler flags: -C embed-bitcode=no -C debuginfo=2 -C incremental --crate-type bin

note: some of the compiler flags provided by cargo are hidden
query stack during panic:
#0 [evaluate_obligation] evaluating trait selection obligation `axum::routing::RouteFuture<axum::service::HandleError<tower_http::services::ServeFile, [closure@src/main.rs:18:65: 23:6], hyper::Body>, axum::routing::EmptyRouter, hyper::Body>: std::marker::Sized`
#1 [type_op_prove_predicate] evaluating `type_op_prove_predicate` `Canonical { max_universe: U0, variables: [], value: ParamEnvAnd { param_env: ParamEnv { caller_bounds: [], reveal: UserFacing }, value: ProvePredicate { predicate: Binder(TraitPredicate(<hyper::common::exec::Exec as hyper::common::exec::NewSvcExec<hyper::server::conn::AddrStream, tower::make::make_service::shared::SharedFuture<axum::routing::Route<axum::service::HandleError<tower_http::services::ServeFile, [closure@src/main.rs:18:65: 23:6], hyper::Body>, axum::routing::Route<axum::service::HandleError<tower_http::services::ServeFile, [closure@src/main.rs:18:65: 23:6], hyper::Body>, axum::routing::EmptyRouter>>>, axum::routing::Route<axum::service::HandleError<tower_http::services::ServeFile, [closure@src/main.rs:18:65: 23:6], hyper::Body>, axum::routing::Route<axum::service::HandleError<tower_http::services::ServeFile, [closure@src/main.rs:18:65: 23:6], hyper::Body>, axum::routing::EmptyRouter>>, hyper::common::exec::Exec, hyper::server::conn::spawn_all::NoopWatcher>>), []) } } }`
end of query stack

Note that cg_clif is in use.

Aaron1011 commented 3 years ago

@vi It looks like that's the same issue.

tmpolaczyk commented 3 years ago

Run into this in 1.54 stable.

rust-lang / rust