search

rust-lang / rust

Empowering everyone to build reliable and efficient software.
https://www.rust-lang.org
Other
97.97k stars 12.69k forks source link

internal compiler error using lang items #79559

Closed jdonszelmann closed 3 years ago

jdonszelmann commented 3 years ago

Code

This bug is isolated to nightly as lang items aren't yet supported in beta and up.

The code for this bug comes as far as I tested down to these two parts:

First some library crate with this in it:

(crate = langstart, src/lib.rs)

use core::panic::PanicInfo;

#[lang = "start"]
#[no_mangle]
fn lang_start(
    main: fn() -> (),
    _argc: isize,
    _argv: *const *const u8,
) -> isize {
    main();

    return 0;
}

#[lang = "eh_personality"] extern fn rust_eh_personality() {}
#[lang = "panic_impl"] extern fn rust_begin_panic(_info: &PanicInfo) -> ! {
    panic!("Yeet!");
}

#[lang = "oom"]
fn rust_alloc_error(_: core::alloc::Layout) -> ! {
    panic!("Yeet oom!");
}

(basically a no_std program start similar to how it's done in std)

and then in some other crate (In the example I'll give it's a example of the library):

#![no_std]
use langstart;

fn main( ){

}

I'm not quite sure if it's correct to use lang items in libraries, but in any case it should not give an internal compiler error.

I created a minimal repository demonstrating the bug. To run use 

cargo run --example teststart

https://github.com/jonay2000/langstartbug

Meta

rustc --version --verbose:

rustc 1.50.0-nightly (1c389ffef 2020-11-24)
binary: rustc
commit-hash: 1c389ffeff814726dec325f0f2b0c99107df2673
commit-date: 2020-11-24
host: x86_64-unknown-linux-gnu
release: 1.50.0-nightly

Error output

error: internal compiler error: compiler/rustc_metadata/src/rmeta/decoder.rs:1212:17: get_optimized_mir: missing MIR for `DefId(4:5 ~ langstart[abf0]::lang_start)`

thread 'rustc' panicked at 'Box<Any>', compiler/rustc_errors/src/lib.rs:958:9
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

note: the compiler unexpectedly panicked. this is a bug.

note: we would appreciate a bug report: https://github.com/rust-lang/rust/issues/new?labels=C-bug%2C+I-ICE%2C+T-compiler&template=ice.md

note: rustc 1.50.0-nightly (1c389ffef 2020-11-24) running on x86_64-unknown-linux-gnu

note: compiler flags: -C embed-bitcode=no -C debuginfo=2 -C incremental --crate-type bin

note: some of the compiler flags provided by cargo are hidden

query stack during panic:
#0 [optimized_mir] optimizing MIR for `langstart::lang_start`
#1 [collect_and_partition_mono_items] collect_and_partition_mono_items
end of query stack
error: aborting due to previous error; 1 warning emitted

error: could not compile `langstart`

To learn more, run the command again with --verbose.
Backtrace

``` thread 'rustc' panicked at 'Box', compiler/rustc_errors/src/lib.rs:958:9 stack backtrace: 0: 0x7f462841fe30 - std::backtrace_rs::backtrace::libunwind::trace::h746c3e9529d524bc at /rustc/1c389ffeff814726dec325f0f2b0c99107df2673/library/std/src/../../backtrace/src/backtrace/libunwind.rs:90:5 1: 0x7f462841fe30 - std::backtrace_rs::backtrace::trace_unsynchronized::h86340908ff889faa at /rustc/1c389ffeff814726dec325f0f2b0c99107df2673/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5 2: 0x7f462841fe30 - std::sys_common::backtrace::_print_fmt::h43f85f9b18230404 at /rustc/1c389ffeff814726dec325f0f2b0c99107df2673/library/std/src/sys_common/backtrace.rs:67:5 3: 0x7f462841fe30 - ::fmt::hc132ae1a5b5aa7cd at /rustc/1c389ffeff814726dec325f0f2b0c99107df2673/library/std/src/sys_common/backtrace.rs:46:22 4: 0x7f4628492c4c - core::fmt::write::hdf023a0036d2a25f at /rustc/1c389ffeff814726dec325f0f2b0c99107df2673/library/core/src/fmt/mod.rs:1078:17 5: 0x7f46284119a2 - std::io::Write::write_fmt::h8580846154bcb66a at /rustc/1c389ffeff814726dec325f0f2b0c99107df2673/library/std/src/io/mod.rs:1519:15 6: 0x7f4628423a95 - std::sys_common::backtrace::_print::h7ee55fed88d107a3 at /rustc/1c389ffeff814726dec325f0f2b0c99107df2673/library/std/src/sys_common/backtrace.rs:49:5 7: 0x7f4628423a95 - std::sys_common::backtrace::print::h54a7d3e52a524177 at /rustc/1c389ffeff814726dec325f0f2b0c99107df2673/library/std/src/sys_common/backtrace.rs:36:9 8: 0x7f4628423a95 - std::panicking::default_hook::{{closure}}::h60921e857bf55a40 at /rustc/1c389ffeff814726dec325f0f2b0c99107df2673/library/std/src/panicking.rs:208:50 9: 0x7f46284235ea - std::panicking::default_hook::hf0f9afb1017317fc at /rustc/1c389ffeff814726dec325f0f2b0c99107df2673/library/std/src/panicking.rs:225:9 10: 0x7f4628cb0bb8 - rustc_driver::report_ice::ha25ae86a5858acc3 11: 0x7f4628424396 - std::panicking::rust_panic_with_hook::h8d66bf42b407aaea at /rustc/1c389ffeff814726dec325f0f2b0c99107df2673/library/std/src/panicking.rs:595:17 12: 0x7f462bdd96bd - std::panicking::begin_panic::{{closure}}::he3ac55d11a883a10 13: 0x7f462bdd93c6 - std::sys_common::backtrace::__rust_end_short_backtrace::h4402bc3ed558879b 14: 0x7f462bdd965f - std::panicking::begin_panic::hd2137c659c375844 15: 0x7f462be13bdc - rustc_errors::HandlerInner::bug::hbfb11e3c8ba1475f 16: 0x7f462be122f0 - rustc_errors::Handler::bug::ha00f48e1291906a1 17: 0x7f462b6ed524 - rustc_middle::util::bug::opt_span_bug_fmt::{{closure}}::h121c3336b55e047b 18: 0x7f462b6e814b - rustc_middle::ty::context::tls::with_opt::{{closure}}::h61e852c60289ba39 19: 0x7f462b6e80f2 - rustc_middle::ty::context::tls::with_opt::h7523943876fa0859 20: 0x7f462b6ed449 - rustc_middle::util::bug::opt_span_bug_fmt::h131d336df190dd17 21: 0x7f462b6ed3be - rustc_middle::util::bug::bug_fmt::hd70ee7a62354ffb4 22: 0x7f462abcd99e - rustc_metadata::rmeta::decoder::::get_optimized_mir::{{closure}}::h39f7817a07c77c87 23: 0x7f462abcd8cd - rustc_metadata::rmeta::decoder::::get_optimized_mir::h9941977f74cd53b1 24: 0x7f462aac9cb4 - rustc_metadata::rmeta::decoder::cstore_impl::provide_extern::optimized_mir::h5f31f7a688ccd8b6 25: 0x7f462ba0ab31 - rustc_middle::dep_graph::::with_deps::h3eab829577a8f302 26: 0x7f462bca745e - rustc_query_system::dep_graph::graph::DepGraph::with_task::h2f5addcabc56d839 27: 0x7f462b8929a1 - rustc_data_structures::stack::ensure_sufficient_stack::h93f3da63c0fd2961 28: 0x7f462ba9d920 - rustc_query_system::query::plumbing::get_query_impl::h02907482693d81b7 29: 0x7f462b7e152e - rustc_middle::ty::::instance_mir::h088f05d39950979e 30: 0x7f462a2f8778 - rustc_mir::monomorphize::collector::collect_neighbours::h3a2d11a7f35726ff 31: 0x7f462a2f3672 - rustc_mir::monomorphize::collector::collect_items_rec::h592c99f8b97ec733 32: 0x7f462a4e1811 - rustc_session::utils::::time::h6bba076e45971cf3 33: 0x7f462a2f25ed - rustc_mir::monomorphize::collector::collect_crate_mono_items::h5434448f8a3f6dd0 34: 0x7f462a3c4f57 - rustc_mir::monomorphize::partitioning::collect_and_partition_mono_items::h226257a3cee4be40 35: 0x7f462910be02 - rustc_middle::ty::query:: for rustc_middle::ty::query::queries::collect_and_partition_mono_items>::compute::hb65c2f1422abd379 36: 0x7f46290168de - rustc_middle::dep_graph::::with_deps::h656735bca657df98 37: 0x7f462908525a - rustc_query_system::dep_graph::graph::DepGraph::with_eval_always_task::hf04d412bb00e8bbe 38: 0x7f46290fbf19 - rustc_data_structures::stack::ensure_sufficient_stack::h95acab77ebc7bf02 39: 0x7f4628fdf47c - rustc_query_system::query::plumbing::get_query_impl::h3d4812047a177271 40: 0x7f462910d48c - rustc_codegen_ssa::base::codegen_crate::h4ca0c07e9283eb44 41: 0x7f4629141415 - ::codegen_crate::h067fffb3870bc5b0 42: 0x7f4628f014f9 - rustc_interface::queries::Queries::ongoing_codegen::h1d025e9038b6dd3f 43: 0x7f4628d0adc8 - rustc_interface::queries::::enter::hcfd2504bb4e8d292 44: 0x7f4628cd90e7 - rustc_span::with_source_map::hdf17fa3a4d8fe35b 45: 0x7f4628d071ac - scoped_tls::ScopedKey::set::h5dca480b178bbfab 46: 0x7f4628d0e2c5 - std::sys_common::backtrace::__rust_begin_short_backtrace::ha4cbf96baa7e4912 47: 0x7f4628c7734a - core::ops::function::FnOnce::call_once{{vtable.shim}}::he2d69591c700ddc8 48: 0x7f462843365a - as core::ops::function::FnOnce>::call_once::hea1090dbdcecbf5a at /rustc/1c389ffeff814726dec325f0f2b0c99107df2673/library/alloc/src/boxed.rs:1318:9 49: 0x7f462843365a - as core::ops::function::FnOnce>::call_once::h8d5723d3912bd325 at /rustc/1c389ffeff814726dec325f0f2b0c99107df2673/library/alloc/src/boxed.rs:1318:9 50: 0x7f462843365a - std::sys::unix::thread::Thread::new::thread_start::hc17a425ca2995724 at /rustc/1c389ffeff814726dec325f0f2b0c99107df2673/library/std/src/sys/unix/thread.rs:71:17 51: 0x7f46283383e9 - start_thread 52: 0x7f4628255293 - __GI___clone 53: 0x0 - note: the compiler unexpectedly panicked. this is a bug. ```

jdonszelmann commented 3 years ago

The exact place of the bug seems to be here: https://github.com/rust-lang/rust/blob/f8e5209a21c698398a0f8c04af55e2ad6cbd113c/compiler/rustc_metadata/src/rmeta/decoder.rs#L1181

But interestingly enough not here: https://github.com/rust-lang/rust/blob/f8e5209a21c698398a0f8c04af55e2ad6cbd113c/compiler/rustc_metadata/src/rmeta/decoder.rs#L994

jdonszelmann commented 3 years ago

I have done some research. The issue seems to be caused by the fact that #[lang=start] is in a different crate as main(){}. the #[lang=start] function gets a reference to main in the other crate. From now on I will call the crates the "start" crate and the "main" crate respectively.

So now there are two options:

Either the main crate uses #![no-main]. This causes an expected linker error because the start crate can't find the symbol main.

Alternatively, the main crate doesn't use #![no-main]. This is what causes the compiler panic. Because rustc uses the main() {} function as a starting point for monomorphization. This triggers a debug assertion: https://github.com/rust-lang/rust/blob/f8e5209a21c698398a0f8c04af55e2ad6cbd113c/compiler/rustc_mir/src/monomorphize/collector.rs#L376. I believe this is because it sees the entry function in the start crate as the entrypoint for monomorphization. However, this start function may never be allowed to be used as an entrypoint for monomorphization exactly because it is in a foreign crate. It should instead use the main function in the main crate as the entrypoint for monomorphization. (relevant line: https://github.com/rust-lang/rust/blob/f8e5209a21c698398a0f8c04af55e2ad6cbd113c/compiler/rustc_mir/src/monomorphize/collector.rs#L319, called in https://github.com/rust-lang/rust/blob/f8e5209a21c698398a0f8c04af55e2ad6cbd113c/compiler/rustc_mir/src/monomorphize/collector.rs#L284).

This all is very interesting because this exact code is what std uses (https://github.com/rust-lang/rust/blob/afa995b2dd1e194845f2082707e6045d539230a5/library/std/src/rt.rs#L60). I believe this may be because std is handled specially somewhere, haven't figured out where yet.

I'd like to mention @NULLx76 for helping to trace down the source of this bug.

jdonszelmann commented 3 years ago

Update: https://github.com/rust-lang/rust/blob/afa995b2dd1e194845f2082707e6045d539230a5/compiler/rustc_passes/src/entry.rs#L160. This code finds main and is used by ctx.entry_fn(). However, it always looks them up in this order:

So in short, it always prefers start over main. But when start is in another crate this fails the monomorphization process.

jdonszelmann commented 3 years ago

We figured out that making lang_start generic in any way (the example in the original post above would become the codeblock below) fixes the ICE but still produces a linking error. 

use core::panic::PanicInfo;

#[lang = "start"]
#[no_mangle]
fn lang_start<T: 'static>(
    main: fn() -> T,
    _argc: isize,
    _argv: *const *const u8,
) -> isize {
    main();

    return 0;
}

#[lang = "eh_personality"] extern fn rust_eh_personality() {}
#[lang = "panic_impl"] extern fn rust_begin_panic(_info: &PanicInfo) -> ! {
    panic!("Yeet!");
}

#[lang = "oom"]
fn rust_alloc_error(_: core::alloc::Layout) -> ! {
    panic!("Yeet oom!");
}

This suggest that everything I said about what I thought the problem was may be false.

The linking error:

  = note: "cc" "-Wl,--as-needed" "-Wl,-z,noexecstack" "-m64" "-Wl,--eh-frame-hdr" "-L" "/home/jonathan/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib" "/home/jonathan/src/langstartbug/target/debug/examples/teststart-c94a695dacb81923.150bybnj090jvy04.rcgu.o" "/home/jonathan/src/langstartbug/target/debug/examples/teststart-c94a695dacb81923.mnpwjudxw992adt.rcgu.o" "-o" "/home/jonathan/src/langstartbug/target/debug/examples/teststart-c94a695dacb81923" "-Wl,--gc-sections" "-pie" "-Wl,-zrelro" "-Wl,-znow" "-nodefaultlibs" "-L" "/home/jonathan/src/langstartbug/target/debug/deps" "-L" "/home/jonathan/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-Wl,--start-group" "-Wl,-Bstatic" "/home/jonathan/src/langstartbug/target/debug/deps/liblangstart-57411b13d382a48d.rlib" "/home/jonathan/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_std_workspace_core-90996f4879673567.rlib" "/home/jonathan/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcore-9ea09a899c3eda46.rlib" "-Wl,--end-group" "/home/jonathan/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcompiler_builtins-ef2408da76957905.rlib" "-Wl,-Bdynamic"
  = note: /usr/bin/ld: /usr/lib/gcc/x86_64-pc-linux-gnu/10.2.0/../../../../lib/Scrt1.o: in function `_start':
          (.text+0x16): undefined reference to `__libc_csu_fini'
          /usr/bin/ld: (.text+0x1d): undefined reference to `__libc_csu_init'
          /usr/bin/ld: (.text+0x2a): undefined reference to `__libc_start_main'
          collect2: error: ld returned 1 exit status
bjorn3 commented 3 years ago

attr_main (not entirely sure what that is)

You can add the #[main] attribute to any function. It will then behave similar to fn main in the crate root.

bjorn3 commented 3 years ago

The linking error:

That linking error has nothing to do with rustc. It simply means that crt0.o has been linked to provide the _start function that provides the entry point from the kernel, but libc hasn't been linked, which is a dependency of crt0.o.

jdonszelmann commented 3 years ago

attr_main (not entirely sure what that is)

You can add the #[main] attribute to any function. It will then behave similar to fn main in the crate root.

ah of course

jdonszelmann commented 3 years ago

The linking error:

That linking error has nothing to do with rustc. It simply means that crt0.o has been linked to provide the _start function that provides the entry point from the kernel, but libc hasn't been linked, which is a dependency of crt0.o.

alright, so given that that is linked in, this basically works. So the only problem now is that the lang=start item cannot not be generic or else it's an ICE

bjorn3 commented 3 years ago

It does work if the #[lang = "start"] is in the main executable crate, but it should also be made to work with it being in a different crate I think.

jdonszelmann commented 3 years ago

Either that or at least give a meaningful error

asquared31415 commented 3 years ago

@bjorn3 Can you give an example of it working correctly in the same crate? Whenever I try to use it in the same crate I get an LLVM error called function is not the same type as the call when the lang item is not generic when using a distributed rustc and the aforementioned linker errors when using the rustc test suite whether or not it is generic.

Honestly I'm surprised it gets this far at all because it is meant to be generic always and I would have expected a generic substitutions error. I think the correct course of action is to require the start lang item to have one generic type, because we should substitute in the return type of the main fn that gets passed to it. This would end up fixing this ICE because then it would not assume it can codegen locally and defer to linking to the proper crate.