ICE when moving bin to a new place when having a bin and a lib in one project

ftvkyo commented 3 years ago

Reproducing

Step 1

Run cargo new ice && cd ice Run cargo test

Step 2

Append to main.rs (this example won't get tested anyway)

/// Examples:
/// ```
/// assert_eq!(fun(), 5);
/// ```
fn fun() -> u8 {
    5
}

Run cargo test

Step 3

Append to Cargo.toml

[[bin]]
name = "icebin"
path = "src/bin.rs"

[lib]
name = "icelib"
path = "src/lib.rs"

Remove fun from main.rs and add this to lib.rs

/// Examples:
/// ```
/// assert_eq!(icelib::fun(), 5);
/// ```
pub fn fun() -> u8 {
    5
}

Rename main.rs to bin.rs Run cargo test

Step 4

Move bin.rs to bin/main.rs Update Cargo.toml Run cargo test

Reproduced, cargo test fails with ICE.

Running cargo clean and then cargo build removes both problems and everything starts to work fine.

Error output

   Compiling ice v0.1.0 (/home/ftvkyo/Proj/ice)
thread 'rustc' panicked at 'failed to lookup `SourceFile` in new context', compiler/rustc_middle/src/ty/query/on_disk_cache.rs:710:22
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

error: internal compiler error: unexpected panic

note: the compiler unexpectedly panicked. this is a bug.

note: we would appreciate a bug report: https://github.com/rust-lang/rust/issues/new?labels=C-bug%2C+I-ICE%2C+T-compiler&template=ice.md

note: rustc 1.51.0 (2fd73fabe 2021-03-23) running on x86_64-unknown-linux-gnu

note: compiler flags: -C embed-bitcode=no -C debuginfo=2 -C incremental

note: some of the compiler flags provided by cargo are hidden

query stack during panic:
#0 [optimized_mir] optimizing MIR for `main`
#1 [collect_and_partition_mono_items] collect_and_partition_mono_items
end of query stack

Backtrace

``` stack backtrace: 0: rust_begin_unwind at /rustc/2fd73fabe469357a12c2c974c140f67e7cdd76d0/library/std/src/panicking.rs:493:5 1: core::panicking::panic_fmt at /rustc/2fd73fabe469357a12c2c974c140f67e7cdd76d0/library/core/src/panicking.rs:92:14 2: core::option::expect_failed at /rustc/2fd73fabe469357a12c2c974c140f67e7cdd76d0/library/core/src/option.rs:1292:5 3: rustc_middle::ty::query::on_disk_cache::CacheDecoder::file_index_to_file 4: rustc_middle::ty::query::on_disk_cache:: for rustc_span::span_encoding::Span>::decode 5: rustc_middle::mir::_DERIVE_rustc_serialize_Decodable_D_FOR_SourceInfo:: for rustc_middle::mir::SourceInfo>::decode 6: rustc_middle::mir::_DERIVE_rustc_serialize_Decodable_D_FOR_Statement:: for rustc_middle::mir::Statement>::decode 7: rustc_serialize::serialize::Decoder::read_seq 8: rustc_middle::mir::_DERIVE_rustc_serialize_Decodable_D_FOR_BasicBlockData:: for rustc_middle::mir::BasicBlockData>::decode 9: rustc_serialize::serialize::Decoder::read_seq 10: rustc_middle::mir::_DERIVE_rustc_serialize_Decodable_D_FOR_Body:: for rustc_middle::mir::Body>::decode 11: rustc_middle::ty::codec:: for &rustc_middle::mir::Body>::decode 12: rustc_middle::ty::query::on_disk_cache::OnDiskCache::try_load_query_result 13: rustc_query_system::query::plumbing::load_from_disk_and_cache_in_memory 14: rustc_data_structures::stack::ensure_sufficient_stack 15: rustc_query_system::query::plumbing::get_query_impl 16: rustc_middle::ty::::instance_mir 17: rustc_mir::monomorphize::collector::collect_neighbours 18: rustc_mir::monomorphize::collector::collect_items_rec 19: rustc_session::utils::::time 20: rustc_mir::monomorphize::collector::collect_crate_mono_items 21: rustc_mir::monomorphize::partitioning::collect_and_partition_mono_items 22: rustc_middle::ty::query:: for rustc_middle::ty::query::queries::collect_and_partition_mono_items>::compute 23: rustc_middle::dep_graph::::with_deps 24: rustc_query_system::dep_graph::graph::DepGraph::with_task_impl 25: rustc_data_structures::stack::ensure_sufficient_stack 26: rustc_query_system::query::plumbing::force_query_with_job 27: rustc_query_system::query::plumbing::get_query_impl 28: rustc_codegen_ssa::base::codegen_crate 29: ::codegen_crate 30: rustc_session::utils::::time 31: rustc_interface::passes::QueryContext::enter 32: rustc_interface::queries::Queries::ongoing_codegen 33: rustc_interface::queries::::enter 34: rustc_span::with_source_map 35: rustc_interface::interface::create_compiler_and_run 36: rustc_span::with_session_globals ```

Extra comments

I think the sequence is not the shortest, but I'll leave that long version here, because when I first experienced the problem and starting reducing the sequence, I once got this (so it might be a useful clue): ``` Compiling succotash v0.1.0 (/home/ftvkyo/Proj/succotash) thread 'rustc' panicked at 'index out of bounds: the len is 0 but the index is 0', compiler/rustc_middle/src/ty/query/on_disk_cache.rs:883:18 note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace error: internal compiler error: unexpected panic note: the compiler unexpectedly panicked. this is a bug. note: we would appreciate a bug report: https://github.com/rust-lang/rust/issues/new?labels=C-bug%2C+I-ICE%2C+T-compiler&template=ice.md note: rustc 1.51.0 (2fd73fabe 2021-03-23) running on x86_64-unknown-linux-gnu note: compiler flags: -C embed-bitcode=no -C debuginfo=2 -C incremental note: some of the compiler flags provided by cargo are hidden query stack during panic: #0 [optimized_mir] optimizing MIR for `main` #1 [collect_and_partition_mono_items] collect_and_partition_mono_items end of query stack thread 'rustc' panicked at 'failed to lookup `SourceFile` in new context', compiler/rustc_middle/src/ty/query/on_disk_cache.rs:710:22 error: could not compile `succotash` To learn more, run the command again with --verbose. warning: build failed, waiting for other jobs to finish... note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace error: internal compiler error: unexpected panic note: the compiler unexpectedly panicked. this is a bug. note: we would appreciate a bug report: https://github.com/rust-lang/rust/issues/new?labels=C-bug%2C+I-ICE%2C+T-compiler&template=ice.md note: rustc 1.51.0 (2fd73fabe 2021-03-23) running on x86_64-unknown-linux-gnu note: compiler flags: -C embed-bitcode=no -C debuginfo=2 -C incremental note: some of the compiler flags provided by cargo are hidden query stack during panic: #0 [optimized_mir] optimizing MIR for `main` #1 [collect_and_partition_mono_items] collect_and_partition_mono_items end of query stack error: build failed ``` However, I don't have exact steps for this error. I can try to provide a shorter reproduction sequence for the original problem if needed.

Enselic commented 1 year ago

I made a script based on the work of fanninpm and JohnTitor but I get a regular error and not an ICE. Can you still reproduce this? What changes does the script need to reproduce the ICE?

#!/bin/bash
set -o errexit -o nounset -o pipefail

cd $(mktemp -d)
echo "Running in $(pwd)"

echo "Step 1"

echo "Run cargo new ice && cd ice"
cat > Cargo.toml <<'EOF'
[package]
name = "icelib"
version = "0.0.1"
edition = "2018"
EOF

mkdir -p src

cat > src/main.rs <<'EOF'
fn main() {
    println!("Hello, world!");
}
EOF

echo "Run cargo test"
cargo +nightly-2021-03-24 test

echo "Step 2"

echo "Append to main.rs (this example won't get tested anyway)"
cat >> src/main.rs <<'EOF'
/// Examples:
/// ```
/// assert_eq!(fun(), 5);
/// ```
fn fun() -> u8 {
    5
}
EOF

echo "Run cargo test"
cargo +nightly-2021-03-24 test

echo "Step 3"

echo "Append to Cargo.toml"
cat >> Cargo.toml <<'EOF'

[[bin]]
name = "icebin"
path = "src/bin.rs"

[lib]
name = "icelib"
path = "src/lib.rs"
EOF

echo "Remove fun from main.rs and add this to lib.rs"
cat > src/main.rs <<'EOF'
fn main() {
    println!("Hello, world!");
}
EOF

cat > src/lib.rs <<'EOF'
/// Examples:
/// ```
/// assert_eq!(icelib::fun(), 5);
/// ```
pub fn fun() -> u8 {
    5
}
EOF

echo "Rename main.rs to bin.rs"
mv src/main.rs src/bin.rs

echo "Run cargo test"
cargo +nightly-2021-03-24 test

echo "Step 4"

echo "Move bin.rs to bin/main.rs"
mkdir -p src/bin
mv src/bin.rs src/bin/main.rs

echo "Update Cargo.toml"
sed 's|src/bin.rs|src/bin/main.rs|' Cargo.toml

echo "Run cargo test"
cargo +nightly-2021-03-24 test

   Compiling icelib v0.0.1 (/tmp/tmp.VpVo9yHaIa)
error: couldn't read src/bin.rs: No such file or directory (os error 2)

error: aborting due to previous error

error: could not compile `icelib`

To learn more, run the command again with --verbose.
warning: build failed, waiting for other jobs to finish...
error: build failed

ftvkyo commented 1 year ago

Thanks for this!

There was a minor bug in the suggested script. I added -i flag to sed, as it didn't edit the file but just printed it out.

Here is the file to reproduce it with 1.51.0 which I used. You can modify the TOOLCHAIN variable if you want to bisect.

Interestingly, in the project where I experienced this originally, I actually put main.rs into <project root>/bin/main.rs, rather than <project root>/src/bin/main.rs. However, this file also reproduces it. Output at the bottom.

#!/bin/bash
set -o errexit -o nounset -o pipefail

TOOLCHAIN=1.51.0

cd $(mktemp -d)
echo "Running in $(pwd)"

echo "Step 1"

echo "Run cargo new ice && cd ice"
cat > Cargo.toml <<'EOF'
[package]
name = "icelib"
version = "0.0.1"
edition = "2018"
EOF

mkdir -p src

cat > src/main.rs <<'EOF'
fn main() {
    println!("Hello, world!");
}
EOF

echo "Run cargo test"
cargo +$TOOLCHAIN test

echo "Step 2"

echo "Append to main.rs (this example won't get tested anyway)"
cat >> src/main.rs <<'EOF'
/// Examples:
/// ```
/// assert_eq!(fun(), 5);
/// ```
fn fun() -> u8 {
    5
}
EOF

echo "Run cargo test"
cargo +$TOOLCHAIN test

echo "Step 3"

echo "Append to Cargo.toml"
cat >> Cargo.toml <<'EOF'

[[bin]]
name = "icebin"
path = "src/bin.rs"

[lib]
name = "icelib"
path = "src/lib.rs"
EOF

echo "Remove fun from main.rs and add this to lib.rs"
cat > src/main.rs <<'EOF'
fn main() {
    println!("Hello, world!");
}
EOF

cat > src/lib.rs <<'EOF'
/// Examples:
/// ```
/// assert_eq!(icelib::fun(), 5);
/// ```
pub fn fun() -> u8 {
    5
}
EOF

echo "Rename main.rs to bin.rs"
mv src/main.rs src/bin.rs

echo "Run cargo test"
cargo +$TOOLCHAIN test

echo "Step 4"

echo "Move bin.rs to bin/main.rs"
mkdir -p src/bin
mv src/bin.rs src/bin/main.rs

echo "Update Cargo.toml"
sed -i 's|src/bin.rs|src/bin/main.rs|' Cargo.toml

echo "Run cargo test"
cargo +$TOOLCHAIN test

... truncated ...

Step 4
Move bin.rs to bin/main.rs
Update Cargo.toml
Run cargo test
   Compiling icelib v0.0.1 (/tmp/tmp.VjVZ3Vko23)
thread 'rustc' panicked at 'failed to lookup `SourceFile` in new context', compiler/rustc_middle/src/ty/query/on_disk_cache.rs:710:22
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

error: internal compiler error: unexpected panic

note: the compiler unexpectedly panicked. this is a bug.

note: we would appreciate a bug report: https://github.com/rust-lang/rust/issues/new?labels=C-bug%2C+I-ICE%2C+T-compiler&template=ice.md

note: rustc 1.51.0 (2fd73fabe 2021-03-23) running on x86_64-unknown-linux-gnu

note: compiler flags: -C embed-bitcode=no -C debuginfo=2 -C incremental

note: some of the compiler flags provided by cargo are hidden

query stack during panic:
#0 [optimized_mir] optimizing MIR for `main`
#1 [collect_and_partition_mono_items] collect_and_partition_mono_items
end of query stack
error: could not compile `icelib`

To learn more, run the command again with --verbose.

rust-lang / rust