UB from a crater run

[Dylan-DPC-zz]

I got a panic from a crater run which included this crate (in case you are not aware, crater is a tool that runs rustc against a set number of locked crates to check for regressions). It was with 0.10.9 version so i'm not sure if it still holds true in the 0.11.0 or not. Also, it could be caused by one of the dependencies. I tried tracking it down but I couldn't find anything.

The main point of the crater run was to find UB according to this advisory: https://github.com/RustSec/advisory-db/issues/298.

Edit: there is a small possibility that this could be due to the file permissions being denied in crater runs

The crater output is:

[INFO] [stdout] failures:
[INFO] [stdout] 
[INFO] [stdout] ---- src/lib.rs - deduplicate_dirs (line 216) stdout ----
[INFO] [stdout] Test executable failed (terminated by signal).
[INFO] [stdout] 
[INFO] [stdout] stderr:
[INFO] [stdout] thread '<unnamed>' panicked at 'attempted to leave type `internal::Local` uninitialized, which is invalid', /rustc/41997647ba6a77908f6ef64401414feb00bccf16/src/libcore/mem/mod.rs:661:5
[INFO] [stdout] stack backtrace:
[INFO] [stdout] thread '<unnamed>' panicked at 'attempted to leave type `internal::Local` uninitialized, which is invalid', /rustc/41997647ba6a77908f6ef64401414feb00bccf16/src/libcore/mem/mod.rs:661:5
[INFO] [stdout] thread '<unnamed>' panicked at 'attempted to leave type `internal::Local` uninitialized, which is invalid', /rustc/41997647ba6a77908f6ef64401414feb00bccf16/src/libcore/mem/mod.rs:661:5
[INFO] [stdout] thread '<unnamed>' panicked at 'attempted to leave type `internal::Local` uninitialized, which is invalid', /rustc/41997647ba6a77908f6ef64401414feb00bccf16/src/libcore/mem/mod.rs:661:5
[INFO] [stdout] thread '<unnamed>' panicked at 'attempted to leave type `internal::Local` uninitialized, which is invalid', /rustc/41997647ba6a77908f6ef64401414feb00bccf16/src/libcore/mem/mod.rs:661:5
[INFO] [stdout] thread '<unnamed>' panicked at 'attempted to leave type `internal::Local` uninitialized, which is invalid', /rustc/41997647ba6a77908f6ef64401414feb00bccf16/src/libcore/mem/mod.rs:661:5
[INFO] [stdout] thread '<unnamed>' panicked at 'attempted to leave type `internal::Local` uninitialized, which is invalid', /rustc/41997647ba6a77908f6ef64401414feb00bccf16/src/libcore/mem/mod.rs:661:5
[INFO] [stdout] thread '<unnamed>' panicked at 'attempted to leave type `internal::Local` uninitialized, which is invalid', /rustc/41997647ba6a77908f6ef64401414feb00bccf16/src/libcore/mem/mod.rs:661:5
[INFO] [stdout]    0:     0x55caa32b01c4 - backtrace::backtrace::libunwind::trace::h61666fa4b9834a45
[INFO] [stdout]                                at /cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.46/src/backtrace/libunwind.rs:86
[INFO] [stdout]    1:     0x55caa32b01c4 - backtrace::backtrace::trace_unsynchronized::h01d3725fb5da8653
[INFO] [stdout]                                at /cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.46/src/backtrace/mod.rs:66
[INFO] [stdout]    2:     0x55caa32b01c4 - std::sys_common::backtrace::_print_fmt::h5c7c4751bc296f90
[INFO] [stdout]                                at src/libstd/sys_common/backtrace.rs:78
[INFO] [stdout]    3:     0x55caa32b01c4 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::h7729313ad2d729c7
[INFO] [stdout]                                at src/libstd/sys_common/backtrace.rs:59
[INFO] [stdout]    4:     0x55caa32cf9bc - core::fmt::write::hb0049f6c12bdaffc
[INFO] [stdout]                                at src/libcore/fmt/mod.rs:1069
[INFO] [stdout]    5:     0x55caa32ae113 - std::io::Write::write_fmt::hfb486fa6ccb90f14
[INFO] [stdout]                                at src/libstd/io/mod.rs:1504
[INFO] [stdout]    6:     0x55caa32b29a5 - std::sys_common::backtrace::_print::ha087cac92e84a414
[INFO] [stdout]                                at src/libstd/sys_common/backtrace.rs:62
[INFO] [stdout]    7:     0x55caa32b29a5 - std::sys_common::backtrace::print::h6b013d3bfe6cbcb8
[INFO] [stdout]                                at src/libstd/sys_common/backtrace.rs:49
[INFO] [stdout]    8:     0x55caa32b29a5 - std::panicking::default_hook::{{closure}}::h016d7de02def2736
[INFO] [stdout]                                at src/libstd/panicking.rs:198
[INFO] [stdout]    9:     0x55caa32b26e2 - std::panicking::default_hook::h9386f7e484840463
[INFO] [stdout]                                at src/libstd/panicking.rs:218
[INFO] [stdout]   10:     0x55caa32b3002 - std::panicking::rust_panic_with_hook::h3846eb1b2e5a9312
[INFO] [stdout]                                at src/libstd/panicking.rs:511
[INFO] [stdout]   11:     0x55caa32b2beb - rust_begin_unwind
[INFO] [stdout]                                at src/libstd/panicking.rs:419
[INFO] [stdout]   12:     0x55caa32ceda1 - core::panicking::panic_fmt::hc5953a7331f9805c
[INFO] [stdout]                                at src/libcore/panicking.rs:111
[INFO] [stdout]   13:     0x55caa32ceced - core::panicking::panic::hbfb03ddd2cbb509e
[INFO] [stdout]                                at src/libcore/panicking.rs:54
[INFO] [stdout]   14:     0x55caa3297489 - core::mem::uninitialized::h869ad29ea76dd030
[INFO] [stdout]                                at /rustc/41997647ba6a77908f6ef64401414feb00bccf16/src/libcore/mem/mod.rs:661
[INFO] [stdout]   15:     0x55caa3297489 - <crossbeam_epoch::internal::Local as crossbeam_epoch::sync::list::IsElement<crossbeam_epoch::internal::Local>>::entry_of::hdb4452d7a6e0a610
[INFO] [stdout]                                at /opt/rustwide/cargo-home/registry/src/github.com-1ecc6299db9ec823/crossbeam-epoch-0.7.1/src/internal.rs:493
[INFO] [stdout]   16:     0x55caa329e823 - crossbeam_epoch::sync::list::List<T,C>::insert::h3ac4d85186c907f0
[INFO] [stdout]                                at /opt/rustwide/cargo-home/registry/src/github.com-1ecc6299db9ec823/crossbeam-epoch-0.7.1/src/sync/list.rs:176
[INFO] [stdout]   17:     0x55caa3296454 - crossbeam_epoch::internal::Local::register::h91b5c90d84206ca9
[INFO] [stdout]                                at /opt/rustwide/cargo-home/registry/src/github.com-1ecc6299db9ec823/crossbeam-epoch-0.7.1/src/internal.rs:291
[INFO] [stdout]   18:     0x55caa32a0a5e - crossbeam_epoch::collector::Collector::register::h5691db836e05a7d5
[INFO] [stdout]                                at /opt/rustwide/cargo-home/registry/src/github.com-1ecc6299db9ec823/crossbeam-epoch-0.7.1/src/collector.rs:39
[INFO] [stdout]   19:     0x55caa329a7b4 - crossbeam_epoch::default::HANDLE::__init::h05b698916945b7d4
[INFO] [stdout]                                at /opt/rustwide/cargo-home/registry/src/github.com-1ecc6299db9ec823/crossbeam-epoch-0.7.1/src/default.rs:17
[INFO] [stdout]   20:     0x55caa3299ec9 - core::ops::function::FnOnce::call_once::hf43b3b136e2a1b46
[INFO] [stdout]                                at /rustc/41997647ba6a77908f6ef64401414feb00bccf16/src/libcore/ops/function.rs:232
[INFO] [stdout]   21:     0x55caa32a053c - std::thread::local::lazy::LazyKeyInner<T>::initialize::h72653fcc8deb826a
[INFO] [stdout]                                at /rustc/41997647ba6a77908f6ef64401414feb00bccf16/src/libstd/thread/local.rs:289
[INFO] [stdout]   22:     0x55caa329ced2 - std::thread::local::fast::Key<T>::try_initialize::h30683c7ab15cafc2
[INFO] [stdout]                                at /rustc/41997647ba6a77908f6ef64401414feb00bccf16/src/libstd/thread/local.rs:425
[INFO] [stdout]   23:     0x55caa329d027 - std::thread::local::fast::Key<T>::get::hd10d89894258aff9
[INFO] [stdout]                                at /rustc/41997647ba6a77908f6ef64401414feb00bccf16/src/libstd/thread/local.rs:410
[INFO] [stdout]   24:     0x55caa329a7ed - crossbeam_epoch::default::HANDLE::__getit::h962c871568622af7
[INFO] [stdout]                                at /rustc/41997647ba6a77908f6ef64401414feb00bccf16/src/libstd/thread/local.rs:175
[INFO] [stdout]   25:     0x55caa32749d5 - std::thread::local::LocalKey<T>::try_with::h3f261a7c4f47ce5a
[INFO] [stdout]                                at /rustc/41997647ba6a77908f6ef64401414feb00bccf16/src/libstd/thread/local.rs:262
[INFO] [stdout]   26:     0x55caa3283a53 - crossbeam_epoch::default::with_handle::h269db34920d39f6c
[INFO] [stdout]                                at /opt/rustwide/cargo-home/registry/src/github.com-1ecc6299db9ec823/crossbeam-epoch-0.7.1/src/default.rs:42
[INFO] [stdout]   27:     0x55caa3283d37 - crossbeam_epoch::default::is_pinned::hb6e44e79801d0481
[INFO] [stdout]                                at /opt/rustwide/cargo-home/registry/src/github.com-1ecc6299db9ec823/crossbeam-epoch-0.7.1/src/default.rs:29
[INFO] [stdout]   28:     0x55caa328b6d2 - crossbeam_deque::Stealer<T>::steal::h9ae6be401072c7cd
[INFO] [stdout]                                at /opt/rustwide/cargo-home/registry/src/github.com-1ecc6299db9ec823/crossbeam-deque-0.6.3/src/lib.rs:660
[INFO] [stdout]   29:     0x55caa326c2fa - rayon_core::registry::WorkerThread::steal::{{closure}}::hca21cf343bc94b68
[INFO] [stdout]                                at /opt/rustwide/cargo-home/registry/src/github.com-1ecc6299db9ec823/rayon-core-1.5.0/src/registry.rs:764
[INFO] [stdout]   30:     0x55caa32694cb - core::ops::function::impls::<impl core::ops::function::FnMut<A> for &mut F>::call_mut::h59bd862209ac9487
[INFO] [stdout]                                at /rustc/41997647ba6a77908f6ef64401414feb00bccf16/src/libcore/ops/function.rs:274
[INFO] [stdout]   31:     0x55caa3287921 - core::iter::traits::iterator::Iterator::find_map::check::{{closure}}::h6647419eb8e863d9
[INFO] [stdout]                                at /rustc/41997647ba6a77908f6ef64401414feb00bccf16/src/libcore/iter/traits/iterator.rs:2239
[INFO] [stdout]   32:     0x55caa327fd65 - core::iter::adapters::filter_try_fold::{{closure}}::h0e27e4a4e21c5718
[INFO] [stdout]                                at /rustc/41997647ba6a77908f6ef64401414feb00bccf16/src/libcore/iter/adapters/mod.rs:935
[INFO] [stdout]   33:     0x55caa327effc - core::ops::function::impls::<impl core::ops::function::FnMut<A> for &mut F>::call_mut::h5b9d2db37f7cb14f
[INFO] [stdout]                                at /rustc/41997647ba6a77908f6ef64401414feb00bccf16/src/libcore/ops/function.rs:274
[INFO] [stdout]   34:     0x55caa3287ed4 - core::iter::traits::iterator::Iterator::try_fold::h9c4820e0b9b6a2ec
[INFO] [stdout]                                at /rustc/41997647ba6a77908f6ef64401414feb00bccf16/src/libcore/iter/traits/iterator.rs:1878
[INFO] [stdout]   35:     0x55caa327e3c3 - <core::iter::adapters::chain::Chain<A,B> as core::iter::traits::iterator::Iterator>::try_fold::h80243e00b6de1e21
[INFO] [stdout]                                at /rustc/41997647ba6a77908f6ef64401414feb00bccf16/src/libcore/iter/adapters/chain.rs:83
[INFO] [stdout]   36:     0x55caa327e2a6 - <core::iter::adapters::Filter<I,P> as core::iter::traits::iterator::Iterator>::try_fold::ha3e1ae2a61dda215
[INFO] [stdout]                                at /rustc/41997647ba6a77908f6ef64401414feb00bccf16/src/libcore/iter/adapters/mod.rs:984
[INFO] [stdout]   37:     0x55caa327fb01 - core::iter::traits::iterator::Iterator::find_map::he70084dfab691d9a
[INFO] [stdout]                                at /rustc/41997647ba6a77908f6ef64401414feb00bccf16/src/libcore/iter/traits/iterator.rs:2245
[INFO] [stdout]   38:     0x55caa327e316 - <core::iter::adapters::FilterMap<I,F> as core::iter::traits::iterator::Iterator>::next::hdc570465366b354c
[INFO] [stdout]                                at /rustc/41997647ba6a77908f6ef64401414feb00bccf16/src/libcore/iter/adapters/mod.rs:1084
[INFO] [stdout]   39:     0x55caa326c228 - rayon_core::registry::WorkerThread::steal::h9bd36c493aa21796
[INFO] [stdout]                                at /opt/rustwide/cargo-home/registry/src/github.com-1ecc6299db9ec823/rayon-core-1.5.0/src/registry.rs:758
[INFO] [stdout]   40:     0x55caa326bf52 - rayon_core::registry::WorkerThread::wait_until_cold::{{closure}}::h590a1fcd9de02876
[INFO] [stdout]                                at /opt/rustwide/cargo-home/registry/src/github.com-1ecc6299db9ec823/rayon-core-1.5.0/src/registry.rs:714
[INFO] [stdout]   41:     0x55caa327916f - core::option::Option<T>::or_else::hf431f9e43aa9978c
[INFO] [stdout]                                at /rustc/41997647ba6a77908f6ef64401414feb00bccf16/src/libcore/option.rs:766
[INFO] [stdout]   42:     0x55caa326bc00 - rayon_core::registry::WorkerThread::wait_until_cold::hd72b2229314009a4
[INFO] [stdout]                                at /opt/rustwide/cargo-home/registry/src/github.com-1ecc6299db9ec823/rayon-core-1.5.0/src/registry.rs:712
[INFO] [stdout]   43:     0x55caa326b5f6 - rayon_core::registry::WorkerThread::wait_until::h608ca86b940e41fd
[INFO] [stdout]                                at /opt/rustwide/cargo-home/registry/src/github.com-1ecc6299db9ec823/rayon-core-1.5.0/src/registry.rs:692
[INFO] [stdout]   44:     0x55caa326c87d - rayon_core::registry::main_loop::hf39058cfb43bdbf9
[INFO] [stdout]                                at /opt/rustwide/cargo-home/registry/src/github.com-1ecc6299db9ec823/rayon-core-1.5.0/src/registry.rs:812
[INFO] [stdout]   45:     0x55caa3269720 - rayon_core::registry::ThreadBuilder::run::ha145b7a27b5851e2
[INFO] [stdout]                                at /opt/rustwide/cargo-home/registry/src/github.com-1ecc6299db9ec823/rayon-core-1.5.0/src/registry.rs:58
[INFO] [stdout]   46:     0x55caa3269c21 - <rayon_core::registry::DefaultSpawn as rayon_core::registry::ThreadSpawn>::spawn::{{closure}}::ha5a4cee564920880
[INFO] [stdout]                                at /opt/rustwide/cargo-home/registry/src/github.com-1ecc6299db9ec823/rayon-core-1.5.0/src/registry.rs:103
[INFO] [stdout]   47:     0x55caa3283e90 - std::sys_common::backtrace::__rust_begin_short_backtrace::hffc2c09a197264dd
[INFO] [stdout]                                at /rustc/41997647ba6a77908f6ef64401414feb00bccf16/src/libstd/sys_common/backtrace.rs:130
[INFO] [stdout]   48:     0x55caa3285171 - std::thread::Builder::spawn_unchecked::{{closure}}::{{closure}}::hfdfb97e3734c02e3
[INFO] [stdout]                                at /rustc/41997647ba6a77908f6ef64401414feb00bccf16/src/libstd/thread/mod.rs:475
[INFO] [stdout]   49:     0x55caa32838b3 - <std::panic::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once::h27a0583c4b616fd0
[INFO] [stdout]                                at /rustc/41997647ba6a77908f6ef64401414feb00bccf16/src/libstd/panic.rs:318
[INFO] [stdout]   50:     0x55caa326e384 - std::panicking::try::do_call::h8929afe6053c86ae
[INFO] [stdout]                                at /rustc/41997647ba6a77908f6ef64401414feb00bccf16/src/libstd/panicking.rs:331
[INFO] [stdout]   51:     0x55caa326f6dd - __rust_try
[INFO] [stdout]   52:     0x55caa326e174 - std::panicking::try::h3fadfd4c27ff95a2
[INFO] [stdout]                                at /rustc/41997647ba6a77908f6ef64401414feb00bccf16/src/libstd/panicking.rs:274
[INFO] [stdout]   53:     0x55caa3284323 - std::panic::catch_unwind::h422593581e096c2f
[INFO] [stdout]                                at /rustc/41997647ba6a77908f6ef64401414feb00bccf16/src/libstd/panic.rs:394
[INFO] [stdout]   54:     0x55caa3284f61 - std::thread::Builder::spawn_unchecked::{{closure}}::hf49384bceeda1e85
[INFO] [stdout]                                at /rustc/41997647ba6a77908f6ef64401414feb00bccf16/src/libstd/thread/mod.rs:474
[INFO] [stdout]   55:     0x55caa3274fbf - core::ops::function::FnOnce::call_once{{vtable.shim}}::h6ecc5d6bbd1cf8dc
[INFO] [stdout]                                at /rustc/41997647ba6a77908f6ef64401414feb00bccf16/src/libcore/ops/function.rs:232
[INFO] [stdout]   56:     0x55caa32a9f9f - <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once::h7deaeeba480c6ce2
[INFO] [stdout]                                at /rustc/41997647ba6a77908f6ef64401414feb00bccf16/src/liballoc/boxed.rs:1008
[INFO] [stdout]   57:     0x55caa32b53f3 - <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once::h968b1dfb87756523
[INFO] [stdout]                                at /rustc/41997647ba6a77908f6ef64401414feb00bccf16/src/liballoc/boxed.rs:1008
[INFO] [stdout]   58:     0x55caa32b53f3 - std::sys::unix::thread::Thread::new::thread_start::h2399addeb6370595
[INFO] [stdout]                                at src/libstd/sys/unix/thread.rs:87
[INFO] [stdout]   59:     0x7f326ac7a6db - start_thread
[INFO] [stdout]   60:     0x7f326a78b88f - __clone
[INFO] [stdout]   61:                0x0 - <unknown>
[INFO] [stdout] Rayon: detected unexpected panic; aborting

[darakian]

Hey, thanks for the bug report. I wonder if this issue isn't related to a symlink handling bug discovered in https://github.com/darakian/ddh/issues/23 Is it possible to rerun the crater test with a feature branch?

[Dylan-DPC-zz]

I don't think that would be possible.

[darakian]

Dang that's unfortunate. I'm not sure what I can do to fix this without a test case :(

[darakian]

Closing this issue do not having possible solution merged and no replicable test case.

[RalfJung]

@darakian I think this is a bug in crossbeam-epoch; the crate seems to depend on v0.7.1. v0.7.2 fixes a critical bug.

I am wondering why that version was not picked automatically? crossbeam-epoch is a transitive dependency it seems.

(Btw, the relevant rustc change here is https://github.com/rust-lang/rust/pull/71274.)

[RalfJung]

The dependency seems to be via rayon. But that doesn't explain why it did not use v0.7.2.

Crater has been testing ddh-0.10.9; unfortunately there seem to be no tags in this repository so I cannot easily look at the code as it was back then.

[darakian]

@RalfJung Thanks for the follow up and sorry about the lack of tags. The dependencies at the time of 0.10.9 were

[dependencies]
clap = "2.33.0"
rayon = "1.1"
serde = "1.0"
serde_json = "1.0"
serde_derive = "1.0"
siphasher = "0.3"
nohash-hasher = "0.1.1"

Is it just 0.10.9 that shows this problem? If so that's certainly odd as 0.11.0 was only a license change release and is identical code.

Here's a link to the repo on 0.10.9 commit if it helps https://github.com/darakian/ddh/tree/1d4f4922ba24a49c31d3359e584df9903c22b87c

[RalfJung]

0.10.9 was the latest version at the time the test was run, so I cannot say if newer versions still show this problem.

In your local lockfile, which version of crossbeam-epoch does it use? cargo tree can help figure that out.

[darakian]

Looks like 0.8.2

jon~/G/!/ddh:(HEAD detached at 1d4f492)❯❯❯ cargo tree | rg crossbeam
│   ├── crossbeam-deque v0.7.3
│   │   ├── crossbeam-epoch v0.8.2
│   │   │   ├── crossbeam-utils v0.7.2
│   │   ├── crossbeam-utils v0.7.2 (*)
│       ├── crossbeam-deque v0.7.3 (*)
│       ├── crossbeam-queue v0.2.3
│       │   ├── crossbeam-utils v0.7.2 (*)
│       ├── crossbeam-utils v0.7.2 (*)

[RalfJung]

Okay. I think we can consider this issue resolved then, and the next time around crater should pick up the latest version of crossbeam-epoch I hope.

[darakian]

🤞

Thanks again for reaching out :)