rust-lang / hashbrown

Rust port of Google's SwissTable hash map
https://rust-lang.github.io/hashbrown
Apache License 2.0
2.38k stars 275 forks source link

Library test `map::test_map::test_clone_from_memory_leaks` errors with using uninitialized data under valgrind and miri #510

Closed udoprog closed 6 months ago

udoprog commented 6 months ago

Steps to reproduce:

cargo test --lib

Find the resulting test binary and run it through valgrind:

valgrind target/debug/deps/hashbrown-e06c00d15197ef31 --exact map::test_map::test_clone_from_memory_leaks

This is what I get:

Valgrind error ``` ==132117== Thread 9 map::test_map::: ==132117== Conditional jump or move depends on uninitialised value(s) ==132117== at 0x139B47: alloc::raw_vec::RawVec::current_memory (raw_vec.rs:256) ==132117== by 0x1346DA: as core::ops::drop::Drop>::drop (raw_vec.rs:530) ==132117== by 0x131AC9: core::ptr::drop_in_place> (mod.rs:507) ==132117== by 0x1318AA: core::ptr::drop_in_place> (mod.rs:507) ==132117== by 0x132749: core::ptr::drop_in_place (mod.rs:507) ==132117== by 0x1302CD: core::ptr::drop_in_place<(i32,hashbrown::map::test_map::test_clone_from_memory_leaks::CheckedClone)> (mod.rs:507) ==132117== by 0x178D4A: drop_in_place<(i32, hashbrown::map::test_map::test_clone_from_memory_leaks::CheckedClone)> (mut_ptr.rs:1469) ==132117== by 0x178D4A: hashbrown::raw::Bucket::drop (mod.rs:590) ==132117== by 0x1CBE07: hashbrown::raw::RawTable::clone_from_impl::{{closure}} (mod.rs:3587) ==132117== by 0x135531: as core::ops::drop::Drop>::drop (scopeguard.rs:70) ==132117== by 0x1313D9: core::ptr::drop_in_place),hashbrown::raw::RawTable<(i32,hashbrown::map::test_map::test_clone_from_memory_leaks::CheckedClone)>::clone_from_impl::{{closure}}>> (mod.rs:507) ==132117== by 0x1CB124: hashbrown::raw::RawTable::clone_from_impl (mod.rs:3607) ==132117== by 0x1D4532: as hashbrown::raw::RawTableClone>::clone_from_spec (mod.rs:3545) ==132117== ```

Miri also fails:

cargo +nightly miri test map::test_map::test_clone_from_memory_leaks
MIRI error ``` test map::test_map::test_clone_from_memory_leaks - should panic ... error: Undefined Behavior: using uninitialized data, but this operation requires initialized memory --> /home/udoprog/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/alloc/src/raw_vec.rs:235:9 | 235 | self.ptr.as_ptr() | ^^^^^^^^ using uninitialized data, but this operation requires initialized memory | = help: this indicates a bug in the program: it performed an invalid operation, and caused Undefined Behavior = help: see https://doc.rust-lang.org/nightly/reference/behavior-considered-undefined.html for further information = note: BACKTRACE on thread `map::test_map::`: = note: inside `alloc::raw_vec::RawVec::::ptr` at /home/udoprog/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/alloc/src/raw_vec.rs:235:9: 235:17 = note: inside `std::vec::Vec::::as_mut_ptr` at /home/udoprog/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/alloc/src/vec/mod.rs:1331:9: 1331:23 = note: inside ` as core::ops::Drop>::drop` at /home/udoprog/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/alloc/src/vec/mod.rs:3171:62: 3171:79 = note: inside `core::ptr::drop_in_place::> - shim(Some(std::vec::Vec))` at /home/udoprog/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/ptr/mod.rs:515:1: 515:56 = note: inside `core::ptr::drop_in_place:: - shim(Some(map::test_map::test_clone_from_memory_leaks::CheckedClone))` at /home/udoprog/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/ptr/mod.rs:515:1: 515:56 = note: inside `core::ptr::drop_in_place::<(i32, map::test_map::test_clone_from_memory_leaks::CheckedClone)> - shim(Some((i32, map::test_map::test_clone_from_memory_leaks::CheckedClone)))` at /home/udoprog/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/ptr/mod.rs:515:1: 515:56 = note: inside `core::ptr::mut_ptr::::drop_in_place` at /home/udoprog/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/ptr/mut_ptr.rs:1473:18: 1473:37 note: inside `raw::Bucket::<(i32, map::test_map::test_clone_from_memory_leaks::CheckedClone)>::drop` --> src/raw/mod.rs:590:9 | 590 | self.as_ptr().drop_in_place(); | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ note: inside closure --> src/raw/mod.rs:3587:25 | 3587 | self_.bucket(i).drop(); | ^^^^^^^^^^^^^^^^^^^^^^ note: inside `), {closure@src/raw/mod.rs:3583:48: 3583:64}> as core::ops::Drop>::drop` --> src/scopeguard.rs:70:9 | 70 | (self.dropfn)(&mut self.value); | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ = note: inside `core::ptr::drop_in_place::), {closure@src/raw/mod.rs:3583:48: 3583:64}>> - shim(Some(scopeguard::ScopeGuard<(usize, &mut raw::RawTable<(i32, map::test_map::test_clone_from_memory_leaks::CheckedClone)>), {closure@src/raw/mod.rs:3583:48: 3583:64}>))` at /home/udoprog/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/ptr/mod.rs:515:1: 515:56 note: inside `raw::RawTable::<(i32, map::test_map::test_clone_from_memory_leaks::CheckedClone)>::clone_from_impl` --> src/raw/mod.rs:3607:5 | 3607 | } | ^ note: inside ` as raw::RawTableClone>::clone_from_spec` --> src/raw/mod.rs:3545:13 | 3545 | self.clone_from_impl(source); | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ note: inside ` as core::clone::Clone>::clone` --> src/raw/mod.rs:3464:17 | 3464 | new_table.clone_from_spec(self); | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ note: inside ` as core::clone::Clone>::clone` --> src/map.rs:199:20 | 199 | table: self.table.clone(), | ^^^^^^^^^^^^^^^^^^ note: inside `map::test_map::test_clone_from_memory_leaks` --> src/map.rs:8566:21 | 8566 | let _map2 = map1.clone(); | ^^^^^^^^^^^^ note: inside closure --> src/map.rs:8526:38 | 8524 | #[test] | ------- in this procedural macro expansion 8525 | #[should_panic = "panic in clone"] 8526 | fn test_clone_from_memory_leaks() { | ^ = note: this error originates in the attribute macro `test` (in Nightly builds, run with -Z macro-backtrace for more info) ```

I've encountered memory corruption in tests of a fork of hashbrown which might or might not be related. While trying to track down the root cause I noted that the above was also present here. I've not investigated deeper but the test case itself is implemented without unsafe.