Tests panick with `misaligned pointer dereference: address must be a multiple of ... but is ...`

[qmonnet]

Tests consistently fail with recent versions of the toolchain, with messages such as:

thread 'test_jit_mbuff' panicked at 'misaligned pointer dereference: address must be a multiple of 0x4 but is 0x7f8038001011', src/jit.rs:100:5

Full error message

``` Running tests/misc.rs (target/debug/deps/misc-6f56f75d4e6d43c9) running 20 tests thread 'test_jit_mbuff' panicked at 'misaligned pointer dereference: address must be a multiple of 0x4 but is 0x7f8038001011', src/jit.rs:100:5 note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace thread 'test_jit_mbuff' panicked at 'panic in a function that cannot unwind', library/core/src/panicking.rs:126:5 stack backtrace: 0: 0x562d63bf4351 - std::backtrace_rs::backtrace::libunwind::trace::h28494931c73179b2 at /rustc/2f6bc5d259e7ab25ddfdd33de53b892770218918/library/std/src/../../backtrace/src/backtrace/libunwind.rs:93:5 1: 0x562d63bf4351 - std::backtrace_rs::backtrace::trace_unsynchronized::h9032c52edccf7bd1 at /rustc/2f6bc5d259e7ab25ddfdd33de53b892770218918/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5 2: 0x562d63bf4351 - std::sys_common::backtrace::_print_fmt::hd90562e967f4e4e1 at /rustc/2f6bc5d259e7ab25ddfdd33de53b892770218918/library/std/src/sys_common/backtrace.rs:65:5 3: 0x562d63bf4351 - ::fmt::h113657117676131e at /rustc/2f6bc5d259e7ab25ddfdd33de53b892770218918/library/std/src/sys_common/backtrace.rs:44:22 4: 0x562d63c1995f - core::fmt::rt::Argument::fmt::hd56cdfa11c364505 at /rustc/2f6bc5d259e7ab25ddfdd33de53b892770218918/library/core/src/fmt/rt.rs:138:9 5: 0x562d63c1995f - core::fmt::write::h24c20284e5d6be9e at /rustc/2f6bc5d259e7ab25ddfdd33de53b892770218918/library/core/src/fmt/mod.rs:1094:21 6: 0x562d63bf1b41 - std::io::Write::write_fmt::hbf02c94f0e7342d1 at /rustc/2f6bc5d259e7ab25ddfdd33de53b892770218918/library/std/src/io/mod.rs:1712:15 7: 0x562d63bf4165 - std::sys_common::backtrace::_print::he85212e2c716c859 at /rustc/2f6bc5d259e7ab25ddfdd33de53b892770218918/library/std/src/sys_common/backtrace.rs:47:5 8: 0x562d63bf4165 - std::sys_common::backtrace::print::h888aaf3ad10f084e at /rustc/2f6bc5d259e7ab25ddfdd33de53b892770218918/library/std/src/sys_common/backtrace.rs:34:9 9: 0x562d63bf5cb7 - std::panicking::default_hook::{{closure}}::hba0edb58dc223add 10: 0x562d63bf5aa4 - std::panicking::default_hook::h1555b8bada2010d7 at /rustc/2f6bc5d259e7ab25ddfdd33de53b892770218918/library/std/src/panicking.rs:288:9 11: 0x562d63b6cdd4 - as core::ops::function::Fn>::call::h52e6cd440f597cb6 at /rustc/2f6bc5d259e7ab25ddfdd33de53b892770218918/library/alloc/src/boxed.rs:1999:9 12: 0x562d63b6cdd4 - test::test_main::{{closure}}::hdcda637653172e31 at /rustc/2f6bc5d259e7ab25ddfdd33de53b892770218918/library/test/src/lib.rs:134:21 13: 0x562d63bf62c7 - as core::ops::function::Fn>::call::h07438796673f3d04 at /rustc/2f6bc5d259e7ab25ddfdd33de53b892770218918/library/alloc/src/boxed.rs:1999:9 14: 0x562d63bf62c7 - std::panicking::rust_panic_with_hook::h72a06453beb2cbcb at /rustc/2f6bc5d259e7ab25ddfdd33de53b892770218918/library/std/src/panicking.rs:695:13 15: 0x562d63bf6001 - std::panicking::begin_panic_handler::{{closure}}::h0281d6cc05cfd2a4 at /rustc/2f6bc5d259e7ab25ddfdd33de53b892770218918/library/std/src/panicking.rs:580:13 16: 0x562d63bf4796 - std::sys_common::backtrace::__rust_end_short_backtrace::h1c79565770be27d9 at /rustc/2f6bc5d259e7ab25ddfdd33de53b892770218918/library/std/src/sys_common/backtrace.rs:150:18 17: 0x562d63bf5db2 - rust_begin_unwind at /rustc/2f6bc5d259e7ab25ddfdd33de53b892770218918/library/std/src/panicking.rs:578:5 18: 0x562d63b32b33 - core::panicking::panic_nounwind_fmt::hdb51e63c7c599e80 at /rustc/2f6bc5d259e7ab25ddfdd33de53b892770218918/library/core/src/panicking.rs:96:14 19: 0x562d63b32bd7 - core::panicking::panic_nounwind::hc4511f17cef1f7da at /rustc/2f6bc5d259e7ab25ddfdd33de53b892770218918/library/core/src/panicking.rs:126:5 20: 0x562d63b32d63 - core::panicking::panic_cannot_unwind::hbac7ba3f5f929c6c at /rustc/2f6bc5d259e7ab25ddfdd33de53b892770218918/library/core/src/panicking.rs:188:5 21: 0x562d63b8f4e4 - rbpf::jit::emit4::haff8aacbfffbe8b6 at /home/runner/work/rbpf/rbpf/src/jit.rs:99:1 22: 0x562d63b8ff07 - rbpf::jit::emit_alu64_imm32::hd930f0d06f862872 at /home/runner/work/rbpf/rbpf/src/jit.rs:211:5 23: 0x562d63b911af - rbpf::jit::JitMemory::jit_compile::h0a631e7ea59867f2 at /home/runner/work/rbpf/rbpf/src/jit.rs:544:9 24: 0x562d63b93722 - rbpf::jit::compile::hb7b83a17bcef25db at /home/runner/work/rbpf/rbpf/src/jit.rs:1014:5 25: 0x562d63b842d4 - rbpf::EbpfVmMbuff::jit_compile::h1c675e69584d7361 at /home/runner/work/rbpf/rbpf/src/lib.rs:643:25 26: 0x562d63b34ed0 - misc::test_jit_mbuff::h8eb4bceb4b166566 at /home/runner/work/rbpf/rbpf/tests/misc.rs:336:9 27: 0x562d63b37b57 - misc::test_jit_mbuff::{{closure}}::h55d6770297b8bea5 at /home/runner/work/rbpf/rbpf/tests/misc.rs:314:21 28: 0x562d63b377e5 - core::ops::function::FnOnce::call_once::hafeaeb2bdb95188a at /rustc/2f6bc5d259e7ab25ddfdd33de53b892770218918/library/core/src/ops/function.rs:250:5 29: 0x562d63b7232f - core::ops::function::FnOnce::call_once::heba63a2808b93cd2 at /rustc/2f6bc5d259e7ab25ddfdd33de53b892770218918/library/core/src/ops/function.rs:250:5 30: 0x562d63b7232f - test::__rust_begin_short_backtrace::he9a5c6c59e1b0086 at /rustc/2f6bc5d259e7ab25ddfdd33de53b892770218918/library/test/src/lib.rs:655:18 31: 0x562d63b3e22c - test::run_test::{{closure}}::h1b8c22c7d438b531 at /rustc/2f6bc5d259e7ab25ddfdd33de53b892770218918/library/test/src/lib.rs:646:30 32: 0x562d63b3e22c - core::ops::function::FnOnce::call_once{{vtable.shim}}::h36a9083506ac2a12 at /rustc/2f6bc5d259e7ab25ddfdd33de53b892770218918/library/core/src/ops/function.rs:250:5 33: 0x562d63b711f7 - as core::ops::function::FnOnce>::call_once::h11e5ff78080e1397 at /rustc/2f6bc5d259e7ab25ddfdd33de53b892770218918/library/alloc/src/boxed.rs:1985:9 34: 0x562d63b711f7 - as core::ops::function::FnOnce<()>>::call_once::hfbfbda226f7a7c27 at /rustc/2f6bc5d259e7ab25ddfdd33de53b892770218918/library/core/src/panic/unwind_safe.rs:271:9 35: 0x562d63b711f7 - std::panicking::try::do_call::h42ac7b4bd35c5b5d at /rustc/2f6bc5d259e7ab25ddfdd33de53b892770218918/library/std/src/panicking.rs:485:40 36: 0x562d63b711f7 - std::panicking::try::hc6ad089a257e422e at /rustc/2f6bc5d259e7ab25ddfdd33de53b892770218918/library/std/src/panicking.rs:449:19 37: 0x562d63b711f7 - std::panic::catch_unwind::hdf4784c0c06ac0ba at /rustc/2f6bc5d259e7ab25ddfdd33de53b892770218918/library/std/src/panic.rs:140:14 38: 0x562d63b711f7 - test::run_test_in_process::hbe86cacc7510eb8b at /rustc/2f6bc5d259e7ab25ddfdd33de53b892770218918/library/test/src/lib.rs:678:27 39: 0x562d63b711f7 - test::run_test::run_test_inner::{{closure}}::hbaa8a391d0f22860 at /rustc/2f6bc5d259e7ab25ddfdd33de53b892770218918/library/test/src/lib.rs:572:39 40: 0x562d63b38c28 - test::run_test::run_test_inner::{{closure}}::hb4232323322f922d at /rustc/2f6bc5d259e7ab25ddfdd33de53b892770218918/library/test/src/lib.rs:599:37 41: 0x562d63b38c28 - std::sys_common::backtrace::__rust_begin_short_backtrace::hca81c15946d6f154 at /rustc/2f6bc5d259e7ab25ddfdd33de53b892770218918/library/std/src/sys_common/backtrace.rs:134:18 42: 0x562d63b3e45b - std::thread::Builder::spawn_unchecked_::{{closure}}::{{closure}}::h97d990ba33daf685 at /rustc/2f6bc5d259e7ab25ddfdd33de53b892770218918/library/std/src/thread/mod.rs:529:17 43: 0x562d63b3e45b - as core::ops::function::FnOnce<()>>::call_once::he0d612b643c049a4 at /rustc/2f6bc5d259e7ab25ddfdd33de53b892770218918/library/core/src/panic/unwind_safe.rs:271:9 44: 0x562d63b3e45b - std::panicking::try::do_call::haf308dcb076117ac at /rustc/2f6bc5d259e7ab25ddfdd33de53b892770218918/library/std/src/panicking.rs:485:40 45: 0x562d63b3e45b - std::panicking::try::h51fd60d29cf6feee at /rustc/2f6bc5d259e7ab25ddfdd33de53b892770218918/library/std/src/panicking.rs:449:19 46: 0x562d63b3e45b - std::panic::catch_unwind::h8e7f9d8c4a32bc2f at /rustc/2f6bc5d259e7ab25ddfdd33de53b892770218918/library/std/src/panic.rs:140:14 47: 0x562d63b3e45b - std::thread::Builder::spawn_unchecked_::{{closure}}::h07454567f5413f25 at /rustc/2f6bc5d259e7ab25ddfdd33de53b892770218918/library/std/src/thread/mod.rs:528:30 48: 0x562d63b3e45b - core::ops::function::FnOnce::call_once{{vtable.shim}}::hfb80b7cf501e16ff at /rustc/2f6bc5d259e7ab25ddfdd33de53b892770218918/library/core/src/ops/function.rs:250:5 49: 0x562d63bfa815 - as core::ops::function::FnOnce>::call_once::h71c821b130855373 at /rustc/2f6bc5d259e7ab25ddfdd33de53b892770218918/library/alloc/src/boxed.rs:1985:9 50: 0x562d63bfa815 - as core::ops::function::FnOnce>::call_once::h1701cec9acb1061c at /rustc/2f6bc5d259e7ab25ddfdd33de53b892770218918/library/alloc/src/boxed.rs:1985:9 51: 0x562d63bfa815 - std::sys::unix::thread::Thread::new::thread_start::h6dd5ef62dde103f1 at /rustc/2f6bc5d259e7ab25ddfdd33de53b892770218918/library/std/src/sys/unix/thread.rs:108:17 52: 0x7f803c6e9b43 - 53: 0x7f803c77ba00 - 54: 0x0 - thread caused non-unwinding panic. aborting. error: test failed, to rerun pass `--test misc` ```

This seems to be a consequence of https://github.com/rust-lang/rust/pull/98112.

As an easy “fix”, we can disable debug assertions for the tests, but we risk introducing more bugs. Not sure how to fix it otherwise, given that we do support unaligned memory access in rbpf at this time.

[qmonnet]

Fixed with https://github.com/qmonnet/rbpf/commit/a339ce81927279f747ee66f6a5b0f15947daba09 from #78, but it would be great to find a solution to re-enable the asserts. Leaving the issue open in the meantime.

[saethlin]

It looks to me like the offending code is this: https://github.com/qmonnet/rbpf/blob/196e5375896cc17870f38f6f23b7f7b3c755a16a/src/jit.rs#L81-L82

This could do ptr.write_unaligned($data);

[saethlin]

I'm not sure what you mean by

we do support unaligned memory access in rbpf at this time

The checks are inserted as part of compiling the Rust code that is the JIT itself, so I don't think the semantics of BPF here are relevant, right? I feel like I must be misunderstanding something here.

[qmonnet]

Hi Ben, thanks a lot for helping!

I feel like I must be misunderstanding something here.

This would be entirely possible, my experience with coding with Rust is limited to this project :)

In this case, let me explain more: in rbpf, we have two ways to execute eBPF code, the interpreter (contained in src/lib.rs) and the JIT-compiler (srs/jit.rs). I tested your suggestion to use ptr.write_unaligned($data); for the JIT-compiler, and it works well indeed for taming the asserts that appear when compiling the JIT itself, and fixes a number of tests :tada:. I'll push a PR with this change [edit: #80, now merged].

Then I've also observed error messages referring to the interpreter, and related to the tests using the interpreter for the eBPF programs, in particular the instructions for loading multiple bytes of memory into registers: ebpf::LD_H_REG/ebpf::LD_W_REG/ebpf::LD_DW_REG, around this line. This is what I was referring to earlier. Maybe there's a similar trick to use here with write_unaligned() or something else to tell Rust that these are possibly not aligned, I need to look more into it.

[qmonnet]

List of failing test (running on the interpreter), after fixing the JIT, for the record:

- README.md: - Block starting at line 393 - test/misc.rs: - test_vm_block_port - test_vm_jit_ldabsh - test_vm_jit_ldabsw - test_vm_jit_ldabsdw - test_vm_jit_ldindw - test_vm_jit_ldinddw - test/ubpf_vm.rs - test_vm_ldxdw - test_vm_ldxw - test_vm_stdw - test_vm_stw - test_vm_stxdw - test_vm_stxw