tikv / pprof-rs

A Rust CPU profiler implemented with the help of backtrace-rs
Apache License 2.0
1.26k stars 94 forks source link

trace is trying to dereferencing misaligned pointer #216

Open gen-xu opened 1 year ago

gen-xu commented 1 year ago

rust panic with following error message

Thread { id: ThreadId(1), name: Some("main"), .. }: panicked at 'misaligned pointer dereference: address must be a multiple of 0x8 but is 0x7ffd57e2a81c', /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/pprof-0.11.1/src/backtrace/frame_pointer.rs:107:30

after some investigation these two lines in frame_pointer.rs turned out to be the root cause

            let frame = Frame {
                ip: unsafe { (*frame_pointer).ret }, // trying to dereference misaligned pointer here
            };

            if !cb(&frame) {
                break;
            }
            frame_pointer = unsafe { (*frame_pointer).frame_pointer }; // trying to dereference misaligned pointer here
alindima commented 11 months ago

I'm seeing similar errors also when using pprof-rs with libunwind on Mac M2:

libunwind::CFI_Parser<libunwind::LocalAddressSpace>::decodeFDE(libunwind::LocalAddressSpace&, unsigned long, libunwind::CFI_Parser<libunwind::LocalAddressSpace>::FDE_Info*, libunwind::CFI_Parser<libunwind::LocalAddressSpace>::CIE_Info*, bool) (@libunwind::CFI_Parser<libunwind::LocalAddressSpace>::decodeFDE(libunwind::LocalAddressSpace&, unsigned long, libunwind::CFI_Parser<libunwind::LocalAddressSpace>::FDE_Info*, libunwind::CFI_Parser<libunwind::LocalAddressSpace>::CIE_Info*, bool):15)
libunwind::UnwindCursor<libunwind::LocalAddressSpace, libunwind::Registers_arm64>::step() (@libunwind::UnwindCursor<libunwind::LocalAddressSpace, libunwind::Registers_arm64>::step():55)
_Unwind_Backtrace (@_Unwind_Backtrace:90)
backtrace::backtrace::libunwind::trace (/Users/alindima/.cargo/registry/src/index.crates.io-6f17d22bba15001f/backtrace-0.3.68/src/backtrace/libunwind.rs:93)
backtrace::backtrace::trace_unsynchronized (/Users/alindima/.cargo/registry/src/index.crates.io-6f17d22bba15001f/backtrace-0.3.68/src/backtrace/mod.rs:66)
<pprof::backtrace::backtrace_rs::Trace as pprof::backtrace::Trace>::trace (/Users/alindima/Desktop/code/pprof-rs/src/backtrace/backtrace_rs.rs:23)
perf_signal_handler (/Users/alindima/Desktop/code/pprof-rs/src/profiler.rs:291)
_sigtramp (@_sigtramp:17)
...
DavidBJaffe commented 9 months ago

I got the same error message. This is on an Apple M1 Pro running OSX 13.5.2 and Rust 1.72.0. I am fairly confident that the behavior changed upon OSX upgrade, although I'm not sure which upgrade, and hypothetically the behavior could have changed when I upgraded Rust. Regardless, profiling always crashes now.

YangKeao commented 9 months ago

I got the same error message. This is on an Apple M1 Pro running OSX 13.5.2 and Rust 1.72.0. I am fairly confident that the behavior changed upon OSX upgrade, although I'm not sure which upgrade, and hypothetically the behavior could have changed when I upgraded Rust. Regardless, profiling always crashes now.

I'm getting a Macbook today. Maybe I can try to reproduce the bug on it :laughing: .

The backtrace provided by @alindima above is really helpful and I think I have known the problem, just need to verify / validate the fix :beers: .