search

rr-debugger / rr

Record and Replay Framework
http://rr-project.org/
Other
9.12k stars 583 forks source link

Setting watchpoint and reverse-continuing causes replay divergence #3367

Closed staticfloat closed 2 years ago

staticfloat commented 2 years ago

I am using rr to debug a segfault in Julia when working with a USB device. I have noticed that a certain order of commands reliably causes a replay divergence. The commands are:

$ rr replay .
(gdb) c
(gdb) b *0x7f4246aff6f0
(gdb) rc
`rr` output The output of running `rr replay`: ``` $ ~/src/rr/bin/rr replay . GNU gdb (Ubuntu 12.0.90-0ubuntu1) 12.0.90 Copyright (C) 2022 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-linux-gnu". Type "show configuration" for configuration details. For bug reporting instructions, please see: . Find the GDB manual and other documentation resources online at: . For help, type "help". Type "apropos word" to search for commands related to "word"... Reading symbols from /home/sabae/.local/share/rr/julia-3/mmap_pack_32_julia... Really redefine built-in command "restart"? (y or n) [answered Y; input not from terminal] Really redefine built-in command "jump"? (y or n) [answered Y; input not from terminal] Remote debugging using 127.0.0.1:1337 Reading symbols from /lib64/ld-linux-x86-64.so.2... Reading symbols from /usr/lib/debug/.build-id/61/ef896a699bb1c2e4e231642b2e1688b2f1a61e.debug... BFD: warning: system-supplied DSO at 0x6fffd000 has a section extending past end of file 0x00007f42642a62b0 in _start () from /lib64/ld-linux-x86-64.so.2 (rr) c Continuing. [ Info: Make connection: 'LimeSDR-USB [USB 3.0] 9060A02433A1F' [ Info: Reference clock 30.72 MHz [ Info: Device name: LimeSDR-USB [ Info: Reference: 30.72 MHz CGEN: Freq=61.44 MHz, VCO=2.33472 GHz, INT=75, FRAC=0, DIV_OUTCH_CGEN=18 CGEN: Freq=61.44 MHz, VCO=2.33472 GHz, INT=75, FRAC=0, DIV_OUTCH_CGEN=18 CGEN: Freq=80 MHz, VCO=2.4 GHz, INT=77, FRAC=131072, DIV_OUTCH_CGEN=14 CGEN: Freq=80 MHz, VCO=2.4 GHz, INT=77, FRAC=131072, DIV_OUTCH_CGEN=14 [ Info: LMS7002M register cache: Disabled CGEN: Freq=256 MHz, VCO=2.048 GHz, INT=65, FRAC=699050, DIV_OUTCH_CGEN=3 CGEN: Freq=256 MHz, VCO=2.048 GHz, INT=65, FRAC=699050, DIV_OUTCH_CGEN=3 MCU programming : 16384/16384 MCU Programming finished, 4281 ms [ Info: RX LPF configured CGEN: Freq=256 MHz, VCO=2.048 GHz, INT=65, FRAC=699050, DIV_OUTCH_CGEN=3 CGEN: Freq=256 MHz, VCO=2.048 GHz, INT=65, FRAC=699050, DIV_OUTCH_CGEN=3 [ Info: RX LPF configured CGEN: Freq=256 MHz, VCO=2.048 GHz, INT=65, FRAC=699050, DIV_OUTCH_CGEN=3 CGEN: Freq=256 MHz, VCO=2.048 GHz, INT=65, FRAC=699050, DIV_OUTCH_CGEN=3 [ Info: Filter calibrated. Filter order-4th, filter bandwidth set to 5 MHz.Real pole 1st order filter set to 2.5 MHz. Preemphasis filter not active CGEN: Freq=61.44 MHz, VCO=2.33472 GHz, INT=75, FRAC=0, DIV_OUTCH_CGEN=18 [ Info: TX LPF configured ┌ Warning: TXLPF set to 5.000 MHz (requested 2.000 MHz [out of range]) └ @ SoapySDR ~/src/DARPA-TRIAD/xtrx_julia/software/SoapySDR.jl/src/logger.jl:20 [ Info: Filter calibrated. Filter order-4th, filter bandwidth set to 5 MHz.Real pole 1st order filter set to 2.5 MHz. Preemphasis filter not active CGEN: Freq=61.44 MHz, VCO=2.33472 GHz, INT=75, FRAC=0, DIV_OUTCH_CGEN=18 [ Info: TX LPF configured CGEN: Freq=256 MHz, VCO=2.048 GHz, INT=65, FRAC=699050, DIV_OUTCH_CGEN=3 CGEN: Freq=256 MHz, VCO=2.048 GHz, INT=65, FRAC=699050, DIV_OUTCH_CGEN=3 [ Info: Filter calibrated. Filter order-4th, filter bandwidth set to 5 MHz.Real pole 1st order filter set to 2.5 MHz. Preemphasis filter not active CGEN: Freq=61.44 MHz, VCO=2.33472 GHz, INT=75, FRAC=0, DIV_OUTCH_CGEN=18 [ Info: TX LPF configured ┌ Warning: TXLPF set to 5.000 MHz (requested 2.000 MHz [out of range]) └ @ SoapySDR ~/src/DARPA-TRIAD/xtrx_julia/software/SoapySDR.jl/src/logger.jl:20 [ Info: Filter calibrated. Filter order-4th, filter bandwidth set to 5 MHz.Real pole 1st order filter set to 2.5 MHz. Preemphasis filter not active CGEN: Freq=61.44 MHz, VCO=2.33472 GHz, INT=75, FRAC=0, DIV_OUTCH_CGEN=18 [ Info: TX LPF configured Tx calibration using MCU INTERNAL loopback Tx ch.A @ 2495 MHz, BW: 2.5 MHz, RF output: BAND1, Gain: 2 Current MCU firmware: 5, DC/IQ calibration full MCU Ref. clock: 30.72 MHz ┌ Error: Tx Calibration: MCU error 5 (Loopback signal weak: not connected/insufficient gain?) └ @ SoapySDR ~/src/DARPA-TRIAD/xtrx_julia/software/SoapySDR.jl/src/logger.jl:18 Tx calibration using MCU INTERNAL loopback Tx ch.B @ 2495 MHz, BW: 2.5 MHz, RF output: BAND1, Gain: 2 Current MCU firmware: 5, DC/IQ calibration full MCU Ref. clock: 30.72 MHz ┌ Error: Tx Calibration: MCU error 5 (Loopback signal weak: not connected/insufficient gain?) └ @ SoapySDR ~/src/DARPA-TRIAD/xtrx_julia/software/SoapySDR.jl/src/logger.jl:18 Rx calibration using INTERNAL loopback Rx ch.A @ 2495 MHz, BW: 2.5 MHz, RF input: LNAL, PGA: 5, LNA: 15, TIA: 2 Current MCU firmware: 5, DC/IQ calibration full MCU Ref. clock: 30.72 MHz [ Info: Rx calibration finished RX | DC | GAIN | PHASE ---+-----+------+------ I: | 13 | 2047 | 32 Q: | 63 | 1973 | Duration: 379 ms Rx calibration using INTERNAL loopback Rx ch.B @ 2495 MHz, BW: 2.5 MHz, RF input: LNAL, PGA: 5, LNA: 15, TIA: 2 Current MCU firmware: 5, DC/IQ calibration full MCU Ref. clock: 30.72 MHz ┌ Error: Rx calibration: MCU error 5 (Loopback signal weak: not connected/insufficient gain?) └ @ SoapySDR ~/src/DARPA-TRIAD/xtrx_julia/software/SoapySDR.jl/src/logger.jl:18 ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x00000000000007f8 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x0000000000000ff0 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x00000000000017e8 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x0000000000001fe0 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x00000000000027d8 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x0000000000002fd0 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x00000000000037c8 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x0000000000003fc0 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x00000000000047b8 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x0000000000004fb0 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x00000000000057a8 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x0000000000005fa0 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x0000000000006798 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x0000000000006f90 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x0000000000007788 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x0000000000007f80 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x0000000000008778 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x0000000000008f70 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x0000000000009768 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x0000000000009f60 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x000000000000a758 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x000000000000af50 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x000000000000b748 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x000000000000bf40 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x000000000000c738 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x000000000000cf30 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x000000000000d728 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x000000000000df20 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x000000000000e718 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x000000000000ef10 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x000000000000f708 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x000000000000ff00 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x00000000000106f8 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x0000000000010ef0 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x00000000000116e8 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x0000000000011ee0 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x00000000000126d8 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x0000000000012ed0 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x00000000000136c8 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x0000000000013ec0 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x00000000000146b8 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x0000000000014eb0 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x00000000000156a8 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x0000000000015ea0 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x0000000000016698 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x0000000000016e90 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x0000000000017688 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x0000000000017e80 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x0000000000018678 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x0000000000018e70 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x0000000000019668 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x0000000000019e60 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x000000000001a658 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x000000000001ae50 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x000000000001b648 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x000000000001be40 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x000000000001c638 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x000000000001ce30 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x000000000001d628 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x000000000001de20 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x000000000001e618 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x000000000001ee10 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x000000000001f608 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x000000000001fe00 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x00000000000205f8 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x0000000000020df0 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x00000000000215e8 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x0000000000021de0 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x00000000000225d8 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x0000000000022dd0 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x00000000000235c8 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x0000000000023dc0 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x00000000000245b8 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x0000000000024db0 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x00000000000255a8 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x0000000000025da0 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x0000000000026598 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x0000000000026d90 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x0000000000027588 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x0000000000027d80 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x0000000000028578 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x0000000000028d70 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x0000000000029568 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x0000000000029d60 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x000000000002a558 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x000000000002ad50 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x000000000002b548 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x000000000002bd40 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x000000000002c538 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x000000000002cd30 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x000000000002d528 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x000000000002dd20 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x000000000002e518 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x000000000002ed10 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x000000000002f508 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x000000000002fd00 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x00000000000304f8 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x0000000000030cf0 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x00000000000314e8 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x0000000000031ce0 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x00000000000324d8 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x0000000000032cd0 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x00000000000334c8 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x0000000000033cc0 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x00000000000344b8 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x0000000000034cb0 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x00000000000354a8 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x0000000000035ca0 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x0000000000036498 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x0000000000036c90 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x0000000000037488 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x0000000000037c80 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x0000000000038478 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x0000000000038c70 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x0000000000039468 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x0000000000039c60 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x000000000003a458 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x000000000003ac50 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x000000000003b448 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x000000000003bc40 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x000000000003c438 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x000000000003cc30 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x000000000003d428 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x000000000003dc20 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x000000000003e418 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x000000000003ec10 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x000000000003f408 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x000000000003fc00 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x00000000000403f8 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x0000000000040bf0 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x00000000000413e8 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x0000000000041be0 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x00000000000423d8 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x0000000000042bd0 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x00000000000433c8 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true ┌ Info: RX in │ buff_idx * s_rx.mtu = 0x0000000000043bc0 │ num_samples = 0x00000000ffffffff └ buff_idx * s_rx.mtu < num_samples = true [New Thread 3378328.3380203] [New Thread 3378328.3378329] [New Thread 3378328.3378330] [New Thread 3378328.3379680] [New Thread 3378328.3379681] [New Thread 3378328.3379682] Thread 2 received signal SIGSEGV, Segmentation fault. [Switching to Thread 3378328.3380203] 0x00007f424735135e in ?? () (rr) b *0x7f4246aff6f0 Breakpoint 1 at 0x7f4246aff6f0 (rr) rc Continuing. [FATAL src/ReplaySession.cc:1173:check_ticks_consistency()] (task 3542915 (rec:3380203) at time 404459) -> Assertion `ticks_now == trace_ticks' failed to hold. ticks mismatch for 'SIGNAL: SIGSEGV(det)'; expected 1347532, got 1347534 Tail of trace dump: { real_time:856160.463735 global_time:404439, event:`SYSCALL: ioctl' (state:EXITING_SYSCALL) tid:3379681, ticks:5220644 rax:0x0 rbx:0x1e9ec88 rcx:0xffffffffffffffff rdx:0x7f42365d8bc8 rsi:0x4008550d rdi:0x1b rbp:0x7f42365d8bf0 rsp:0x7f42365d8b50 r8:0x7f42365d8a50 r9:0x0 r10:0x0 r11:0x246 r12:0x7f42365d9640 r13:0x0 r14:0x7f42640c5850 r15:0x7ffde16d3210 rip:0x7f426414baff eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x10 fs_base:0x7f42365d9640 gs_base:0x0 { tid:3379681, addr:0x7f42365d8bc8, length:0x8 } { tid:3379681, addr:0x7f422c042130, length:0x38 } { tid:3379681, addr:0x7f422c028bc0, length:0x4000 } } { real_time:856160.463873 global_time:404440, event:`SYSCALL: ioctl' (state:ENTERING_SYSCALL) tid:3379681, ticks:5220789 rax:0xffffffffffffffda rbx:0x1e9ec88 rcx:0xffffffffffffffff rdx:0x7f42365d8bc8 rsi:0x4008550d rdi:0x1b rbp:0x7f42365d8bf0 rsp:0x7f42365d8b50 r8:0x7f42365d8a50 r9:0x0 r10:0x0 r11:0x246 r12:0x7f42365d9640 r13:0x0 r14:0x7f42640c5850 r15:0x7ffde16d3210 rip:0x7f426414baff eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x10 fs_base:0x7f42365d9640 gs_base:0x0 } { real_time:856160.463949 global_time:404441, event:`SYSCALL: ioctl' (state:EXITING_SYSCALL) tid:3379681, ticks:5220789 rax:0x0 rbx:0x1e9ec88 rcx:0xffffffffffffffff rdx:0x7f42365d8bc8 rsi:0x4008550d rdi:0x1b rbp:0x7f42365d8bf0 rsp:0x7f42365d8b50 r8:0x7f42365d8a50 r9:0x0 r10:0x0 r11:0x246 r12:0x7f42365d9640 r13:0x0 r14:0x7f42640c5850 r15:0x7ffde16d3210 rip:0x7f426414baff eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x10 fs_base:0x7f42365d9640 gs_base:0x0 { tid:3379681, addr:0x7f42365d8bc8, length:0x8 } { tid:3379681, addr:0x7f422c041f70, length:0x38 } { tid:3379681, addr:0x7f422c02cbc0, length:0x4000 } } { real_time:856160.464093 global_time:404442, event:`SYSCALL: ioctl' (state:ENTERING_SYSCALL) tid:3379681, ticks:5220934 rax:0xffffffffffffffda rbx:0x1e9ec88 rcx:0xffffffffffffffff rdx:0x7f42365d8bc8 rsi:0x4008550d rdi:0x1b rbp:0x7f42365d8bf0 rsp:0x7f42365d8b50 r8:0x7f42365d8a50 r9:0x0 r10:0x0 r11:0x246 r12:0x7f42365d9640 r13:0x0 r14:0x7f42640c5850 r15:0x7ffde16d3210 rip:0x7f426414baff eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x10 fs_base:0x7f42365d9640 gs_base:0x0 } { real_time:856160.464169 global_time:404443, event:`SYSCALL: ioctl' (state:EXITING_SYSCALL) tid:3379681, ticks:5220934 rax:0x0 rbx:0x1e9ec88 rcx:0xffffffffffffffff rdx:0x7f42365d8bc8 rsi:0x4008550d rdi:0x1b rbp:0x7f42365d8bf0 rsp:0x7f42365d8b50 r8:0x7f42365d8a50 r9:0x0 r10:0x0 r11:0x246 r12:0x7f42365d9640 r13:0x0 r14:0x7f42640c5850 r15:0x7ffde16d3210 rip:0x7f426414baff eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x10 fs_base:0x7f42365d9640 gs_base:0x0 { tid:3379681, addr:0x7f42365d8bc8, length:0x8 } { tid:3379681, addr:0x7f422c0421b0, length:0x38 } { tid:3379681, addr:0x7f422c030bc0, length:0x4000 } } { real_time:856160.464306 global_time:404444, event:`SYSCALL: ioctl' (state:ENTERING_SYSCALL) tid:3379681, ticks:5221079 rax:0xffffffffffffffda rbx:0x1e9ec88 rcx:0xffffffffffffffff rdx:0x7f42365d8bc8 rsi:0x4008550d rdi:0x1b rbp:0x7f42365d8bf0 rsp:0x7f42365d8b50 r8:0x7f42365d8a50 r9:0x0 r10:0x0 r11:0x246 r12:0x7f42365d9640 r13:0x0 r14:0x7f42640c5850 r15:0x7ffde16d3210 rip:0x7f426414baff eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x10 fs_base:0x7f42365d9640 gs_base:0x0 } { real_time:856160.464381 global_time:404445, event:`SYSCALL: ioctl' (state:EXITING_SYSCALL) tid:3379681, ticks:5221079 rax:0x0 rbx:0x1e9ec88 rcx:0xffffffffffffffff rdx:0x7f42365d8bc8 rsi:0x4008550d rdi:0x1b rbp:0x7f42365d8bf0 rsp:0x7f42365d8b50 r8:0x7f42365d8a50 r9:0x0 r10:0x0 r11:0x246 r12:0x7f42365d9640 r13:0x0 r14:0x7f42640c5850 r15:0x7ffde16d3210 rip:0x7f426414baff eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x10 fs_base:0x7f42365d9640 gs_base:0x0 { tid:3379681, addr:0x7f42365d8bc8, length:0x8 } { tid:3379681, addr:0x7f422c042070, length:0x38 } { tid:3379681, addr:0x7f422c034bc0, length:0x4000 } } { real_time:856160.464519 global_time:404446, event:`SYSCALL: ioctl' (state:ENTERING_SYSCALL) tid:3379681, ticks:5221224 rax:0xffffffffffffffda rbx:0x1e9ec88 rcx:0xffffffffffffffff rdx:0x7f42365d8bc8 rsi:0x4008550d rdi:0x1b rbp:0x7f42365d8bf0 rsp:0x7f42365d8b50 r8:0x7f42365d8a50 r9:0x0 r10:0x0 r11:0x246 r12:0x7f42365d9640 r13:0x0 r14:0x7f42640c5850 r15:0x7ffde16d3210 rip:0x7f426414baff eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x10 fs_base:0x7f42365d9640 gs_base:0x0 } { real_time:856160.464595 global_time:404447, event:`SYSCALL: ioctl' (state:EXITING_SYSCALL) tid:3379681, ticks:5221224 rax:0x0 rbx:0x1e9ec88 rcx:0xffffffffffffffff rdx:0x7f42365d8bc8 rsi:0x4008550d rdi:0x1b rbp:0x7f42365d8bf0 rsp:0x7f42365d8b50 r8:0x7f42365d8a50 r9:0x0 r10:0x0 r11:0x246 r12:0x7f42365d9640 r13:0x0 r14:0x7f42640c5850 r15:0x7ffde16d3210 rip:0x7f426414baff eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x10 fs_base:0x7f42365d9640 gs_base:0x0 { tid:3379681, addr:0x7f42365d8bc8, length:0x8 } { tid:3379681, addr:0x7f422c0420b0, length:0x38 } { tid:3379681, addr:0x7f422c038bc0, length:0x4000 } } { real_time:856160.464733 global_time:404448, event:`SYSCALL: ioctl' (state:ENTERING_SYSCALL) tid:3379681, ticks:5221369 rax:0xffffffffffffffda rbx:0x1e9ec88 rcx:0xffffffffffffffff rdx:0x7f42365d8bc8 rsi:0x4008550d rdi:0x1b rbp:0x7f42365d8bf0 rsp:0x7f42365d8b50 r8:0x7f42365d8a50 r9:0x0 r10:0x0 r11:0x246 r12:0x7f42365d9640 r13:0x0 r14:0x7f42640c5850 r15:0x7ffde16d3210 rip:0x7f426414baff eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x10 fs_base:0x7f42365d9640 gs_base:0x0 } { real_time:856160.464795 global_time:404449, event:`SYSCALL: ioctl' (state:EXITING_SYSCALL) tid:3379681, ticks:5221369 rax:0xfffffffffffffff5 rbx:0x1e9ec88 rcx:0xffffffffffffffff rdx:0x7f42365d8bc8 rsi:0x4008550d rdi:0x1b rbp:0x7f42365d8bf0 rsp:0x7f42365d8b50 r8:0x7f42365d8a50 r9:0x0 r10:0x0 r11:0x246 r12:0x7f42365d9640 r13:0x0 r14:0x7f42640c5850 r15:0x7ffde16d3210 rip:0x7f426414baff eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x10 fs_base:0x7f42365d9640 gs_base:0x0 { tid:3379681, addr:0x7f42365d8bc8, length:0x8 } } { real_time:856160.464928 global_time:404450, event:`SYSCALLBUF_FLUSH' tid:3379681, ticks:5221521 { syscall:'poll', ret:0x0, size:0x28 } } { real_time:856160.464947 global_time:404451, event:`SYSCALL: poll' (state:ENTERING_SYSCALL) tid:3379681, ticks:5221521 rax:0xffffffffffffffda rbx:0x7f4235dd8fa0 rcx:0xffffffffffffffff rdx:0xfa rsi:0x3 rdi:0x7f4230000bd0 rbp:0x7f4235dd8fa0 rsp:0x7f4235dd8db0 r8:0x0 r9:0xfa r10:0x0 r11:0x246 r12:0x7f42365d9640 r13:0x0 r14:0x7f42640c5850 r15:0x7ffde16d3210 rip:0x70000002 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x7 fs_base:0x7f42365d9640 gs_base:0x0 } { real_time:856160.464957 global_time:404452, event:`SYSCALLBUF_RESET' tid:3379681, ticks:5221521 } { real_time:856160.465011 global_time:404453, event:`SYSCALL: futex' (state:EXITING_SYSCALL) tid:3380203, ticks:1344515 rax:0x0 rbx:0x7f42345d6fa0 rcx:0xffffffffffffffff rdx:0x0 rsi:0x89 rdi:0x1cab300 rbp:0x7f42345d6fa0 rsp:0x7f42345d6db0 r8:0x0 r9:0xffffffff r10:0x7f4234dd69a0 r11:0x246 r12:0x0 r13:0x0 r14:0x1cab2fc r15:0x1cab300 rip:0x70000002 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0xca fs_base:0x7f4234dd7640 gs_base:0x0 } { real_time:856160.465166 global_time:404454, event:`SYSCALLBUF_FLUSH' tid:3380203, ticks:1347205 { syscall:'clock_gettime', ret:0x0, size:0x20 } { syscall:'futex', ret:0x0, size:0x14 } } { real_time:856160.465178 global_time:404455, event:`SYSCALL: ioctl' (state:ENTERING_SYSCALL) tid:3380203, ticks:1347205 rax:0xffffffffffffffda rbx:0x7f422c000bac rcx:0xffffffffffffffff rdx:0x7f422c0420b0 rsi:0x8038550a rdi:0x1b rbp:0x7f4234dd6a60 rsp:0x7f4234dd69a0 r8:0x7f422c042 r9:0x7f422c0420b0 r10:0x7f42372d8a58 r11:0x246 r12:0x2592e50 r13:0x7f4237144aa2 r14:0x7f422c03cbc0 r15:0x7f42351d7610 rip:0x7f426414baff eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x10 fs_base:0x7f4234dd7640 gs_base:0x0 } { real_time:856160.465188 global_time:404456, event:`SYSCALLBUF_RESET' tid:3380203, ticks:1347205 } { real_time:856160.465244 global_time:404457, event:`SYSCALL: ioctl' (state:EXITING_SYSCALL) tid:3380203, ticks:1347205 rax:0x0 rbx:0x7f422c000bac rcx:0xffffffffffffffff rdx:0x7f422c0420b0 rsi:0x8038550a rdi:0x1b rbp:0x7f4234dd6a60 rsp:0x7f4234dd69a0 r8:0x7f422c042 r9:0x7f422c0420b0 r10:0x7f42372d8a58 r11:0x246 r12:0x2592e50 r13:0x7f4237144aa2 r14:0x7f422c03cbc0 r15:0x7f42351d7610 rip:0x7f426414baff eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x10 fs_base:0x7f4234dd7640 gs_base:0x0 } { real_time:856160.465412 global_time:404458, event:`SYSCALLBUF_FLUSH' tid:3380203, ticks:1347532 { syscall:'clock_gettime', ret:0x0, size:0x20 } } { real_time:856160.465421 global_time:404459, event:`SIGNAL: SIGSEGV(det)' tid:3380203, ticks:1347532 rax:0x7f4234dd7640 rbx:0x7f4234dd5960 rcx:0x7f4234dd58e0 rdx:0x7efd rsi:0x7f4234dd59b0 rdi:0x7 rbp:0x7f4234dd5950 rsp:0x7f4234dd58a0 r8:0x0 r9:0x7f4247351310 r10:0xffffffffffffffff r11:0x0 r12:0x0 r13:0x7f4237144aa2 r14:0x7f422c000bac r15:0x7f4253586c30 rip:0x7f424735135e eflags:0x10206 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0xffffffffffffffff fs_base:0x7f4234dd7640 gs_base:0x0 } { real_time:856160.465431 global_time:404460, event:`SYSCALLBUF_RESET' tid:3380203, ticks:1347532 } === Start rr backtrace: /home/sabae/src/rr/bin/rr(_ZN2rr13dump_rr_stackEv+0x60)[0x556fc8815e0a] /home/sabae/src/rr/bin/rr(_ZN2rr9GdbServer15emergency_debugEPNS_4TaskE+0xfb)[0x556fc860031b] /home/sabae/src/rr/bin/rr(+0x403d1b)[0x556fc863bd1b] /home/sabae/src/rr/bin/rr(_ZN2rr21EmergencyDebugOstreamD1Ev+0x63)[0x556fc863bfc1] /home/sabae/src/rr/bin/rr(_ZN2rr13ReplaySession23check_ticks_consistencyEPNS_10ReplayTaskERKNS_5EventE+0x143)[0x556fc874cc13] /home/sabae/src/rr/bin/rr(_ZN2rr13ReplaySession28emulate_deterministic_signalEPNS_10ReplayTaskEiRKNS0_15StepConstraintsE+0x3c4)[0x556fc874d060] /home/sabae/src/rr/bin/rr(_ZN2rr13ReplaySession18try_one_trace_stepEPNS_10ReplayTaskERKNS0_15StepConstraintsE+0x174)[0x556fc874f122] /home/sabae/src/rr/bin/rr(_ZN2rr13ReplaySession11replay_stepERKNS0_15StepConstraintsE+0x1d7)[0x556fc8750987] /home/sabae/src/rr/bin/rr(_ZN2rr14ReplayTimeline19replay_step_to_markERKNS0_4MarkERNS0_24ReplayStepToMarkStrategyE+0x1da)[0x556fc876a62c] /home/sabae/src/rr/bin/rr(_ZN2rr14ReplayTimeline16reverse_continueERKSt8functionIFbPNS_10ReplayTaskERKNS_11BreakStatusEEERKS1_IFbvEE+0x657)[0x556fc876e8a3] /home/sabae/src/rr/bin/rr(_ZN2rr9GdbServer14debug_one_stepERNS_10GdbRequestE+0x878)[0x556fc85fcea0] /home/sabae/src/rr/bin/rr(_ZN2rr9GdbServer12serve_replayERKNS0_15ConnectionFlagsE+0x5ba)[0x556fc85ff3aa] /home/sabae/src/rr/bin/rr(+0x50ccf0)[0x556fc8744cf0] /home/sabae/src/rr/bin/rr(_ZN2rr13ReplayCommand3runERSt6vectorINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESaIS7_EE+0x41f)[0x556fc874580f] /home/sabae/src/rr/bin/rr(main+0x278)[0x556fc8834894] /lib/x86_64-linux-gnu/libc.so.6(+0x29d90)[0x7f2946d84d90] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x80)[0x7f2946d84e40] /home/sabae/src/rr/bin/rr(_start+0x25)[0x556fc8542735] === End rr backtrace Launch gdb with gdb '-l' '10000' '-ex' 'set sysroot /' '-ex' 'target extended-remote 127.0.0.1:3971' /home/sabae/.local/share/rr/julia-3/mmap_pack_32_julia ^Cwarning: Error detected on fd 12 Remote communication error. Target disconnected.: Resource temporarily unavailable. (rr) q ```

The packed trace is available here. The machine this was run on is an AMD machine:

$ cat /proc/cpuinfo | head -n 28
processor       : 0
vendor_id       : AuthenticAMD
cpu family      : 23
model           : 113
model name      : AMD Ryzen 9 3950X 16-Core Processor
stepping        : 0
microcode       : 0x8701013
cpu MHz         : 2200.000
cache size      : 512 KB
physical id     : 0
siblings        : 32
core id         : 0
cpu cores       : 16
apicid          : 0
initial apicid  : 0
fpu             : yes
fpu_exception   : yes
cpuid level     : 16
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc cpuid extd_apicid aperfmperf rapl pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_llc mwaitx cpb cat_l3 cdp_l3 hw_pstate ssbd mba ibpb stibp vmmcall fsgsbase bmi1 avx2 smep bmi2 cqm rdt_a rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local clzero irperf xsaveerptr rdpru wbnoinvd arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold avic v_vmsave_vmload vgif v_spec_ctrl umip rdpid overflow_recov succor smca sme sev sev_es
bugs            : sysret_ss_attrs spectre_v1 spectre_v2 spec_store_bypass retbleed
bogomips        : 6987.07
TLB size        : 3072 4K pages
clflush size    : 64
cache_alignment : 64
address sizes   : 43 bits physical, 48 bits virtual
power management: ts ttp tm hwpstate cpb eff_freq_ro [13] [14]

The rr version used was the latest master as of this writing:

$ git -C ~/src/rr log -1
commit 336edc303b12e96b08290a61c9ab87283f8e9149 (HEAD -> master, origin/master, origin/HEAD)
Author: Robert O'Callahan <robert@ocallahan.org>
Date:   Tue Aug 16 08:17:41 2022 +1200

    Dump full siginfo for unexpected signal
khuey commented 2 years ago

Is it reproducible with hardware breakpoints (i.e. hbreak *0xfoo instead of b *0xfoo)?

staticfloat commented 2 years ago

No, using a hardware-assisted breakpoint seems to work:

(rr) hbreak *0x7f4246aff6f0
Hardware assisted breakpoint 1 at 0x7f4246aff6f0
(rr) rc
Continuing.

Thread 2 received signal SIGSEGV, Segmentation fault.
0x00007f424735135e in ?? ()
(rr) rc
Continuing.
khuey commented 2 years ago

Idk if @Keno wants to dig in here but something reading the executable code and noticing the breakpoint would explain this and we've seen it before in other contexts.

Keno commented 2 years ago

If hbreak works, probably not worth debugging.