khuey
commented
2 years ago rr --version # rr version 5.3.0
Step 1 is to retry this on a version of rr that isn't more than 2 years old.
Open alehander92 opened 2 years ago
khuey
commented
2 years ago rr --version # rr version 5.3.0
Step 1 is to retry this on a version of rr that isn't more than 2 years old.
alehander92
commented
2 years ago nice! I haven't updated since a long time.
However I still hit it with 5.5.0
khuey
commented
2 years ago Alright I can reproduce this.
alehander92
commented
2 years ago I think I reproduce it on latest source (1c7c7eb840d191bed71f) as well
khuey
commented
2 years ago I suspect the future executor is defeating our "same execution point" heuristic by storing data about the current future somewhere other than the stack.
The ip where things blow up is at 0x000056183e87d08f in core::future::get_context (cx=...) at /rustc/f1edd0429582dd29cccacaf50fd134b05593bd9c/library/core/src/future/mod.rs:97
Disassembly
Dump of assembler code for function _ZN4core6future11get_context17h3b67940abc5c064bE:
0x000056183e87d080 <+0>: sub $0x18,%rsp
0x000056183e87d084 <+4>: mov %rdi,0x8(%rsp)
0x000056183e87d089 <+9>: call *0x137231(%rip) # 0x56183e9b42c0
=> 0x000056183e87d08f <+15>: mov %rax,(%rsp)
0x000056183e87d093 <+19>: mov (%rsp),%rax
0x000056183e87d097 <+23>: mov %rax,0x10(%rsp)
0x000056183e87d09c <+28>: mov (%rsp),%rax
0x000056183e87d0a0 <+32>: add $0x18,%rsp
0x000056183e87d0a4 <+36>: retThere's nothing interesting on the C stack here, it's all futures boilerplate.
(rr) bt
#0 0x000056183e87d08f in core::future::get_context (cx=...) at /rustc/f1edd0429582dd29cccacaf50fd134b05593bd9c/library/core/src/future/mod.rs:97
#1 0x000056183e87b324 in async_executor::{impl#9}::run::{generator#0}<(), async_std::task::builder::SupportTaskLocals<core::future::from_generator::GenFuture<rr_3031::main::{generator#0}>>> () at /home/khuey/.cargo/registry/src/github.com-1ecc6299db9ec823/async-executor-1.4.1/src/lib.rs:447
#2 0x000056183e87e3bb in core::future::from_generator::{impl#1}::poll<async_executor::{impl#9}::run::{generator#0}> (self=..., cx=0x7ffeb6b46b70)
at /rustc/f1edd0429582dd29cccacaf50fd134b05593bd9c/library/core/src/future/mod.rs:80
#3 0x000056183e87fc94 in async_io::driver::block_on<(), core::future::from_generator::GenFuture<async_executor::{impl#9}::run::{generator#0}>> (future=...)
at /home/khuey/.cargo/registry/src/github.com-1ecc6299db9ec823/async-io-1.6.0/src/driver.rs:142
#4 0x000056183e879c63 in async_global_executor::reactor::block_on::{closure#0}<core::future::from_generator::GenFuture<async_executor::{impl#9}::run::{generator#0}>, ()> ()
at /home/khuey/.cargo/registry/src/github.com-1ecc6299db9ec823/async-global-executor-2.0.2/src/reactor.rs:3
#5 0x000056183e879c2d in async_global_executor::reactor::block_on<core::future::from_generator::GenFuture<async_executor::{impl#9}::run::{generator#0}>, ()> (future=...)
at /home/khuey/.cargo/registry/src/github.com-1ecc6299db9ec823/async-global-executor-2.0.2/src/reactor.rs:12
#6 0x000056183e87bf8d in async_global_executor::executor::block_on::{closure#0}<async_std::task::builder::SupportTaskLocals<core::future::from_generator::GenFuture<rr_3031::main::{generator#0}>>, ()> (executor=0x7fcfb102e7e0) at /home/khuey/.cargo/registry/src/github.com-1ecc6299db9ec823/async-global-executor-2.0.2/src/executor.rs:26
#7 0x000056183e87f633 in std::thread::local::LocalKey<async_executor::LocalExecutor>::try_with<async_executor::LocalExecutor, async_global_executor::executor::block_on::{closure#0}, ()> (
self=0x56183e9aaa08, f=...) at /rustc/f1edd0429582dd29cccacaf50fd134b05593bd9c/library/std/src/thread/local.rs:399
#8 0x000056183e87ea5a in std::thread::local::LocalKey<async_executor::LocalExecutor>::with<async_executor::LocalExecutor, async_global_executor::executor::block_on::{closure#0}, ()> (
self=0x56183e9aaa08, f=...) at /rustc/f1edd0429582dd29cccacaf50fd134b05593bd9c/library/std/src/thread/local.rs:375
#9 0x000056183e87bf37 in async_global_executor::executor::block_on<async_std::task::builder::SupportTaskLocals<core::future::from_generator::GenFuture<rr_3031::main::{generator#0}>>, ()>
(future=...) at /home/khuey/.cargo/registry/src/github.com-1ecc6299db9ec823/async-global-executor-2.0.2/src/executor.rs:26
#10 0x000056183e8799da in async_std::task::builder::{impl#0}::blocking::{closure#0}::{closure#0}<core::future::from_generator::GenFuture<rr_3031::main::{generator#0}>, ()> ()
at /home/khuey/.cargo/registry/src/github.com-1ecc6299db9ec823/async-std-1.10.0/src/task/builder.rs:171
#11 0x000056183e87c801 in async_std::task::task_locals_wrapper::{impl#0}::set_current::{closure#0}<async_std::task::builder::{impl#0}::blocking::{closure#0}::{closure#0}, ()> (
current=0x7fcfb102e790) at /home/khuey/.cargo/registry/src/github.com-1ecc6299db9ec823/async-std-1.10.0/src/task/task_locals_wrapper.rs:60
#12 0x000056183e87f79e in std::thread::local::LocalKey<core::cell::Cell<*const async_std::task::task_locals_wrapper::TaskLocalsWrapper>>::try_with<core::cell::Cell<*const async_std::task::task_locals_wrapper::TaskLocalsWrapper>, async_std::task::task_locals_wrapper::{impl#0}::set_current::{closure#0}, ()> (self=0x56183e9aaa28, f=...)
at /rustc/f1edd0429582dd29cccacaf50fd134b05593bd9c/library/std/src/thread/local.rs:399
#13 0x000056183e87eabc in std::thread::local::LocalKey<core::cell::Cell<*const async_std::task::task_locals_wrapper::TaskLocalsWrapper>>::with<core::cell::Cell<*const async_std::task::task_locals_wrapper::TaskLocalsWrapper>, async_std::task::task_locals_wrapper::{impl#0}::set_current::{closure#0}, ()> (self=0x56183e9aaa28, f=...)
at /rustc/f1edd0429582dd29cccacaf50fd134b05593bd9c/library/std/src/thread/local.rs:375
#14 0x000056183e87c5e7 in async_std::task::task_locals_wrapper::TaskLocalsWrapper::set_current<async_std::task::builder::{impl#0}::blocking::{closure#0}::{closure#0}, ()> (
task=0x7ffeb6b47b80, f=...) at /home/khuey/.cargo/registry/src/github.com-1ecc6299db9ec823/async-std-1.10.0/src/task/task_locals_wrapper.rs:55
#15 0x000056183e87994c in async_std::task::builder::{impl#0}::blocking::{closure#0}<core::future::from_generator::GenFuture<rr_3031::main::{generator#0}>, ()> (
num_nested_blocking=0x7fcfb102e778) at /home/khuey/.cargo/registry/src/github.com-1ecc6299db9ec823/async-std-1.10.0/src/task/builder.rs:168
#16 0x000056183e87eedb in std::thread::local::LocalKey<core::cell::Cell<usize>>::try_with<core::cell::Cell<usize>, async_std::task::builder::{impl#0}::blocking::{closure#0}, ()> (
self=0x56183e9aa8b8, f=...) at /rustc/f1edd0429582dd29cccacaf50fd134b05593bd9c/library/std/src/thread/local.rs:399
#17 0x000056183e87ebaa in std::thread::local::LocalKey<core::cell::Cell<usize>>::with<core::cell::Cell<usize>, async_std::task::builder::{impl#0}::blocking::{closure#0}, ()> (
self=0x56183e9aa8b8, f=...) at /rustc/f1edd0429582dd29cccacaf50fd134b05593bd9c/library/std/src/thread/local.rs:375
#18 0x000056183e879660 in async_std::task::builder::Builder::blocking<core::future::from_generator::GenFuture<rr_3031::main::{generator#0}>, ()> (self=..., future=...)
at /home/khuey/.cargo/registry/src/github.com-1ecc6299db9ec823/async-std-1.10.0/src/task/builder.rs:161
#19 0x000056183e87e8df in async_std::task::block_on::block_on<core::future::from_generator::GenFuture<rr_3031::main::{generator#0}>, ()> (future=...)
at /home/khuey/.cargo/registry/src/github.com-1ecc6299db9ec823/async-std-1.10.0/src/task/block_on.rs:33
#20 0x000056183e879b37 in rr_3031::main () at src/main.rs:10The actual state of the future is going to live in the heap somewhere and not on the stack, so multiple polls of the same future probably look identical to us.
@rocallahan any thoughts?
rocallahan
commented
2 years ago I agree with your diagnosis.
This would likely to be hard to fix. It's hard to get motivated about it because Pernosco handles this so much better :-(.
alehander92
commented
2 years ago what would we need to do if we want to try to contribute a fix? EDIT: what is the direction to go into
rocallahan
commented
2 years ago First verify that you have two different program states where the Registers are the same and the tick count is the same and the ReturnAddressList is the same (but presumably the state of futures on the heap or stack are different).
Then try to figure out how rr could systematically capture the state of those futures and capture them alongside the ReturnAddressList. I have no idea how to do that or if it's possible.
alehander92
commented
2 years ago i thought each jump increases the tick count, having no jumps between those polls sound strange to me indeed, thanks for the info
rocallahan
commented
2 years ago Only conditional branches increase the tick count.
There is a counter for all branches but it's unreliable on some CPU versions.
alehander92
commented
1 month ago
I build this program (I use a project where this is
src/main.rs)cargo file is
I build, record and replay:
in rr I do