antifuchs / ratelimit_futures

Rate-limiting for rust futures v0.1
MIT License
5 stars 2 forks source link

Dead lock in `multiple` test #3

Open Johni0702 opened 5 years ago

Johni0702 commented 5 years ago

Seems like the multiple test can end up in a dead lock. More or less reliably reproducible by just running that test until it locks up (usually within milliseconds, roughly 10% chance). I haven't had a closer look at it.

gdb -p 5315 -ex 't a * bt' --batch ``` [New LWP 5316] [New LWP 5320] [New LWP 5336] [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". 0x00007494babba72c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 warning: Missing auto-load script at offset 0 in section .debug_gdb_scripts of file /home/user/rust/ratelimit_futures/target/debug/deps/ratelimit-d60df3ee53b9e5ed. Use `info auto-load python-scripts [REGEXP]' to list them. Thread 1 (Thread 0x7494ba842980 (LWP 5315)): #0 0x00007494babba72c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x000057e239ac05b3 in wait () at src/libstd/sys/unix/condvar.rs:69 #2 wait () at src/libstd/sys_common/condvar.rs:41 #3 wait<()> () at src/libstd/sync/condvar.rs:204 #4 park () at src/libstd/thread/mod.rs:910 #5 0x000057e239ac6ba2 in wait () at src/libstd/sync/mpsc/blocking.rs:71 #6 0x000057e239a3b485 in recv<(test::TestDesc, test::TestResult, alloc::vec::Vec)> () at /rustc/d595b113584f8f446957469951fd5d31adc2a44e/src/libstd/sync/mpsc/shared.rs:229 #7 0x000057e239a3a5df in recv<(test::TestDesc, test::TestResult, alloc::vec::Vec)> () at /rustc/d595b113584f8f446957469951fd5d31adc2a44e/src/libstd/sync/mpsc/mod.rs:1214 #8 0x000057e239a56502 in run_tests () at src/libtest/lib.rs:1193 #9 run_tests_console () at src/libtest/lib.rs:972 #10 0x000057e239a4c8f8 in test_main () at src/libtest/lib.rs:293 #11 0x000057e239a4d124 in test_main_static () at src/libtest/lib.rs:327 #12 0x000057e239a2f166 in ratelimit::main () #13 0x000057e239a374e0 in std::rt::lang_start::{{closure}} () at /rustc/d595b113584f8f446957469951fd5d31adc2a44e/src/libstd/rt.rs:64 #14 0x000057e239acb5b3 in {{closure}} () at src/libstd/rt.rs:49 #15 do_call () at src/libstd/panicking.rs:293 #16 0x000057e239aceb7a in __rust_maybe_catch_panic () at src/libpanic_unwind/lib.rs:85 #17 0x000057e239acc0bd in try () at src/libstd/panicking.rs:272 #18 catch_unwind () at src/libstd/panic.rs:388 #19 lang_start_internal () at src/libstd/rt.rs:48 #20 0x000057e239a374b9 in std::rt::lang_start (main=0x57e239a2f150 , argc=2, argv=0x7ffd15fe4338) at /rustc/d595b113584f8f446957469951fd5d31adc2a44e/src/libstd/rt.rs:64 #21 0x000057e239a2f19a in main () Thread 2 (Thread 0x7494ba841700 (LWP 5316)): #0 0x00007494babb5a6d in __pthread_timedjoin_ex () from /lib64/libpthread.so.0 #1 0x000057e239ace28d in join () at src/libstd/sys/unix/thread.rs:166 #2 0x000057e239a9cc2e in std::thread::JoinInner::join (self=0x7494ba840540) at /rustc/d595b113584f8f446957469951fd5d31adc2a44e/src/libstd/thread/mod.rs:1326 #3 0x000057e239a9cca2 in std::thread::JoinHandle::join (self=...) at /rustc/d595b113584f8f446957469951fd5d31adc2a44e/src/libstd/thread/mod.rs:1459 #4 0x000057e239a2f0ed in ratelimit::multiple () at tests/ratelimit.rs:48 #5 0x000057e239a3739a in ratelimit::multiple::{{closure}} () at tests/ratelimit.rs:35 #6 0x000057e239a33fee in core::ops::function::FnOnce::call_once () at /rustc/d595b113584f8f446957469951fd5d31adc2a44e/src/libcore/ops/function.rs:231 #7 0x000057e239a4383f in call_once<(),FnBox<()>> () at /rustc/d595b113584f8f446957469951fd5d31adc2a44e/src/liballoc/boxed.rs:704 #8 0x000057e239aceb7a in __rust_maybe_catch_panic () at src/libpanic_unwind/lib.rs:85 #9 0x000057e239a5e138 in try<(),std::panic::AssertUnwindSafe>>> () at /rustc/d595b113584f8f446957469951fd5d31adc2a44e/src/libstd/panicking.rs:272 #10 catch_unwind>>,()> () at /rustc/d595b113584f8f446957469951fd5d31adc2a44e/src/libstd/panic.rs:388 #11 {{closure}} () at src/libtest/lib.rs:1468 #12 0x000057e239a38b55 in std::sys_common::backtrace::__rust_begin_short_backtrace () at /rustc/d595b113584f8f446957469951fd5d31adc2a44e/src/libstd/sys_common/backtrace.rs:136 #13 0x000057e239a3cbf5 in {{closure}} () at /rustc/d595b113584f8f446957469951fd5d31adc2a44e/src/libstd/thread/mod.rs:469 #14 call_once<(),closure> () at /rustc/d595b113584f8f446957469951fd5d31adc2a44e/src/libstd/panic.rs:309 #15 do_call,()> () at /rustc/d595b113584f8f446957469951fd5d31adc2a44e/src/libstd/panicking.rs:293 #16 0x000057e239aceb7a in __rust_maybe_catch_panic () at src/libpanic_unwind/lib.rs:85 #17 0x000057e239a3d1e2 in try<(),std::panic::AssertUnwindSafe> () at /rustc/d595b113584f8f446957469951fd5d31adc2a44e/src/libstd/panicking.rs:272 #18 catch_unwind,()> () at /rustc/d595b113584f8f446957469951fd5d31adc2a44e/src/libstd/panic.rs:388 #19 {{closure}} () at /rustc/d595b113584f8f446957469951fd5d31adc2a44e/src/libstd/thread/mod.rs:468 #20 call_once () at /rustc/d595b113584f8f446957469951fd5d31adc2a44e/src/libcore/ops/function.rs:231 #21 0x000057e239abfb2f in call_once<(),FnOnce<()>> () at /rustc/d595b113584f8f446957469951fd5d31adc2a44e/src/liballoc/boxed.rs:704 #22 0x000057e239ace1d0 in call_once<(),alloc::boxed::Box>> () at /rustc/d595b113584f8f446957469951fd5d31adc2a44e/src/liballoc/boxed.rs:704 #23 start_thread () at src/libstd/sys_common/thread.rs:13 #24 thread_start () at src/libstd/sys/unix/thread.rs:79 #25 0x00007494babb458e in start_thread () from /lib64/libpthread.so.0 #26 0x00007494baac8683 in clone () from /lib64/libc.so.6 Thread 3 (Thread 0x7494ba03d700 (LWP 5320)): #0 0x00007494babba72c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x000057e239ac05b3 in wait () at src/libstd/sys/unix/condvar.rs:69 #2 wait () at src/libstd/sys_common/condvar.rs:41 #3 wait<()> () at src/libstd/sync/condvar.rs:204 #4 park () at src/libstd/thread/mod.rs:910 #5 0x000057e239a9e44b in futures_timer::global::run (timer=..., done=...) at /home/user/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-timer-0.1.1/src/global.rs:73 #6 0x000057e239a9de8c in futures_timer::global::HelperThread::new::{{closure}} () at /home/user/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-timer-0.1.1/src/global.rs:23 #7 0x000057e239aa4e02 in std::sys_common::backtrace::__rust_begin_short_backtrace (f=...) at /rustc/d595b113584f8f446957469951fd5d31adc2a44e/src/libstd/sys_common/backtrace.rs:136 #8 0x000057e239a9d7a1 in std::thread::Builder::spawn_unchecked::{{closure}}::{{closure}} () at /rustc/d595b113584f8f446957469951fd5d31adc2a44e/src/libstd/thread/mod.rs:469 #9 0x000057e239a9e601 in as core::ops::function::FnOnce<()>>::call_once (self=..., _args=()) at /rustc/d595b113584f8f446957469951fd5d31adc2a44e/src/libstd/panic.rs:309 #10 0x000057e239a9b167 in std::panicking::try::do_call (data=0x7494ba03c950 "\200\v\000") at /rustc/d595b113584f8f446957469951fd5d31adc2a44e/src/libstd/panicking.rs:293 #11 0x000057e239aceb7a in __rust_maybe_catch_panic () at src/libpanic_unwind/lib.rs:85 #12 0x000057e239a9b050 in std::panicking::try (f=...) at /rustc/d595b113584f8f446957469951fd5d31adc2a44e/src/libstd/panicking.rs:272 #13 0x000057e239a9e7a3 in std::panic::catch_unwind (f=...) at /rustc/d595b113584f8f446957469951fd5d31adc2a44e/src/libstd/panic.rs:388 #14 0x000057e239a9d58a in std::thread::Builder::spawn_unchecked::{{closure}} () at /rustc/d595b113584f8f446957469951fd5d31adc2a44e/src/libstd/thread/mod.rs:468 #15 0x000057e239aa50b4 in core::ops::function::FnOnce::call_once{{vtable-shim}} () at /rustc/d595b113584f8f446957469951fd5d31adc2a44e/src/libcore/ops/function.rs:231 #16 0x000057e239abfb2f in call_once<(),FnOnce<()>> () at /rustc/d595b113584f8f446957469951fd5d31adc2a44e/src/liballoc/boxed.rs:704 #17 0x000057e239ace1d0 in call_once<(),alloc::boxed::Box>> () at /rustc/d595b113584f8f446957469951fd5d31adc2a44e/src/liballoc/boxed.rs:704 #18 start_thread () at src/libstd/sys_common/thread.rs:13 #19 thread_start () at src/libstd/sys/unix/thread.rs:79 #20 0x00007494babb458e in start_thread () from /lib64/libpthread.so.0 #21 0x00007494baac8683 in clone () from /lib64/libc.so.6 Thread 4 (Thread 0x7494b8224700 (LWP 5336)): #0 0x00007494babba72c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x000057e239ab903e in std::sys::unix::condvar::Condvar::wait (self=0x74946c000ce0, mutex=0x74946c000cb0) at /rustc/d595b113584f8f446957469951fd5d31adc2a44e/src/libstd/sys/unix/condvar.rs:69 #2 0x000057e239ab8ff7 in std::sys_common::condvar::Condvar::wait (self=0x74946c000ce0, mutex=0x74946c000cb0) at /rustc/d595b113584f8f446957469951fd5d31adc2a44e/src/libstd/sys_common/condvar.rs:41 #3 0x000057e239ab71c2 in std::sync::condvar::Condvar::wait (self=0x74946c000d48, guard=...) at /rustc/d595b113584f8f446957469951fd5d31adc2a44e/src/libstd/sync/condvar.rs:204 #4 0x000057e239ab100e in futures::task_impl::std::ThreadNotify::park (self=0x74946c000d30) at /home/user/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-0.1.27/src/task_impl/std/mod.rs:513 #5 0x000057e239a37cb5 in futures::task_impl::std::>::wait_future::{{closure}} (notify=0x7494b8224638) at /home/user/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-0.1.27/src/task_impl/std/mod.rs:237 #6 0x000057e239a3797f in futures::task_impl::std::ThreadNotify::with_current::{{closure}} (notify=0x7494b8224638) at /home/user/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-0.1.27/src/task_impl/std/mod.rs:483 #7 0x000057e239a38603 in std::thread::local::LocalKey::try_with (self=0x57e239b1abc8, f=...) at /rustc/d595b113584f8f446957469951fd5d31adc2a44e/src/libstd/thread/local.rs:299 #8 0x000057e239a38402 in std::thread::local::LocalKey::with (self=0x57e239b1abc8, f=...) at /rustc/d595b113584f8f446957469951fd5d31adc2a44e/src/libstd/thread/local.rs:245 #9 0x000057e239a3793e in futures::task_impl::std::ThreadNotify::with_current (f=...) at /home/user/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-0.1.27/src/task_impl/std/mod.rs:483 #10 0x000057e239a2f504 in futures::task_impl::std::>::wait_future (self=0x7494b82236e0) at /home/user/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-0.1.27/src/task_impl/std/mod.rs:233 #11 0x000057e239a2db10 in futures::future::Future::wait (self=...) at /home/user/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-0.1.27/src/future/mod.rs:299 #12 0x000057e239a37434 in ratelimit::multiple::{{closure}} () at tests/ratelimit.rs:44 #13 0x000057e239a34e12 in std::sys_common::backtrace::__rust_begin_short_backtrace (f=...) at /rustc/d595b113584f8f446957469951fd5d31adc2a44e/src/libstd/sys_common/backtrace.rs:136 #14 0x000057e239a32680 in std::thread::Builder::spawn_unchecked::{{closure}}::{{closure}} () at /rustc/d595b113584f8f446957469951fd5d31adc2a44e/src/libstd/thread/mod.rs:469 #15 0x000057e239a34d60 in as core::ops::function::FnOnce<()>>::call_once (self=..., _args=()) at /rustc/d595b113584f8f446957469951fd5d31adc2a44e/src/libstd/panic.rs:309 #16 0x000057e239a333e5 in std::panicking::try::do_call (data=0x7494b8223a10 " \f\000") at /rustc/d595b113584f8f446957469951fd5d31adc2a44e/src/libstd/panicking.rs:293 #17 0x000057e239aceb7a in __rust_maybe_catch_panic () at src/libpanic_unwind/lib.rs:85 #18 0x000057e239a332c9 in std::panicking::try (f=...) at /rustc/d595b113584f8f446957469951fd5d31adc2a44e/src/libstd/panicking.rs:272 #19 0x000057e239a34da2 in std::panic::catch_unwind (f=...) at /rustc/d595b113584f8f446957469951fd5d31adc2a44e/src/libstd/panic.rs:388 #20 0x000057e239a3247e in std::thread::Builder::spawn_unchecked::{{closure}} () at /rustc/d595b113584f8f446957469951fd5d31adc2a44e/src/libstd/thread/mod.rs:468 #21 0x000057e239a33f33 in core::ops::function::FnOnce::call_once{{vtable-shim}} () at /rustc/d595b113584f8f446957469951fd5d31adc2a44e/src/libcore/ops/function.rs:231 #22 0x000057e239abfb2f in call_once<(),FnOnce<()>> () at /rustc/d595b113584f8f446957469951fd5d31adc2a44e/src/liballoc/boxed.rs:704 #23 0x000057e239ace1d0 in call_once<(),alloc::boxed::Box>> () at /rustc/d595b113584f8f446957469951fd5d31adc2a44e/src/liballoc/boxed.rs:704 #24 start_thread () at src/libstd/sys_common/thread.rs:13 #25 thread_start () at src/libstd/sys/unix/thread.rs:79 #26 0x00007494babb458e in start_thread () from /lib64/libpthread.so.0 #27 0x00007494baac8683 in clone () from /lib64/libc.so.6 [Inferior 1 (process 5315) detached] ```
antifuchs commented 5 years ago

I believe that whatever is going wrong, is happening in the .wait() function of 0.1 Futures, and not necessarily on our end. I can't be quite sure, but I was planning to port ratelimit_futures forward to the std futures anyway; maybe that'll fix this bug too.

antifuchs commented 4 years ago

This was indeed a bug with futures 0.1. I've started a new project since then, governor, which builds on std futures. I likely will abandon this project and ratelimit_meter in favor of governor, so this issue will remain open. I'd encourage you to upgrade! The interface is really similar, and works better!