search

tokio-rs / tokio

A runtime for writing reliable asynchronous applications with Rust. Provides I/O, networking, scheduling, timers, ...
https://tokio.rs
MIT License
25.42k stars 2.3k forks source link

Panic when running in basic runtime (current_thread) #1818

Closed izderadicka closed 4 years ago

izderadicka commented 4 years ago

Version

I have recently cloned version of tokio master branch

cargo tree | grep tokio
├── tokio v0.2.0-alpha.6 (https://github.com/tokio-rs/tokio.git#3ecaa6d9)
│   ├── bytes v0.5.0 (https://github.com/tokio-rs/bytes#44fb2061)
│   └── tokio-macros v0.2.0-alpha.6 (https://github.com/tokio-rs/tokio.git#3ecaa6d9)

Platform

Linux ivan-ThinkPad-T460 4.15.0-66-generic #75~16.04.1-Ubuntu SMP Tue Oct 1 14:01:08 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Nightly rust

$ rustc --version
rustc 1.41.0-nightly (5c5b8afd8 2019-11-16)

Description

I have simple proxy to connect outside through HTTP proxy (CONNECT method), recently updated to latest tokio and new futures. When running with basic runtime builder.basic_scheduler().enable_all().build()? program panicked - see full trace below. I'm not creating any custom futures or streams - just using basic TcpListener and TcpStream and async/await + few combinators. I'm testing with ab and wrk tool via local squid proxy to local nginx. Panick is caused by wrk running for 2 minutes - it's happens almost always. ab also has some issues - some connections timeout.

However if running with thread pool executor builder.threaded_scheduler().enable_all().build()? there are no such problema, everything is running just fine. For reference code is here https://github.com/izderadicka/ptunnel-rust/tree/new-futures.

Panic Stacktrace

thread 'main' panicked at 'assertion failed: prev.is_notified()', /home/ivan/.cargo/git/checkouts/tokio-377c595163f99a10/3ecaa6d/tokio/src/task/state.rs:79:9
stack backtrace:
   0:     0x563ba3457094 - backtrace::backtrace::libunwind::trace::h03a3268e8025f5e7
                               at /cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.40/src/backtrace/libunwind.rs:88
   1:     0x563ba3457094 - backtrace::backtrace::trace_unsynchronized::hfdd426654407b9b1
                               at /cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.40/src/backtrace/mod.rs:66
   2:     0x563ba3457094 - std::sys_common::backtrace::_print_fmt::hf13b3576a7afd1fd
                               at src/libstd/sys_common/backtrace.rs:84
   3:     0x563ba3457094 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::h6120b5cb7f3f6fb4
                               at src/libstd/sys_common/backtrace.rs:61
   4:     0x563ba3473f5c - core::fmt::write::h5519681e0b8a2078
                               at src/libcore/fmt/mod.rs:1030
   5:     0x563ba3452fd7 - std::io::Write::write_fmt::h244d0a52d18abc18
                               at src/libstd/io/mod.rs:1412
   6:     0x563ba345920e - std::sys_common::backtrace::_print::h3f6ea6d8a2e1ae5c
                               at src/libstd/sys_common/backtrace.rs:65
   7:     0x563ba345920e - std::sys_common::backtrace::print::hfaa195f2a5ad8cee
                               at src/libstd/sys_common/backtrace.rs:50
   8:     0x563ba345920e - std::panicking::default_hook::{{closure}}::heac2a010ba47da52
                               at src/libstd/panicking.rs:188
   9:     0x563ba3458f01 - std::panicking::default_hook::h7739a4bc79ee19ed
                               at src/libstd/panicking.rs:205
  10:     0x563ba345990b - std::panicking::rust_panic_with_hook::h4882a1e7ece8be98
                               at src/libstd/panicking.rs:464
  11:     0x563ba3184c53 - std::panicking::begin_panic::h57fc70b031fab58d
                               at /rustc/5c5b8afd80e6fa1d24632153cb2257c686041d41/src/libstd/panicking.rs:400
  12:     0x563ba3117074 - tokio::task::state::State::transition_to_running::hdb69c4c1d9e63ad3
                               at /home/ivan/.cargo/git/checkouts/tokio-377c595163f99a10/3ecaa6d/tokio/src/task/state.rs:79
  13:     0x563ba304d8c6 - tokio::task::harness::Harness<T,S>::poll::h58494483abc9a483
                               at /home/ivan/.cargo/git/checkouts/tokio-377c595163f99a10/3ecaa6d/tokio/src/task/harness.rs:66
  14:     0x563ba302f7d6 - tokio::task::raw::poll::hbbf21808eb697e6d
                               at /home/ivan/.cargo/git/checkouts/tokio-377c595163f99a10/3ecaa6d/tokio/src/task/raw.rs:150
  15:     0x563ba30fecb7 - tokio::task::raw::RawTask::poll::h6cbb0c4da3a0f292
                               at /home/ivan/.cargo/git/checkouts/tokio-377c595163f99a10/3ecaa6d/tokio/src/task/raw.rs:101
  16:     0x563ba304003c - tokio::task::Task<S>::run::h47e9945b9b5bfd59
                               at /home/ivan/.cargo/git/checkouts/tokio-377c595163f99a10/3ecaa6d/tokio/src/task/mod.rs:338
  17:     0x563ba3025cd6 - tokio::runtime::basic_scheduler::SchedulerPriv::tick::h9446f6faee42b3ab
                               at /home/ivan/.cargo/git/checkouts/tokio-377c595163f99a10/3ecaa6d/tokio/src/runtime/basic_scheduler.rs:210
  18:     0x563ba302622c - tokio::runtime::basic_scheduler::BasicScheduler<P>::block_on::{{closure}}::h7b62eb326224c544
                               at /home/ivan/.cargo/git/checkouts/tokio-377c595163f99a10/3ecaa6d/tokio/src/runtime/basic_scheduler.rs:162
  19:     0x563ba303a403 - tokio::runtime::global::with_state::{{closure}}::hd3ea234209f114c9
                               at /home/ivan/.cargo/git/checkouts/tokio-377c595163f99a10/3ecaa6d/tokio/src/runtime/global.rs:107
  20:     0x563ba302d005 - std::thread::local::LocalKey<T>::try_with::hb79055d7bf37dac0
                               at /rustc/5c5b8afd80e6fa1d24632153cb2257c686041d41/src/libstd/thread/local.rs:262
  21:     0x563ba302bb00 - std::thread::local::LocalKey<T>::with::h0a32a335f342170e
                               at /rustc/5c5b8afd80e6fa1d24632153cb2257c686041d41/src/libstd/thread/local.rs:239
  22:     0x563ba3039fba - tokio::runtime::global::with_state::hb9d04e01542ece73
                               at /home/ivan/.cargo/git/checkouts/tokio-377c595163f99a10/3ecaa6d/tokio/src/runtime/global.rs:90
  23:     0x563ba303a5b1 - tokio::runtime::global::with_basic_scheduler::h78610475f803d880
                               at /home/ivan/.cargo/git/checkouts/tokio-377c595163f99a10/3ecaa6d/tokio/src/runtime/global.rs:62
  24:     0x563ba3025f49 - tokio::runtime::basic_scheduler::BasicScheduler<P>::block_on::hcd8ae294a55553d9
                               at /home/ivan/.cargo/git/checkouts/tokio-377c595163f99a10/3ecaa6d/tokio/src/runtime/basic_scheduler.rs:141
  25:     0x563ba303ff10 - tokio::runtime::Runtime::block_on::{{closure}}::h2e477e20ef30010b
                               at /home/ivan/.cargo/git/checkouts/tokio-377c595163f99a10/3ecaa6d/tokio/src/runtime/mod.rs:409
  26:     0x563ba303a286 - tokio::runtime::global::with_state::{{closure}}::h920b2c2ed0b52e23
                               at /home/ivan/.cargo/git/checkouts/tokio-377c595163f99a10/3ecaa6d/tokio/src/runtime/global.rs:107
  27:     0x563ba302cd0d - std::thread::local::LocalKey<T>::try_with::h8f6b928fdd543fe6
                               at /rustc/5c5b8afd80e6fa1d24632153cb2257c686041d41/src/libstd/thread/local.rs:262
  28:     0x563ba302c094 - std::thread::local::LocalKey<T>::with::he8ad319be20f1f3a
                               at /rustc/5c5b8afd80e6fa1d24632153cb2257c686041d41/src/libstd/thread/local.rs:239
  29:     0x563ba3039ee4 - tokio::runtime::global::with_state::h215a3b8d2e9b0664
                               at /home/ivan/.cargo/git/checkouts/tokio-377c595163f99a10/3ecaa6d/tokio/src/runtime/global.rs:90
  30:     0x563ba303a620 - tokio::runtime::global::with_basic_scheduler::h99fda98209faa807
                               at /home/ivan/.cargo/git/checkouts/tokio-377c595163f99a10/3ecaa6d/tokio/src/runtime/global.rs:62
  31:     0x563ba3026315 - tokio::runtime::basic_scheduler::Spawner::enter::hf49bdc4aa20b6405
                               at /home/ivan/.cargo/git/checkouts/tokio-377c595163f99a10/3ecaa6d/tokio/src/runtime/basic_scheduler.rs:189
  32:     0x563ba3052225 - tokio::runtime::spawner::Spawner::enter::h32f4070007d4b031
                               at /home/ivan/.cargo/git/checkouts/tokio-377c595163f99a10/3ecaa6d/tokio/src/runtime/spawner.rs:30
  33:     0x563ba30513f6 - tokio::runtime::handle::Handle::enter::{{closure}}::{{closure}}::h1248d3c8cf024aa5
                               at /home/ivan/.cargo/git/checkouts/tokio-377c595163f99a10/3ecaa6d/tokio/src/runtime/handle.rs:36
  34:     0x563ba30258d3 - tokio::time::clock::Clock::enter::h6334982e1b7a44a3
                               at /home/ivan/.cargo/git/checkouts/tokio-377c595163f99a10/3ecaa6d/tokio/src/time/clock.rs:30
  35:     0x563ba3043ec3 - tokio::runtime::time::variant::with_default::hfc4a92ae5abbe67c
                               at /home/ivan/.cargo/git/checkouts/tokio-377c595163f99a10/3ecaa6d/tokio/src/runtime/time.rs:43
  36:     0x563ba3051317 - tokio::runtime::handle::Handle::enter::{{closure}}::h3dd0afce1c18d9bf
                               at /home/ivan/.cargo/git/checkouts/tokio-377c595163f99a10/3ecaa6d/tokio/src/runtime/handle.rs:36
  37:     0x563ba30268e6 - tokio::runtime::blocking::pool::Spawner::enter::{{closure}}::h90ef747fc3ceecf4
                               at /home/ivan/.cargo/git/checkouts/tokio-377c595163f99a10/3ecaa6d/tokio/src/runtime/blocking/pool.rs:191
  38:     0x563ba302d9d9 - std::thread::local::LocalKey<T>::try_with::he2242699852eb604
                               at /rustc/5c5b8afd80e6fa1d24632153cb2257c686041d41/src/libstd/thread/local.rs:262
  39:     0x563ba302c112 - std::thread::local::LocalKey<T>::with::heedb8e3d42d4932a
                               at /rustc/5c5b8afd80e6fa1d24632153cb2257c686041d41/src/libstd/thread/local.rs:239
  40:     0x563ba30267ab - tokio::runtime::blocking::pool::Spawner::enter::h47463a19b4554732
                               at /home/ivan/.cargo/git/checkouts/tokio-377c595163f99a10/3ecaa6d/tokio/src/runtime/blocking/pool.rs:176
  41:     0x563ba305124a - tokio::runtime::handle::Handle::enter::h41e77de350959f50
                               at /home/ivan/.cargo/git/checkouts/tokio-377c595163f99a10/3ecaa6d/tokio/src/runtime/handle.rs:33
  42:     0x563ba303fdf4 - tokio::runtime::Runtime::block_on::h8b788ab2a42ffba5
                               at /home/ivan/.cargo/git/checkouts/tokio-377c595163f99a10/3ecaa6d/tokio/src/runtime/mod.rs:406
  43:     0x563ba3044ae1 - ptunnel::main::hb4242273db81493e
                               at src/main.rs:44
  44:     0x563ba30414a0 - std::rt::lang_start::{{closure}}::h54a47b1e41226038
                               at /rustc/5c5b8afd80e6fa1d24632153cb2257c686041d41/src/libstd/rt.rs:61
  45:     0x563ba3459333 - std::rt::lang_start_internal::{{closure}}::hc2740ed8935d2053
                               at src/libstd/rt.rs:48
  46:     0x563ba3459333 - std::panicking::try::do_call::h3894402739883b06
                               at src/libstd/panicking.rs:287
  47:     0x563ba345d0da - __rust_maybe_catch_panic
                               at src/libpanic_unwind/lib.rs:86
  48:     0x563ba3459ded - std::panicking::try::h5db65b65b7947775
                               at src/libstd/panicking.rs:265
  49:     0x563ba3459ded - std::panic::catch_unwind::hf99e4cf1cf67ce18
                               at src/libstd/panic.rs:395
  50:     0x563ba3459ded - std::rt::lang_start_internal::h08e212477f161066
                               at src/libstd/rt.rs:47
  51:     0x563ba3041479 - std::rt::lang_start::h65b56a3e0826a8a1
                               at /rustc/5c5b8afd80e6fa1d24632153cb2257c686041d41/src/libstd/rt.rs:61
  52:     0x563ba3044cda - main
  53:     0x7fe94427f830 - __libc_start_main
  54:     0x563ba3018ea9 - _start
  55:                0x0 - <unknown>
swarmer commented 4 years ago

I'm not able to reproduce this, even with this specific nightly and tokio commit. Maybe you can build a docker image that panics on your machine when you run it with default command, so all possible differences except kernel and hardware are excluded? Also, can you try to reproduce this on another machine?

izderadicka commented 4 years ago

I created docker compose file to test the issue tokio-issue.zip On local computer I was also able to duplicate the panic in docker, but not so often - app 10% of runs (while running locally panic happened in more then 50% of cases). Another interesting thing is that locally I was able to observe panic only in debug build, release build run ok (but have not done intensive tests). Looks like problem is some race condition, which demonstrates only in certain circumstances. Will try on another computer.

izderadicka commented 4 years ago

On another system (fresh ubuntu 18.04 with latest docker and docker-compose) I'm able to replicate issue relatively reliably - using docker-compose file from previous comment - 3 runs, 3 times got panic.

swarmer commented 4 years ago

:+1: Not sure if I was doing something different before or I'm luckier today or what, but I can reproduce the panic using the compose file. It's not super reliable - doesn't happen every time. Normally it's the usual stack trace, but once I got something even more spicy:

Collapsed 

ptunnel_1  | thread 'main' panicked at 'assertion failed: prev.is_notified()', /root/.cargo/git/checkouts/tokio-377c595163f99a10/3ecaa6d/tokio/src/task/state.rs:79:9
ptunnel_1  | stack backtrace:
ptunnel_1  | *** Error in `/src/ptunnel-rust/target/debug/ptunnel': corrupted double-linked list: 0x000055bf31dd38f0 ***
ptunnel_1  | ======= Backtrace: =========
ptunnel_1  | /lib/x86_64-linux-gnu/libc.so.6(+0x777e5)[0x7f2cfa3727e5]
ptunnel_1  | /lib/x86_64-linux-gnu/libc.so.6(+0x82970)[0x7f2cfa37d970]
ptunnel_1  | /lib/x86_64-linux-gnu/libc.so.6(__libc_malloc+0x54)[0x7f2cfa37f184]
ptunnel_1  | /src/ptunnel-rust/target/debug/ptunnel(+0x69e713)[0x55bf30a22713]
ptunnel_1  | /src/ptunnel-rust/target/debug/ptunnel(+0x69f603)[0x55bf30a23603]
ptunnel_1  | /src/ptunnel-rust/target/debug/ptunnel(+0x69d098)[0x55bf30a21098]
ptunnel_1  | /src/ptunnel-rust/target/debug/ptunnel(+0x69d24c)[0x55bf30a2124c]
ptunnel_1  | /lib/x86_64-linux-gnu/libc.so.6(dl_iterate_phdr+0xf4)[0x7f2cfa43e0e4]
ptunnel_1  | /src/ptunnel-rust/target/debug/ptunnel(+0x69b5db)[0x55bf30a1f5db]
ptunnel_1  | /src/ptunnel-rust/target/debug/ptunnel(+0x69b21f)[0x55bf30a1f21f]
ptunnel_1  | /src/ptunnel-rust/target/debug/ptunnel(+0x69b293)[0x55bf30a1f293]
ptunnel_1  | /src/ptunnel-rust/target/debug/ptunnel(_ZN9backtrace9symbolize12libbacktrace7resolve17h8cf09db01d00fa07E+0xd8)[0x55bf30a1ee68]
ptunnel_1  | /src/ptunnel-rust/target/debug/ptunnel(+0x693c88)[0x55bf30a17c88]
ptunnel_1  | /src/ptunnel-rust/target/debug/ptunnel(+0x69ab17)[0x55bf30a1eb17]
ptunnel_1  | /lib/x86_64-linux-gnu/libgcc_s.so.1(_Unwind_Backtrace+0x49)[0x7f2cfa6d5649]
ptunnel_1  | /src/ptunnel-rust/target/debug/ptunnel(+0x6935d4)[0x55bf30a175d4]
ptunnel_1  | /src/ptunnel-rust/target/debug/ptunnel(_ZN4core3fmt5write17h01edf6dd68a42c9cE+0x34c)[0x55bf30a34bbc]
ptunnel_1  | /src/ptunnel-rust/target/debug/ptunnel(+0x68f4e7)[0x55bf30a134e7]
ptunnel_1  | /src/ptunnel-rust/target/debug/ptunnel(+0x695ae5)[0x55bf30a19ae5]
ptunnel_1  | /src/ptunnel-rust/target/debug/ptunnel(+0x6957d6)[0x55bf30a197d6]
ptunnel_1  | /src/ptunnel-rust/target/debug/ptunnel(_ZN3std9panicking20rust_panic_with_hook17h825f041245da8739E+0x185)[0x55bf30a1a1e5]
ptunnel_1  | /src/ptunnel-rust/target/debug/ptunnel(+0x3439b3)[0x55bf306c79b3]
ptunnel_1  | /src/ptunnel-rust/target/debug/ptunnel(_ZN5tokio4task5state5State21transition_to_running17h49ec6cda411140a3E+0x84)[0x55bf30667514]
ptunnel_1  | /src/ptunnel-rust/target/debug/ptunnel(+0x1e1b76)[0x55bf30565b76]
ptunnel_1  | /src/ptunnel-rust/target/debug/ptunnel(+0x1f6546)[0x55bf3057a546]
ptunnel_1  | /src/ptunnel-rust/target/debug/ptunnel(+0x2cc417)[0x55bf30650417]
ptunnel_1  | /src/ptunnel-rust/target/debug/ptunnel(+0x1df79c)[0x55bf3056379c]
ptunnel_1  | /src/ptunnel-rust/target/debug/ptunnel(+0x1f5bc0)[0x55bf30579bc0]
ptunnel_1  | /src/ptunnel-rust/target/debug/ptunnel(+0x1f60fc)[0x55bf3057a0fc]
ptunnel_1  | /src/ptunnel-rust/target/debug/ptunnel(+0x1e91b3)[0x55bf3056d1b3]
ptunnel_1  | /src/ptunnel-rust/target/debug/ptunnel(+0x1eda15)[0x55bf30571a15]
ptunnel_1  | /src/ptunnel-rust/target/debug/ptunnel(+0x1ec650)[0x55bf30570650]
ptunnel_1  | /src/ptunnel-rust/target/debug/ptunnel(+0x1e8e1a)[0x55bf3056ce1a]
ptunnel_1  | /src/ptunnel-rust/target/debug/ptunnel(+0x1e9571)[0x55bf3056d571]
ptunnel_1  | /src/ptunnel-rust/target/debug/ptunnel(+0x1f5e19)[0x55bf30579e19]
ptunnel_1  | /src/ptunnel-rust/target/debug/ptunnel(+0x1f6d50)[0x55bf3057ad50]
ptunnel_1  | /src/ptunnel-rust/target/debug/ptunnel(+0x1e9346)[0x55bf3056d346]
ptunnel_1  | /src/ptunnel-rust/target/debug/ptunnel(+0x1ee84d)[0x55bf3057284d]
ptunnel_1  | /src/ptunnel-rust/target/debug/ptunnel(+0x1ebbe4)[0x55bf3056fbe4]
ptunnel_1  | /src/ptunnel-rust/target/debug/ptunnel(+0x1e8f14)[0x55bf3056cf14]
ptunnel_1  | /src/ptunnel-rust/target/debug/ptunnel(+0x1e9510)[0x55bf3056d510]
ptunnel_1  | /src/ptunnel-rust/target/debug/ptunnel(+0x1f61e5)[0x55bf3057a1e5]
ptunnel_1  | /src/ptunnel-rust/target/debug/ptunnel(+0x1f8bf5)[0x55bf3057cbf5]
ptunnel_1  | /src/ptunnel-rust/target/debug/ptunnel(+0x1f5226)[0x55bf30579226]
ptunnel_1  | /src/ptunnel-rust/target/debug/ptunnel(+0x1eb333)[0x55bf3056f333]
ptunnel_1  | /src/ptunnel-rust/target/debug/ptunnel(+0x2092d3)[0x55bf3058d2d3]
ptunnel_1  | /src/ptunnel-rust/target/debug/ptunnel(+0x1f5147)[0x55bf30579147]
ptunnel_1  | /src/ptunnel-rust/target/debug/ptunnel(+0x1eb4d6)[0x55bf3056f4d6]
ptunnel_1  | /src/ptunnel-rust/target/debug/ptunnel(+0x1ed6d9)[0x55bf305716d9]
ptunnel_1  | /src/ptunnel-rust/target/debug/ptunnel(+0x1ebdc2)[0x55bf3056fdc2]
ptunnel_1  | /src/ptunnel-rust/target/debug/ptunnel(+0x1eb39b)[0x55bf3056f39b]
ptunnel_1  | /src/ptunnel-rust/target/debug/ptunnel(+0x1f507a)[0x55bf3057907a]
ptunnel_1  | /src/ptunnel-rust/target/debug/ptunnel(+0x1f6c34)[0x55bf3057ac34]
ptunnel_1  | /src/ptunnel-rust/target/debug/ptunnel(+0x1fdf91)[0x55bf30581f91]
ptunnel_1  | /src/ptunnel-rust/target/debug/ptunnel(+0x1dfdf0)[0x55bf30563df0]
ptunnel_1  | /src/ptunnel-rust/target/debug/ptunnel(+0x695c13)[0x55bf30a19c13]
ptunnel_1  | /src/ptunnel-rust/target/debug/ptunnel(__rust_maybe_catch_panic+0x1a)[0x55bf30a1db4a]
ptunnel_1  | /src/ptunnel-rust/target/debug/ptunnel(_ZN3std2rt19lang_start_internal17h409d4f2fe51133b0E+0x35d)[0x55bf30a1a6dd]
ptunnel_1  | /src/ptunnel-rust/target/debug/ptunnel(+0x1dfdc9)[0x55bf30563dc9]
ptunnel_1  | /src/ptunnel-rust/target/debug/ptunnel(+0x1fe18a)[0x55bf3058218a]
ptunnel_1  | /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0)[0x7f2cfa31b830]
ptunnel_1  | /src/ptunnel-rust/target/debug/ptunnel(+0x1dc6d9)[0x55bf305606d9]
ptunnel_1  | ======= Memory map: ========
ptunnel_1  | 55bf30384000-55bf30b8a000 r-xp 00000000 fd:03 13769026                   /src/ptunnel-rust/target/debug/ptunnel
ptunnel_1  | 55bf30d89000-55bf30dde000 r--p 00805000 fd:03 13769026                   /src/ptunnel-rust/target/debug/ptunnel
ptunnel_1  | 55bf30dde000-55bf30ddf000 rw-p 0085a000 fd:03 13769026                   /src/ptunnel-rust/target/debug/ptunnel
ptunnel_1  | 55bf31d5c000-55bf31ded000 rw-p 00000000 00:00 0                          [heap]
ptunnel_1  | 7f2c90000000-7f2c90021000 rw-p 00000000 00:00 0
ptunnel_1  | 7f2c90021000-7f2c94000000 ---p 00000000 00:00 0
ptunnel_1  | 7f2c94000000-7f2c94021000 rw-p 00000000 00:00 0
ptunnel_1  | 7f2c94021000-7f2c98000000 ---p 00000000 00:00 0
ptunnel_1  | 7f2c98000000-7f2c98021000 rw-p 00000000 00:00 0
ptunnel_1  | 7f2c98021000-7f2c9c000000 ---p 00000000 00:00 0
ptunnel_1  | 7f2c9c000000-7f2c9c021000 rw-p 00000000 00:00 0
ptunnel_1  | 7f2c9c021000-7f2ca0000000 ---p 00000000 00:00 0
ptunnel_1  | 7f2ca0000000-7f2ca0021000 rw-p 00000000 00:00 0
ptunnel_1  | 7f2ca0021000-7f2ca4000000 ---p 00000000 00:00 0
ptunnel_1  | 7f2ca4000000-7f2ca4021000 rw-p 00000000 00:00 0
ptunnel_1  | 7f2ca4021000-7f2ca8000000 ---p 00000000 00:00 0
ptunnel_1  | 7f2ca8000000-7f2ca8021000 rw-p 00000000 00:00 0
ptunnel_1  | 7f2ca8021000-7f2cac000000 ---p 00000000 00:00 0
ptunnel_1  | 7f2cac000000-7f2cac021000 rw-p 00000000 00:00 0
ptunnel_1  | 7f2cac021000-7f2cb0000000 ---p 00000000 00:00 0
ptunnel_1  | 7f2cb0000000-7f2cb0021000 rw-p 00000000 00:00 0
ptunnel_1  | 7f2cb0021000-7f2cb4000000 ---p 00000000 00:00 0
ptunnel_1  | 7f2cb4000000-7f2cb4021000 rw-p 00000000 00:00 0
ptunnel_1  | 7f2cb4021000-7f2cb8000000 ---p 00000000 00:00 0
ptunnel_1  | 7f2cb8000000-7f2cb8021000 rw-p 00000000 00:00 0
ptunnel_1  | 7f2cb8021000-7f2cbc000000 ---p 00000000 00:00 0
ptunnel_1  | 7f2cbc000000-7f2cbc021000 rw-p 00000000 00:00 0
ptunnel_1  | 7f2cbc021000-7f2cc0000000 ---p 00000000 00:00 0
ptunnel_1  | 7f2cc0000000-7f2cc0021000 rw-p 00000000 00:00 0
ptunnel_1  | 7f2cc0021000-7f2cc4000000 ---p 00000000 00:00 0
ptunnel_1  | 7f2cc4000000-7f2cc4021000 rw-p 00000000 00:00 0
ptunnel_1  | 7f2cc4021000-7f2cc8000000 ---p 00000000 00:00 0
ptunnel_1  | 7f2cc8f4c000-7f2ccabf6000 rw-p 00000000 00:00 0
ptunnel_1  | 7f2ccabf6000-7f2ccabf7000 ---p 00000000 00:00 0
ptunnel_1  | 7f2ccabf7000-7f2ccadf7000 rw-p 00000000 00:00 0
ptunnel_1  | 7f2ccadf7000-7f2ccadf8000 ---p 00000000 00:00 0
ptunnel_1  | 7f2ccadf8000-7f2ccaff8000 rw-p 00000000 00:00 0
ptunnel_1  | 7f2ccaff8000-7f2ccaff9000 ---p 00000000 00:00 0
ptunnel_1  | 7f2ccaff9000-7f2ccb1f9000 rw-p 00000000 00:00 0
ptunnel_1  | 7f2ccb1f9000-7f2ccb1fa000 ---p 00000000 00:00 0
ptunnel_1  | 7f2ccb1fa000-7f2ccb3fa000 rw-p 00000000 00:00 0
ptunnel_1  | 7f2ccb3fa000-7f2ccb3fb000 ---p 00000000 00:00 0
ptunnel_1  | 7f2ccb3fb000-7f2ccb5fb000 rw-p 00000000 00:00 0
ptunnel_1  | 7f2ccb5fb000-7f2ccb5fc000 ---p 00000000 00:00 0
ptunnel_1  | 7f2ccb5fc000-7f2ccb7fc000 rw-p 00000000 00:00 0
ptunnel_1  | 7f2ccb7fc000-7f2ccb7fd000 ---p 00000000 00:00 0
ptunnel_1  | 7f2ccb7fd000-7f2ccb9fd000 rw-p 00000000 00:00 0
ptunnel_1  | 7f2ccb9fd000-7f2ccb9fe000 ---p 00000000 00:00 0
ptunnel_1  | 7f2ccb9fe000-7f2ccbbfe000 rw-p 00000000 00:00 0
ptunnel_1  | 7f2ccbbfe000-7f2ccbbff000 ---p 00000000 00:00 0
ptunnel_1  | 7f2ccbbff000-7f2ccbdff000 rw-p 00000000 00:00 0
ptunnel_1  | 7f2ccbdff000-7f2ccbe00000 ---p 00000000 00:00 0
ptunnel_1  | 7f2ccbe00000-7f2ccc000000 rw-p 00000000 00:00 0
ptunnel_1  | 7f2ccc000000-7f2ccc021000 rw-p 00000000 00:00 0
ptunnel_1  | 7f2ccc021000-7f2cd0000000 ---p 00000000 00:00 0
ptunnel_1  | 7f2cd0000000-7f2cd0021000 rw-p 00000000 00:00 0
ptunnel_1  | 7f2cd0021000-7f2cd4000000 ---p 00000000 00:00 0
ptunnel_1  | 7f2cd4000000-7f2cd4021000 rw-p 00000000 00:00 0
ptunnel_1  | 7f2cd4021000-7f2cd8000000 ---p 00000000 00:00 0
ptunnel_1  | 7f2cd8000000-7f2cd8021000 rw-p 00000000 00:00 0
ptunnel_1  | 7f2cd8021000-7f2cdc000000 ---p 00000000 00:00 0
ptunnel_1  | 7f2cdc000000-7f2cdc021000 rw-p 00000000 00:00 0
ptunnel_1  | 7f2cdc021000-7f2ce0000000 ---p 00000000 00:00 0
ptunnel_1  | 7f2ce0000000-7f2ce0021000 rw-p 00000000 00:00 0
ptunnel_1  | 7f2ce0021000-7f2ce4000000 ---p 00000000 00:00 0
ptunnel_1  | 7f2ce4000000-7f2ce4021000 rw-p 00000000 00:00 0
ptunnel_1  | 7f2ce4021000-7f2ce8000000 ---p 00000000 00:00 0
ptunnel_1  | 7f2ce8000000-7f2ce8021000 rw-p 00000000 00:00 0
ptunnel_1  | 7f2ce8021000-7f2cec000000 ---p 00000000 00:00 0
ptunnel_1  | 7f2cec000000-7f2cec021000 rw-p 00000000 00:00 0
ptunnel_1  | 7f2cec021000-7f2cf0000000 ---p 00000000 00:00 0
ptunnel_1  | 7f2cf0000000-7f2cf0021000 rw-p 00000000 00:00 0
ptunnel_1  | 7f2cf0021000-7f2cf4000000 ---p 00000000 00:00 0
ptunnel_1  | 7f2cf4000000-7f2cf4021000 rw-p 00000000 00:00 0
ptunnel_1  | 7f2cf4021000-7f2cf8000000 ---p 00000000 00:00 0
ptunnel_1  | 7f2cf80b9000-7f2cf80ba000 ---p 00000000 00:00 0
ptunnel_1  | 7f2cf80ba000-7f2cf82ba000 rw-p 00000000 00:00 0
ptunnel_1  | 7f2cf82ba000-7f2cf82bb000 ---p 00000000 00:00 0
ptunnel_1  | 7f2cf82bb000-7f2cf84bb000 rw-p 00000000 00:00 0
ptunnel_1  | 7f2cf84bb000-7f2cf84bc000 ---p 00000000 00:00 0
ptunnel_1  | 7f2cf84bc000-7f2cf86bc000 rw-p 00000000 00:00 0
ptunnel_1  | 7f2cf86bc000-7f2cf86bd000 ---p 00000000 00:00 0
ptunnel_1  | 7f2cf86bd000-7f2cf88bd000 rw-p 00000000 00:00 0
ptunnel_1  | 7f2cf88bd000-7f2cf88be000 ---p 00000000 00:00 0
ptunnel_1  | 7f2cf88be000-7f2cf8abe000 rw-p 00000000 00:00 0
ptunnel_1  | 7f2cf8abe000-7f2cf8abf000 ---p 00000000 00:00 0
ptunnel_1  | 7f2cf8abf000-7f2cf8cbf000 rw-p 00000000 00:00 0
ptunnel_1  | 7f2cf8cbf000-7f2cf8cc0000 ---p 00000000 00:00 0
ptunnel_1  | 7f2cf8cc0000-7f2cf8ec0000 rw-p 00000000 00:00 0
ptunnel_1  | 7f2cf8ec0000-7f2cf8ec1000 ---p 00000000 00:00 0
ptunnel_1  | 7f2cf8ec1000-7f2cf90c1000 rw-p 00000000 00:00 0
ptunnel_1  | 7f2cf90c1000-7f2cf90c2000 ---p 00000000 00:00 0
ptunnel_1  | 7f2cf90c2000-7f2cf92c2000 rw-p 00000000 00:00 0
ptunnel_1  | 7f2cf92c2000-7f2cf92c3000 ---p 00000000 00:00 0
ptunnel_1  | 7f2cf92c3000-7f2cf94c3000 rw-p 00000000 00:00 0
ptunnel_1  | 7f2cf94c3000-7f2cf94c4000 ---p 00000000 00:00 0
ptunnel_1  | 7f2cf94c4000-7f2cf96c4000 rw-p 00000000 00:00 0
ptunnel_1  | 7f2cf96c4000-7f2cf96db000 r-xp 00000000 fd:03 12987006                   /lib/x86_64-linux-gnu/libresolv-2.23.so
ptunnel_1  | 7f2cf96db000-7f2cf98db000 ---p 00017000 fd:03 12987006                   /lib/x86_64-linux-gnu/libresolv-2.23.so
ptunnel_1  | 7f2cf98db000-7f2cf98dc000 r--p 00017000 fd:03 12987006                   /lib/x86_64-linux-gnu/libresolv-2.23.so
ptunnel_1  | 7f2cf98dc000-7f2cf98dd000 rw-p 00018000 fd:03 12987006                   /lib/x86_64-linux-gnu/libresolv-2.23.so
ptunnel_1  | 7f2cf98dd000-7f2cf98df000 rw-p 00000000 00:00 0
ptunnel_1  | 7f2cf98df000-7f2cf98e4000 r-xp 00000000 fd:03 12986981                   /lib/x86_64-linux-gnu/libnss_dns-2.23.so
ptunnel_1  | 7f2cf98e4000-7f2cf9ae4000 ---p 00005000 fd:03 12986981                   /lib/x86_64-linux-gnu/libnss_dns-2.23.so
ptunnel_1  | 7f2cf9ae4000-7f2cf9ae5000 r--p 00005000 fd:03 12986981                   /lib/x86_64-linux-gnu/libnss_dns-2.23.so
ptunnel_1  | 7f2cf9ae5000-7f2cf9ae6000 rw-p 00006000 fd:03 12986981                   /lib/x86_64-linux-gnu/libnss_dns-2.23.so
ptunnel_1  | 7f2cf9ae6000-7f2cf9ae7000 ---p 00000000 00:00 0
ptunnel_1  | 7f2cf9ae7000-7f2cf9ce7000 rw-p 00000000 00:00 0
ptunnel_1  | 7f2cf9ce7000-7f2cf9cf2000 r-xp 00000000 fd:03 12986983                   /lib/x86_64-linux-gnu/libnss_files-2.23.so
ptunnel_1  | 7f2cf9cf2000-7f2cf9ef1000 ---p 0000b000 fd:03 12986983                   /lib/x86_64-linux-gnu/libnss_files-2.23.so
ptunnel_1  | 7f2cf9ef1000-7f2cf9ef2000 r--p 0000a000 fd:03 12986983                   /lib/x86_64-linux-gnu/libnss_files-2.23.so
ptunnel_1  | 7f2cf9ef2000-7f2cf9ef3000 rw-p 0000b000 fd:03 12986983                   /lib/x86_64-linux-gnu/libnss_files-2.23.so
ptunnel_1  | 7f2cf9ef3000-7f2cf9ef9000 rw-p 00000000 00:00 0
ptunnel_1  | 7f2cf9ef9000-7f2cf9efa000 ---p 00000000 00:00 0
ptunnel_1  | 7f2cf9efa000-7f2cfa0fa000 rw-p 00000000 00:00 0
ptunnel_1  | 7f2cfa0fa000-7f2cfa0fb000 ---p 00000000 00:00 0
ptunnel_1  | 7f2cfa0fb000-7f2cfa2fb000 rw-p 00000000 00:00 0
ptunnel_1  | 7f2cfa2fb000-7f2cfa4bb000 r-xp 00000000 fd:03 12986934                   /lib/x86_64-linux-gnu/libc-2.23.so
ptunnel_1  | 7f2cfa4bb000-7f2cfa6bb000 ---p 001c0000 fd:03 12986934                   /lib/x86_64-linux-gnu/libc-2.23.so
ptunnel_1  | 7f2cfa6bb000-7f2cfa6bf000 r--p 001c0000 fd:03 12986934                   /lib/x86_64-linux-gnu/libc-2.23.so
ptunnel_1  | 7f2cfa6bf000-7f2cfa6c1000 rw-p 001c4000 fd:03 12986934                   /lib/x86_64-linux-gnu/libc-2.23.so
ptunnel_1  | 7f2cfa6c1000-7f2cfa6c5000 rw-p 00000000 00:00 0
ptunnel_1  | 7f2cfa6c5000-7f2cfa6db000 r-xp 00000000 fd:03 12986955                   /lib/x86_64-linux-gnu/libgcc_s.so.1
ptunnel_1  | 7f2cfa6db000-7f2cfa8da000 ---p 00016000 fd:03 12986955                   /lib/x86_64-linux-gnu/libgcc_s.so.1
ptunnel_1  | 7f2cfa8da000-7f2cfa8db000 rw-p 00015000 fd:03 12986955                   /lib/x86_64-linux-gnu/libgcc_s.so.1
ptunnel_1  | 7f2cfa8db000-7f2cfa8f3000 r-xp 00000000 fd:03 12987002                   /lib/x86_64-linux-gnu/libpthread-2.23.so
ptunnel_1  | 7f2cfa8f3000-7f2cfaaf2000 ---p 00018000 fd:03 12987002                   /lib/x86_64-linux-gnu/libpthread-2.23.so
ptunnel_1  | 7f2cfaaf2000-7f2cfaaf3000 r--p 00017000 fd:03 12987002                   /lib/x86_64-linux-gnu/libpthread-2.23.so
ptunnel_1  | 7f2cfaaf3000-7f2cfaaf4000 rw-p 00018000 fd:03 12987002                   /lib/x86_64-linux-gnu/libpthread-2.23.so
ptunnel_1  | 7f2cfaaf4000-7f2cfaaf8000 rw-p 00000000 00:00 0
ptunnel_1  | 7f2cfaaf8000-7f2cfaaff000 r-xp 00000000 fd:03 12987008                   /lib/x86_64-linux-gnu/librt-2.23.so
ptunnel_1  | 7f2cfaaff000-7f2cfacfe000 ---p 00007000 fd:03 12987008                   /lib/x86_64-linux-gnu/librt-2.23.so
ptunnel_1  | 7f2cfacfe000-7f2cfacff000 r--p 00006000 fd:03 12987008                   /lib/x86_64-linux-gnu/librt-2.23.so
ptunnel_1  | 7f2cfacff000-7f2cfad00000 rw-p 00007000 fd:03 12987008                   /lib/x86_64-linux-gnu/librt-2.23.so
ptunnel_1  | 7f2cfad00000-7f2cfad03000 r-xp 00000000 fd:03 12986947                   /lib/x86_64-linux-gnu/libdl-2.23.so
ptunnel_1  | 7f2cfad03000-7f2cfaf02000 ---p 00003000 fd:03 12986947                   /lib/x86_64-linux-gnu/libdl-2.23.so
ptunnel_1  | 7f2cfaf02000-7f2cfaf03000 r--p 00002000 fd:03 12986947                   /lib/x86_64-linux-gnu/libdl-2.23.so
ptunnel_1  | 7f2cfaf03000-7f2cfaf04000 rw-p 00003000 fd:03 12986947                   /lib/x86_64-linux-gnu/libdl-2.23.so
ptunnel_1  | 7f2cfaf04000-7f2cfaf2a000 r-xp 00000000 fd:03 12986914                   /lib/x86_64-linux-gnu/ld-2.23.so
ptunnel_1  | 7f2cfaf9b000-7f2cfb02f000 rw-p 00000000 00:00 0
ptunnel_1  | 7f2cfb0f1000-7f2cfb129000 rw-p 00000000 00:00 0
ptunnel_1  | 7f2cfb129000-7f2cfb12a000 r--p 00025000 fd:03 12986914                   /lib/x86_64-linux-gnu/ld-2.23.so
ptunnel_1  | 7f2cfb12a000-7f2cfb12b000 rw-p 00026000 fd:03 12986914                   /lib/x86_64-linux-gnu/ld-2.23.so
ptunnel_1  | 7f2cfb12b000-7f2cfb12c000 rw-p 00000000 00:00 0
ptunnel_1  | 7ffd4d2e3000-7ffd4d304000 rw-p 00000000 00:00 0                          [stack]
ptunnel_1  | 7ffd4d396000-7ffd4d399000 r--p 00000000 00:00 0                          [vvar]
ptunnel_1  | 7ffd4d399000-7ffd4d39a000 r-xp 00000000 00:00 0                          [vdso]
ptunnel_1  | ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0                  [vsyscall]
tokioissue_ptunnel_1 exited with code 139
carllerche commented 4 years ago

Could you try again w/ Tokio v0.2.2? There was a basic_scheduler bug fix included in the release that feels related to this issue.

https://github.com/tokio-rs/tokio/releases/tag/tokio-0.2.2

izderadicka commented 4 years ago

With v0.2.2 everything seems to be working fine. @swarmer if you rebuild ptunnel image you can also check.

swarmer commented 4 years ago

I can't reproduce it either :+1:.

carllerche commented 4 years ago

Great! Fixed!