tokio-rs / tokio-minihttp

Protocol implementation experimentations
Apache License 2.0
448 stars 50 forks source link

Panic in BufMut::put under load #19

Closed aptomaKetil closed 7 years ago

aptomaKetil commented 7 years ago

This may be the wrong place to report this, but it's not clear to me if the problem lies in bytes, tokio-* or in how those crates are used in this repo.

After the update to tokio-io, the examples panic when put under non-trivial load.

Trace:

RUST_BACKTRACE=full cargo run --example hello-world
    Finished dev [unoptimized + debuginfo] target(s) in 0.0 secs
     Running `target/debug/examples/hello-world`
thread 'main' panicked at 'assertion failed: self.remaining_mut() >= src.remaining()', /Users/ketilovre/.cargo/registry/src/github.com-1ecc6299db9ec823/bytes-0.4.1/src/buf/buf_mut.rs:227
stack backtrace:
   0:        0x105bd4513 - std::sys::imp::backtrace::tracing::imp::unwind_backtrace::h7b906ec435ded35c
   1:        0x105bd6354 - std::panicking::default_hook::{{closure}}::h44d92332f38028b9
   2:        0x105bd5f6d - std::panicking::default_hook::ha8486ead32221b33
   3:        0x105bd8ab7 - std::panicking::rust_panic_with_hook::hdbc3bba6a9dc0bb9
   4:        0x105b4e9e8 - std::panicking::begin_panic::h0f47c8d04ec91556
   5:        0x105b53852 - bytes::buf::buf_mut::BufMut::put::hc5f5bd330eeee7a9
   6:        0x105b57802 - <tokio_minihttp::response::FastWrite<'a> as core::fmt::Write>::write_str::h792143836f1f5834
   7:        0x105bfbc71 - core::fmt::write::h58310826cef1ccf3
   8:        0x105b578de - <tokio_minihttp::response::FastWrite<'a> as core::fmt::Write>::write_fmt::h2652f00186740201
   9:        0x105b57463 - tokio_minihttp::response::encode::h2185b8a1f48158f6
  10:        0x105b57bd7 - <tokio_minihttp::HttpCodec as tokio_io::framed_write::Encoder>::encode::h5ca278959f9b8bac
  11:        0x105b38ee3 - <tokio_io::framed::Fuse<T, U> as tokio_io::framed_write::Encoder>::encode::h85dd5e9bdfdc7b8c
  12:        0x105b35ef4 - <tokio_io::framed_write::FramedWrite2<T> as futures::sink::Sink>::start_send::h4130b348317693b0
  13:        0x105b34181 - <tokio_io::framed::Framed<T, U> as futures::sink::Sink>::start_send::h1bd88f74077d8308
  14:        0x105b0324f - <tokio_proto::simple::pipeline::lift::LiftTransport<T, E> as futures::sink::Sink>::start_send::hebe49de85d193657
  15:        0x105b02ab1 - <tokio_proto::streaming::pipeline::advanced::DispatchSink<T> as futures::sink::Sink>::start_send::h208d0f44e9991630
  16:        0x105b34f69 - <tokio_proto::buffer_one::BufferOne<S> as futures::sink::Sink>::start_send::hfbfc5c8c5430a97f
  17:        0x105b062c9 - tokio_proto::streaming::pipeline::advanced::assert_send::hf5dca39ad84063ac
  18:        0x105b2da78 - <tokio_proto::streaming::pipeline::advanced::Pipeline<T>>::write_in_message::h96142417c70f858b
  19:        0x105b2cb85 - <tokio_proto::streaming::pipeline::advanced::Pipeline<T>>::write_in_frames::hc73697a5153b606a
  20:        0x105b02deb - <tokio_proto::streaming::pipeline::advanced::Pipeline<T> as futures::future::Future>::poll::h5086eacdfcb8a0cd
  21:        0x105b283f7 - <futures::future::chain::Chain<A, B, C>>::poll::hdde88008f9a841a7
  22:        0x105b01ccb - <futures::future::and_then::AndThen<A, B, F> as futures::future::Future>::poll::h50c8f2cc62409c1c
  23:        0x105b39356 - <futures::future::map_err::MapErr<A, F> as futures::future::Future>::poll::h4a2d08077cc84387
  24:        0x105b7dfe1 - <alloc::boxed::Box<F> as futures::future::Future>::poll::h9a960269f9a76152
  25:        0x105b6913c - <futures::task_impl::Spawn<F>>::poll_future::{{closure}}::ha0b202d821be324b
  26:        0x105b6956e - <futures::task_impl::Spawn<T>>::enter::{{closure}}::hc64329dbab7744a3
  27:        0x105b80343 - futures::task_impl::set::{{closure}}::habfb871a03b7dd50
  28:        0x105b6aa51 - <std::thread::local::LocalKey<T>>::with::h9ae714bd6e7eb97f
  29:        0x105b800ef - futures::task_impl::set::h570c5abc832e47b3
  30:        0x105b6933e - <futures::task_impl::Spawn<T>>::enter::h647e6c6ef6939275
  31:        0x105b690bf - <futures::task_impl::Spawn<F>>::poll_future::hca31303e3d536d67
  32:        0x105b8c7d2 - tokio_core::reactor::Core::dispatch_task::{{closure}}::h44990ee43a8db0c0
  33:        0x105b65484 - <scoped_tls::ScopedKey<T>>::set::h99534fc75e0aad19
  34:        0x105b8c2ba - tokio_core::reactor::Core::dispatch_task::h27cfe92f36bc634c
  35:        0x105b8b625 - tokio_core::reactor::Core::dispatch::hfe5d7b05d4c77ffd
  36:        0x105b8b19d - tokio_core::reactor::Core::poll::h86199da1274f8a32
  37:        0x105b03a65 - tokio_core::reactor::Core::run::hed937874801fc25b
  38:        0x105b05641 - tokio_proto::tcp_server::serve::h2e7a3dcb2e45af2c
  39:        0x105b2773e - <tokio_proto::tcp_server::TcpServer<Kind, P>>::with_handle::h1e9b6824f4250ecf
  40:        0x105b28072 - <tokio_proto::tcp_server::TcpServer<Kind, P>>::serve::hac80897fbd6b48ac
  41:        0x105b3a3d6 - hello_world::main::h5741a7b6e7843651
  42:        0x105bd8785 - std::panicking::try::do_call::h24a2756282b9a31c
  43:        0x105bd9a6a - __rust_maybe_catch_panic
  44:        0x105bd8eb0 - std::rt::lang_start::hd19f94db0c6a490e
  45:        0x105b3a519 - main

This can be reproduced consistently by starting the example and using wrk or similar to generate load. Tested on OS X with rustc 1.17.0-nightly (a559452b0 2017-03-17) and rustc 1.16.0 (30cf806ef 2017-03-10), with and without --release. The same panic occurs in all cases.

carllerche commented 7 years ago

Thanks, this was due to BufMut::put not growing the buffer if it is full. extend will grow as needed.