hyperium / h2

HTTP 2.0 client & server implementation for Rust.
MIT License
1.34k stars 269 forks source link

Assertion failure panic when being fed fuzzed input #581

Closed 5225225 closed 2 years ago

5225225 commented 2 years ago

I used rocket (0.5.0-rc.1) as a handy test case for this, but I found the bug when fuzzing hyper, and it seems to be in this repo.

To reproduce, feed the uploaded file into a rocket instance (base64 -d ... | nc localhost 8000)

DVBSSSAqIEhUVFAvMi4wDQoNClNNDQoNCgAAKgEAAAAAAKa8jry8vLy8vLy8vLy8vLy8vLy8vLy8
vLysvLy8pqampqampqaupqampqampqamJqampqamplpZWVl5WVlXpqampqaAgICAgICAgICAgICA
gICA
#[macro_use] extern crate rocket;

#[get("/")]
fn index() -> &'static str {
    "Hello, world!"
}

#[launch]
fn rocket() -> _ {
    rocket::build().mount("/", routes![index])
}

I just used the hello world, it likely really doesn't matter.

thread 'rocket-worker-thread' panicked at 'assertion failed: !id.is_zero()', /home/jess/.cargo/registry/src/github.com-1ecc6299db9ec823/h2-0.3.7/src/proto/peer.rs:54:9
stack backtrace:
   0: rust_begin_unwind
             at /rustc/532d2b14c05f9bc20b2d27cbb5f4550d28343a36/library/std/src/panicking.rs:498:5
   1: core::panicking::panic_fmt
             at /rustc/532d2b14c05f9bc20b2d27cbb5f4550d28343a36/library/core/src/panicking.rs:107:14
   2: core::panicking::panic
             at /rustc/532d2b14c05f9bc20b2d27cbb5f4550d28343a36/library/core/src/panicking.rs:48:5
   3: h2::proto::peer::Dyn::is_local_init
             at /home/jess/.cargo/registry/src/github.com-1ecc6299db9ec823/h2-0.3.7/src/proto/peer.rs:54:9
   4: h2::proto::streams::streams::Inner::send_reset
             at /home/jess/.cargo/registry/src/github.com-1ecc6299db9ec823/h2-0.3.7/src/proto/streams/streams.rs:880:20
   5: h2::proto::streams::streams::DynStreams<B>::send_reset
             at /home/jess/.cargo/registry/src/github.com-1ecc6299db9ec823/h2-0.3.7/src/proto/streams/streams.rs:349:9
   6: h2::proto::connection::DynConnection<B>::handle_poll2_result
             at /home/jess/.cargo/registry/src/github.com-1ecc6299db9ec823/h2-0.3.7/src/proto/connection.rs:427:17
   7: h2::proto::connection::Connection<T,P,B>::poll
             at /home/jess/.cargo/registry/src/github.com-1ecc6299db9ec823/h2-0.3.7/src/proto/connection.rs:267:21
   8: h2::server::Connection<T,B>::poll_closed
             at /home/jess/.cargo/registry/src/github.com-1ecc6299db9ec823/h2-0.3.7/src/server.rs:487:9
   9: h2::server::Connection<T,B>::poll_accept
             at /home/jess/.cargo/registry/src/github.com-1ecc6299db9ec823/h2-0.3.7/src/server.rs:412:33
  10: hyper::proto::h2::server::Serving<T,B>::poll_server
             at /home/jess/.cargo/registry/src/github.com-1ecc6299db9ec823/hyper-0.14.15/src/proto/h2/server.rs:264:30
  11: <hyper::proto::h2::server::Server<T,S,B,E> as core::future::future::Future>::poll
             at /home/jess/.cargo/registry/src/github.com-1ecc6299db9ec823/hyper-0.14.15/src/proto/h2/server.rs:198:28
  12: <hyper::server::conn::ProtoServer<T,B,S,E> as core::future::future::Future>::poll
             at /home/jess/.cargo/registry/src/github.com-1ecc6299db9ec823/hyper-0.14.15/src/server/conn.rs:1100:43
  13: <hyper::server::conn::upgrades::UpgradeableConnection<I,S,E> as core::future::future::Future>::poll
             at /home/jess/.cargo/registry/src/github.com-1ecc6299db9ec823/hyper-0.14.15/src/server/conn.rs:1302:30
  14: <hyper::common::drain::Watching<F,FN> as core::future::future::Future>::poll
             at /home/jess/.cargo/registry/src/github.com-1ecc6299db9ec823/hyper-0.14.15/src/common/drain.rs:95:36
  15: <hyper::server::conn::spawn_all::NewSvcTask<I,N,S,E,W> as core::future::future::Future>::poll
             at /home/jess/.cargo/registry/src/github.com-1ecc6299db9ec823/hyper-0.14.15/src/server/conn.rs:1239:36
  16: tokio::runtime::task::core::CoreStage<T>::poll::{{closure}}
             at /home/jess/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.14.0/src/runtime/task/core.rs:161:17
  17: tokio::loom::std::unsafe_cell::UnsafeCell<T>::with_mut
             at /home/jess/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.14.0/src/loom/std/unsafe_cell.rs:14:9
  18: tokio::runtime::task::core::CoreStage<T>::poll
             at /home/jess/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.14.0/src/runtime/task/core.rs:151:13
  19: tokio::runtime::task::harness::poll_future::{{closure}}
             at /home/jess/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.14.0/src/runtime/task/harness.rs:461:19
  20: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
             at /rustc/532d2b14c05f9bc20b2d27cbb5f4550d28343a36/library/core/src/panic/unwind_safe.rs:271:9
  21: std::panicking::try::do_call
             at /rustc/532d2b14c05f9bc20b2d27cbb5f4550d28343a36/library/std/src/panicking.rs:406:40
  22: __rust_try
  23: std::panicking::try
             at /rustc/532d2b14c05f9bc20b2d27cbb5f4550d28343a36/library/std/src/panicking.rs:370:19
  24: std::panic::catch_unwind
             at /rustc/532d2b14c05f9bc20b2d27cbb5f4550d28343a36/library/std/src/panic.rs:133:14
  25: tokio::runtime::task::harness::poll_future
             at /home/jess/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.14.0/src/runtime/task/harness.rs:449:18
  26: tokio::runtime::task::harness::Harness<T,S>::poll_inner
             at /home/jess/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.14.0/src/runtime/task/harness.rs:98:27
  27: tokio::runtime::task::harness::Harness<T,S>::poll
             at /home/jess/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.14.0/src/runtime/task/harness.rs:53:15
  28: tokio::runtime::task::raw::poll
             at /home/jess/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.14.0/src/runtime/task/raw.rs:113:5
  29: tokio::runtime::task::raw::RawTask::poll
             at /home/jess/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.14.0/src/runtime/task/raw.rs:70:18
  30: tokio::runtime::task::LocalNotified<S>::run
             at /home/jess/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.14.0/src/runtime/task/mod.rs:343:9
  31: tokio::runtime::thread_pool::worker::Context::run_task::{{closure}}
             at /home/jess/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.14.0/src/runtime/thread_pool/worker.rs:420:13
  32: tokio::coop::with_budget::{{closure}}
             at /home/jess/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.14.0/src/coop.rs:106:9
  33: std::thread::local::LocalKey<T>::try_with
             at /rustc/532d2b14c05f9bc20b2d27cbb5f4550d28343a36/library/std/src/thread/local.rs:413:16
  34: std::thread::local::LocalKey<T>::with
             at /rustc/532d2b14c05f9bc20b2d27cbb5f4550d28343a36/library/std/src/thread/local.rs:389:9
  35: tokio::coop::with_budget
             at /home/jess/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.14.0/src/coop.rs:99:5
  36: tokio::coop::budget
             at /home/jess/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.14.0/src/coop.rs:76:5
  37: tokio::runtime::thread_pool::worker::Context::run_task
             at /home/jess/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.14.0/src/runtime/thread_pool/worker.rs:419:9
  38: tokio::runtime::thread_pool::worker::Context::run
             at /home/jess/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.14.0/src/runtime/thread_pool/worker.rs:386:24
  39: tokio::runtime::thread_pool::worker::run::{{closure}}
             at /home/jess/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.14.0/src/runtime/thread_pool/worker.rs:371:17
  40: tokio::macros::scoped_tls::ScopedKey<T>::set
             at /home/jess/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.14.0/src/macros/scoped_tls.rs:61:9
  41: tokio::runtime::thread_pool::worker::run
             at /home/jess/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.14.0/src/runtime/thread_pool/worker.rs:368:5
  42: tokio::runtime::thread_pool::worker::Launch::launch::{{closure}}
             at /home/jess/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.14.0/src/runtime/thread_pool/worker.rs:347:45
  43: <tokio::runtime::blocking::task::BlockingTask<T> as core::future::future::Future>::poll
             at /home/jess/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.14.0/src/runtime/blocking/task.rs:42:21
  44: tokio::runtime::task::core::CoreStage<T>::poll::{{closure}}
             at /home/jess/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.14.0/src/runtime/task/core.rs:161:17
  45: tokio::loom::std::unsafe_cell::UnsafeCell<T>::with_mut
             at /home/jess/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.14.0/src/loom/std/unsafe_cell.rs:14:9
  46: tokio::runtime::task::core::CoreStage<T>::poll
             at /home/jess/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.14.0/src/runtime/task/core.rs:151:13
  47: tokio::runtime::task::harness::poll_future::{{closure}}
             at /home/jess/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.14.0/src/runtime/task/harness.rs:461:19
  48: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
             at /rustc/532d2b14c05f9bc20b2d27cbb5f4550d28343a36/library/core/src/panic/unwind_safe.rs:271:9
  49: std::panicking::try::do_call
             at /rustc/532d2b14c05f9bc20b2d27cbb5f4550d28343a36/library/std/src/panicking.rs:406:40
  50: __rust_try
  51: std::panicking::try
             at /rustc/532d2b14c05f9bc20b2d27cbb5f4550d28343a36/library/std/src/panicking.rs:370:19
  52: std::panic::catch_unwind
             at /rustc/532d2b14c05f9bc20b2d27cbb5f4550d28343a36/library/std/src/panic.rs:133:14
  53: tokio::runtime::task::harness::poll_future
             at /home/jess/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.14.0/src/runtime/task/harness.rs:449:18
  54: tokio::runtime::task::harness::Harness<T,S>::poll_inner
             at /home/jess/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.14.0/src/runtime/task/harness.rs:98:27
  55: tokio::runtime::task::harness::Harness<T,S>::poll
             at /home/jess/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.14.0/src/runtime/task/harness.rs:53:15
  56: tokio::runtime::task::raw::poll
             at /home/jess/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.14.0/src/runtime/task/raw.rs:113:5
  57: tokio::runtime::task::raw::RawTask::poll
             at /home/jess/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.14.0/src/runtime/task/raw.rs:70:18
  58: tokio::runtime::task::UnownedTask<S>::run
             at /home/jess/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.14.0/src/runtime/task/mod.rs:379:9
  59: tokio::runtime::blocking::pool::Inner::run
             at /home/jess/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.14.0/src/runtime/blocking/pool.rs:264:17
  60: tokio::runtime::blocking::pool::Spawner::spawn_thread::{{closure}}
             at /home/jess/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.14.0/src/runtime/blocking/pool.rs:244:17

is a full stack trace

seanmonstar commented 2 years ago

While slightly different input, it seems this is also fixed with https://github.com/hyperium/h2/commit/dbaa3a42856d23c6bdebb6125c2205d965532178 (release planned for today). When I patch in h2 from git, it no longer panics.

5225225 commented 2 years ago

Hm, I can still reproduce it, even when patching h2 from git.

My cargo.toml for the target server looks like

[package]
name = "scratch0dtkkKHQJ"
version = "0.1.0"
edition = "2021"

[patch.crates-io]
h2 = { git = "https://github.com/hyperium/h2/" }

[dependencies]
rocket = "0.5.0-rc.1"

crash.zip is the full file in a zip. I'm not doing anything special to send it over, just netcat localhost 8000 < crash. Github says dbaa3a4 is merged to master, so h2 = { git = ... } should pick up that fix if it exists, right? That or I'm getting the format for patch wrong.

seanmonstar commented 2 years ago

There's now a new version of cargo, so removing the "patch" and just trying cargo update -p h2 and retrying should work.

I tried out your your steps and the newer h2 worked for me, at least.

5225225 commented 2 years ago

Huh.

I ran a cargo update and removed the patch and the test crate is now using 0.3.8 of h2, but I can still reproduce it.

thread 'rocket-worker-thread' panicked at 'assertion failed: !id.is_zero()', /home/jess/.cargo/registry/src/github.com-1ecc6299db9ec823/h2-0.3.8/src/proto/peer.rs:54:9

Maybe it's the way rocket's running it?

It's got h2-0.3.8 in the path so I'm fairly sure it's running the latest code.

Can you post how you're running it?

I am running on a nightly, but even running on stable works fine.

seanmonstar commented 2 years ago

Here's what I did:

I took the hyper repo, I ran in one terminal:

RUST_LOG=hyper,h2 cargo run --features full --example hello

And then in a separate terminal, I ran:

echo "DVBSSSAqIEhUVFAvMi4wDQoNClNNDQoNCgAAKgEAAAAAAKa8jry8vLy8vLy8vLy8vLy8vLy8vLy8vLysvLy8pqampqampqaupqampqampqamJqampqamplpZWVl5WVlXpqampqaAgICAgICAgICAgICAgICA" | base64 -d | nc localhost 3000

This produced the panic message you reported, and then after running cargo update to have the newest h2, it no longer panicked. The logs now show that an error was detected, and the connection state transitions to closing, and then closes.

5225225 commented 2 years ago

Yeah, I can reproduce that command working fine now, but

echo "DVBSSSAqIEhUVFAvMi4wDQoNClNNDQoNCgAAKgUAAAAAAAAAAYODg4ODg4ODZoODg4O+vr6+vr6+vr6+vr6+vr6+vr6+vr6+vr6+vr6+vr6+vr6+vr6Dg4Ourq6uCwAAAL6+vr6+" | base64 -d | nc localhost 3000

still fails on the same assert as the original.

Should I close this issue and re-open one for that?