tokio-rs / axum

Ergonomic and modular web framework built with Tokio, Tower, and Hyper
19.11k stars 1.06k forks source link

Can state be cloned less when handling requests? #1827

Open mcatanzariti opened 1 year ago

mcatanzariti commented 1 year ago

Hi,

Running this little test with axum 0.6.10 :

use axum::{extract::State, routing::get, Router};
use std::{net::SocketAddr};

struct MyState;

impl Clone for MyState {
    fn clone(&self) -> Self {
        println!("MyState::Clone");
        Self
    }
}

#[tokio::main]
async fn main() {
    let state = MyState;

    // build our application with a route
    let app = Router::new().route("/", get(read)).with_state(state);

    // run it
    let addr = SocketAddr::from(([127, 0, 0, 1], 3000));
    println!("listening on {}", addr);
    axum::Server::bind(&addr)
        .serve(app.into_make_service())
        .await
        .unwrap();
}

async fn read(State(_state): State<MyState>) -> &'static str {
    println!("read route called!");
    "OK"
}

I can see in the output that MyState is cloned 5 times at each call.

MyState::Clone
MyState::Clone
listening on 127.0.0.1:3000
MyState::Clone
MyState::Clone
MyState::Clone
MyState::Clone
MyState::Clone
read route called!
MyState::Clone
MyState::Clone
MyState::Clone
MyState::Clone
MyState::Clone
read route called!

Would it be possible to clone it juste once per route call?

Thank you,

Michaël

jplatte commented 1 year ago

Hm, I wonder if this is because Router is cloned more often than necessary. Could you update your Clone impl to capture and print a Backtrace?

mcatanzariti commented 1 year ago

sure !

code updated:

use axum::{extract::State, routing::get, Router};
use std::{backtrace::Backtrace, net::SocketAddr};

struct MyState;

impl Clone for MyState {
    fn clone(&self) -> Self {
        println!("MyState::Clone:\n===============\n{}", Backtrace::force_capture());
        Self
    }
}

#[tokio::main]
async fn main() {
    let state = MyState;

    // build our application with a route
    let app = Router::new().route("/", get(read)).with_state(state);

    // run it
    let addr = SocketAddr::from(([127, 0, 0, 1], 3000));
    println!("listening on {}", addr);
    axum::Server::bind(&addr)
        .serve(app.into_make_service())
        .await
        .unwrap();
}

async fn read(State(_state): State<MyState>) -> &'static str {
    println!("read route called!");
    "OK"
}
MyState::Clone:
===============
   0: std::backtrace_rs::backtrace::dbghelp::trace
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library\std\src\..\..\backtrace\src\backtrace\dbghelp.rs:98
   1: std::backtrace_rs::backtrace::trace_unsynchronized
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library\std\src\..\..\backtrace\src\backtrace\mod.rs:66
   2: std::backtrace::Backtrace::create
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library\std\src\backtrace.rs:332
   3: std::backtrace::Backtrace::force_capture
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library\std\src\backtrace.rs:314
   4: axum_test::impl$0::clone
             at .\src\main.rs:8
   5: axum::routing::impl$3::with_state::closure$0<axum_test::MyState,hyper::body::body::Body,tuple$<> >
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\axum-0.6.10\src\routing\mod.rs:399
   6: core::iter::adapters::map::map_fold::closure$0<tuple$<axum::routing::RouteId,enum2$<axum::routing::Endpoint<axum_test::MyState,hyper::body::body::Body> > >,tuple$<axum::routing::RouteId,enum2$<axum::routing::Endpoint<tuple$<>,hyper::body::body::Body> > >,
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\core\src\iter\adapters\map.rs:84
   7: core::iter::traits::iterator::Iterator::fold<std::collections::hash::map::IntoIter<axum::routing::RouteId,enum2$<axum::routing::Endpoint<axum_test::MyState,hyper::body::body::Body> > >,tuple$<>,core::iter::adapters::map::map_fold::closure_env$0<tuple$<axu
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\core\src\iter\traits\iterator.rs:2414
   8: core::iter::adapters::map::impl$2::fold<tuple$<axum::routing::RouteId,enum2$<axum::routing::Endpoint<tuple$<>,hyper::body::body::Body> > >,std::collections::hash::map::IntoIter<axum::routing::RouteId,enum2$<axum::routing::Endpoint<axum_test::MyState,hyper
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\core\src\iter\adapters\map.rs:124
   9: core::iter::traits::iterator::Iterator::for_each<core::iter::adapters::map::Map<std::collections::hash::map::IntoIter<axum::routing::RouteId,enum2$<axum::routing::Endpoint<axum_test::MyState,hyper::body::body::Body> > >,axum::routing::impl$3::with_state::
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\core\src\iter\traits\iterator.rs:831
  10: hashbrown::map::impl$97::extend<axum::routing::RouteId,enum2$<axum::routing::Endpoint<tuple$<>,hyper::body::body::Body> >,std::collections::hash::map::RandomState,alloc::alloc::Global,core::iter::adapters::map::Map<std::collections::hash::map::IntoIter<ax
             at C:\Users\runneradmin\.cargo\registry\src\github.com-1ecc6299db9ec823\hashbrown-0.12.3\src\map.rs:6407
  11: std::collections::hash::map::impl$80::extend
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\std\src\collections\hash\map.rs:3037
  12: std::collections::hash::map::impl$79::from_iter<axum::routing::RouteId,enum2$<axum::routing::Endpoint<tuple$<>,hyper::body::body::Body> >,std::collections::hash::map::RandomState,core::iter::adapters::map::Map<std::collections::hash::map::IntoIter<axum::r
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\std\src\collections\hash\map.rs:3022
  13: core::iter::traits::iterator::Iterator::collect<core::iter::adapters::map::Map<std::collections::hash::map::IntoIter<axum::routing::RouteId,enum2$<axum::routing::Endpoint<axum_test::MyState,hyper::body::body::Body> > >,axum::routing::impl$3::with_state::c
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\core\src\iter\traits\iterator.rs:1836
  14: axum::routing::Router<axum_test::MyState,hyper::body::body::Body>::with_state<axum_test::MyState,hyper::body::body::Body,tuple$<> >
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\axum-0.6.10\src\routing\mod.rs:393
  15: axum_test::main::async_block$0
             at .\src\main.rs:18
  16: tokio::runtime::park::impl$4::block_on::closure$0<enum2$<axum_test::main::async_block_env$0> >
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\park.rs:283
  17: tokio::runtime::coop::with_budget
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\coop.rs:107
  18: tokio::runtime::coop::budget
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\coop.rs:73
  19: tokio::runtime::park::CachedParkThread::block_on<enum2$<axum_test::main::async_block_env$0> >
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\park.rs:283
  20: tokio::runtime::context::BlockingRegionGuard::block_on<enum2$<axum_test::main::async_block_env$0> >
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\context.rs:315
  21: tokio::runtime::scheduler::multi_thread::MultiThread::block_on<enum2$<axum_test::main::async_block_env$0> >
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\scheduler\multi_thread\mod.rs:66
  22: tokio::runtime::runtime::Runtime::block_on<enum2$<axum_test::main::async_block_env$0> >
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\runtime.rs:304
  23: axum_test::main
             at .\src\main.rs:23
  24: core::ops::function::FnOnce::call_once<void (*)(),tuple$<> >
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\core\src\ops\function.rs:507
  25: std::sys_common::backtrace::__rust_begin_short_backtrace<void (*)(),tuple$<> >
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\std\src\sys_common\backtrace.rs:121
  26: std::sys_common::backtrace::__rust_begin_short_backtrace<void (*)(),tuple$<> >
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\std\src\sys_common\backtrace.rs:121
  27: std::rt::lang_start::closure$0<tuple$<> >
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\std\src\rt.rs:166
  28: core::ops::function::impls::impl$2::call_once
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library\core\src\ops\function.rs:606
  29: std::panicking::try::do_call
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library\std\src\panicking.rs:483
  30: std::panicking::try
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library\std\src\panicking.rs:447
  31: std::panic::catch_unwind
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library\std\src\panic.rs:137
  32: std::rt::lang_start_internal::closure$2
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library\std\src\rt.rs:148
  33: std::panicking::try::do_call
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library\std\src\panicking.rs:483
  34: std::panicking::try
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library\std\src\panicking.rs:447
  35: std::panic::catch_unwind
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library\std\src\panic.rs:137
  36: std::rt::lang_start_internal
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library\std\src\rt.rs:148
  37: std::rt::lang_start<tuple$<> >
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\std\src\rt.rs:165
  38: main
  39: invoke_main
             at D:\a\_work\1\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:78
  40: __scrt_common_main_seh
             at D:\a\_work\1\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:288
  41: BaseThreadInitThunk
  42: RtlUserThreadStart

MyState::Clone:
===============
   0: std::backtrace_rs::backtrace::dbghelp::trace
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library\std\src\..\..\backtrace\src\backtrace\dbghelp.rs:98
   1: std::backtrace_rs::backtrace::trace_unsynchronized
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library\std\src\..\..\backtrace\src\backtrace\mod.rs:66
   2: std::backtrace::Backtrace::create
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library\std\src\backtrace.rs:332
   3: std::backtrace::Backtrace::force_capture
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library\std\src\backtrace.rs:314
   4: axum_test::impl$0::clone
             at .\src\main.rs:8
   5: enum2$<axum::routing::method_routing::MethodEndpoint<axum_test::MyState,hyper::body::body::Body,enum2$<core::convert::Infallible> > >::with_state<axum_test::MyState,hyper::body::body::Body,enum2$<core::convert::Infallible>,tuple$<> >
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\axum-0.6.10\src\routing\method_routing.rs:1226
   6: axum::routing::method_routing::MethodRouter<axum_test::MyState,hyper::body::body::Body,enum2$<core::convert::Infallible> >::with_state<axum_test::MyState,hyper::body::body::Body,enum2$<core::convert::Infallible>,tuple$<> >
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\axum-0.6.10\src\routing\method_routing.rs:738
   7: axum::routing::impl$3::with_state::closure$0<axum_test::MyState,hyper::body::body::Body,tuple$<> >
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\axum-0.6.10\src\routing\mod.rs:399
   8: core::iter::adapters::map::map_fold::closure$0<tuple$<axum::routing::RouteId,enum2$<axum::routing::Endpoint<axum_test::MyState,hyper::body::body::Body> > >,tuple$<axum::routing::RouteId,enum2$<axum::routing::Endpoint<tuple$<>,hyper::body::body::Body> > >,
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\core\src\iter\adapters\map.rs:84
   9: core::iter::traits::iterator::Iterator::fold<std::collections::hash::map::IntoIter<axum::routing::RouteId,enum2$<axum::routing::Endpoint<axum_test::MyState,hyper::body::body::Body> > >,tuple$<>,core::iter::adapters::map::map_fold::closure_env$0<tuple$<axu
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\core\src\iter\traits\iterator.rs:2414
  10: core::iter::adapters::map::impl$2::fold<tuple$<axum::routing::RouteId,enum2$<axum::routing::Endpoint<tuple$<>,hyper::body::body::Body> > >,std::collections::hash::map::IntoIter<axum::routing::RouteId,enum2$<axum::routing::Endpoint<axum_test::MyState,hyper
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\core\src\iter\adapters\map.rs:124
  11: core::iter::traits::iterator::Iterator::for_each<core::iter::adapters::map::Map<std::collections::hash::map::IntoIter<axum::routing::RouteId,enum2$<axum::routing::Endpoint<axum_test::MyState,hyper::body::body::Body> > >,axum::routing::impl$3::with_state::
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\core\src\iter\traits\iterator.rs:831
  12: hashbrown::map::impl$97::extend<axum::routing::RouteId,enum2$<axum::routing::Endpoint<tuple$<>,hyper::body::body::Body> >,std::collections::hash::map::RandomState,alloc::alloc::Global,core::iter::adapters::map::Map<std::collections::hash::map::IntoIter<ax
             at C:\Users\runneradmin\.cargo\registry\src\github.com-1ecc6299db9ec823\hashbrown-0.12.3\src\map.rs:6407
  13: std::collections::hash::map::impl$80::extend
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\std\src\collections\hash\map.rs:3037
  14: std::collections::hash::map::impl$79::from_iter<axum::routing::RouteId,enum2$<axum::routing::Endpoint<tuple$<>,hyper::body::body::Body> >,std::collections::hash::map::RandomState,core::iter::adapters::map::Map<std::collections::hash::map::IntoIter<axum::r
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\std\src\collections\hash\map.rs:3022
  15: core::iter::traits::iterator::Iterator::collect<core::iter::adapters::map::Map<std::collections::hash::map::IntoIter<axum::routing::RouteId,enum2$<axum::routing::Endpoint<axum_test::MyState,hyper::body::body::Body> > >,axum::routing::impl$3::with_state::c
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\core\src\iter\traits\iterator.rs:1836
  16: axum::routing::Router<axum_test::MyState,hyper::body::body::Body>::with_state<axum_test::MyState,hyper::body::body::Body,tuple$<> >
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\axum-0.6.10\src\routing\mod.rs:393
  17: axum_test::main::async_block$0
             at .\src\main.rs:18
  18: tokio::runtime::park::impl$4::block_on::closure$0<enum2$<axum_test::main::async_block_env$0> >
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\park.rs:283
  19: tokio::runtime::coop::with_budget
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\coop.rs:107
  20: tokio::runtime::coop::budget
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\coop.rs:73
  21: tokio::runtime::park::CachedParkThread::block_on<enum2$<axum_test::main::async_block_env$0> >
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\park.rs:283
  22: tokio::runtime::context::BlockingRegionGuard::block_on<enum2$<axum_test::main::async_block_env$0> >
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\context.rs:315
  23: tokio::runtime::scheduler::multi_thread::MultiThread::block_on<enum2$<axum_test::main::async_block_env$0> >
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\scheduler\multi_thread\mod.rs:66
  24: tokio::runtime::runtime::Runtime::block_on<enum2$<axum_test::main::async_block_env$0> >
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\runtime.rs:304
  25: axum_test::main
             at .\src\main.rs:23
  26: core::ops::function::FnOnce::call_once<void (*)(),tuple$<> >
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\core\src\ops\function.rs:507
  27: std::sys_common::backtrace::__rust_begin_short_backtrace<void (*)(),tuple$<> >
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\std\src\sys_common\backtrace.rs:121
  28: std::sys_common::backtrace::__rust_begin_short_backtrace<void (*)(),tuple$<> >
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\std\src\sys_common\backtrace.rs:121
  29: std::rt::lang_start::closure$0<tuple$<> >
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\std\src\rt.rs:166
  30: core::ops::function::impls::impl$2::call_once
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library\core\src\ops\function.rs:606
  31: std::panicking::try::do_call
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library\std\src\panicking.rs:483
  32: std::panicking::try
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library\std\src\panicking.rs:447
  33: std::panic::catch_unwind
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library\std\src\panic.rs:137
  34: std::rt::lang_start_internal::closure$2
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library\std\src\rt.rs:148
  35: std::panicking::try::do_call
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library\std\src\panicking.rs:483
  36: std::panicking::try
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library\std\src\panicking.rs:447
  37: std::panic::catch_unwind
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library\std\src\panic.rs:137
  38: std::rt::lang_start_internal
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library\std\src\rt.rs:148
  39: std::rt::lang_start<tuple$<> >
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\std\src\rt.rs:165
  40: main
  41: invoke_main
             at D:\a\_work\1\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:78
  42: __scrt_common_main_seh
             at D:\a\_work\1\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:288
  43: BaseThreadInitThunk
  44: RtlUserThreadStart

listening on 127.0.0.1:3000
MyState::Clone:
===============
   0: std::backtrace_rs::backtrace::dbghelp::trace
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library\std\src\..\..\backtrace\src\backtrace\dbghelp.rs:98
   1: std::backtrace_rs::backtrace::trace_unsynchronized
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library\std\src\..\..\backtrace\src\backtrace\mod.rs:66
   2: std::backtrace::Backtrace::create
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library\std\src\backtrace.rs:332
   3: std::backtrace::Backtrace::force_capture
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library\std\src\backtrace.rs:314
   4: axum_test::impl$0::clone
             at .\src\main.rs:8
   5: axum::handler::service::impl$3::clone<enum2$<axum_test::read::async_fn_env$0> (*)(axum::extract::state::State<axum_test::MyState>),tuple$<enum2$<axum_core::extract::private::ViaParts>,axum::extract::state::State<axum_test::MyState> >,axum_test::MyState,hy
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\axum-0.6.10\src\handler\service.rs:143
   6: tower::util::map_response::impl$5::clone<axum::handler::service::HandlerService<enum2$<axum_test::read::async_fn_env$0> (*)(axum::extract::state::State<axum_test::MyState>),tuple$<enum2$<axum_core::extract::private::ViaParts>,axum::extract::state::State<a
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tower-0.4.13\src\util\map_response.rs:12
   7: tower::util::map_future::impl$6::clone<tower::util::map_response::MapResponse<axum::handler::service::HandlerService<enum2$<axum_test::read::async_fn_env$0> (*)(axum::extract::state::State<axum_test::MyState>),tuple$<enum2$<axum_core::extract::private::Vi
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tower-0.4.13\src\util\map_future.rs:14
   8: tower::util::boxed_clone::impl$3::clone_box<http::request::Request<hyper::body::body::Body>,tower::util::map_future::MapFuture<tower::util::map_response::MapResponse<axum::handler::service::HandlerService<enum2$<axum_test::read::async_fn_env$0> (*)(axum::
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tower-0.4.13\src\util\boxed_clone.rs:128
   9: tower::util::boxed_clone::impl$2::clone<http::request::Request<hyper::body::body::Body>,http::response::Response<http_body::combinators::box_body::UnsyncBoxBody<bytes::bytes::Bytes,axum_core::error::Error> >,enum2$<core::convert::Infallible> >
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tower-0.4.13\src\util\boxed_clone.rs:107
  10: axum::routing::route::impl$1::clone<hyper::body::body::Body,enum2$<core::convert::Infallible> >
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\axum-0.6.10\src\routing\route.rs:73
  11: axum::routing::method_routing::impl$8::clone<tuple$<>,hyper::body::body::Body,enum2$<core::convert::Infallible> >
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\axum-0.6.10\src\routing\method_routing.rs:1236
  12: axum::routing::method_routing::impl$5::clone<tuple$<>,hyper::body::body::Body,enum2$<core::convert::Infallible> >
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\axum-0.6.10\src\routing\method_routing.rs:1163
  13: axum::routing::impl$12::clone<tuple$<>,hyper::body::body::Body>
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\axum-0.6.10\src\routing\mod.rs:717
  14: core::clone::Clone::clone<tuple$<axum::routing::RouteId,enum2$<axum::routing::Endpoint<tuple$<>,hyper::body::body::Body> > > >
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\core\src\clone.rs:123
  15: hashbrown::raw::RawTable<tuple$<axum::routing::RouteId,enum2$<axum::routing::Endpoint<tuple$<>,hyper::body::body::Body> > >,alloc::alloc::Global>::clone_from_impl<tuple$<axum::routing::RouteId,enum2$<axum::routing::Endpoint<tuple$<>,hyper::body::body::Bod
             at C:\Users\runneradmin\.cargo\registry\src\github.com-1ecc6299db9ec823\hashbrown-0.12.3\src\raw\mod.rs:1735
  16: hashbrown::raw::impl$13::clone_from_spec<tuple$<axum::routing::RouteId,enum2$<axum::routing::Endpoint<tuple$<>,hyper::body::body::Body> > >,alloc::alloc::Global>
             at C:\Users\runneradmin\.cargo\registry\src\github.com-1ecc6299db9ec823\hashbrown-0.12.3\src\raw\mod.rs:1685
  17: hashbrown::raw::impl$12::clone<tuple$<axum::routing::RouteId,enum2$<axum::routing::Endpoint<tuple$<>,hyper::body::body::Body> > >,alloc::alloc::Global>
             at C:\Users\runneradmin\.cargo\registry\src\github.com-1ecc6299db9ec823\hashbrown-0.12.3\src\raw\mod.rs:1622
  18: hashbrown::map::impl$0::clone<axum::routing::RouteId,enum2$<axum::routing::Endpoint<tuple$<>,hyper::body::body::Body> >,std::collections::hash::map::RandomState,alloc::alloc::Global>
             at C:\Users\runneradmin\.cargo\registry\src\github.com-1ecc6299db9ec823\hashbrown-0.12.3\src\map.rs:197
  19: std::collections::hash::map::impl$4::clone<axum::routing::RouteId,enum2$<axum::routing::Endpoint<tuple$<>,hyper::body::body::Body> >,std::collections::hash::map::RandomState>
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\std\src\collections\hash\map.rs:1267
  20: axum::routing::impl$0::clone<tuple$<>,hyper::body::body::Body>
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\axum-0.6.10\src\routing\mod.rs:62
  21: axum::routing::into_make_service::impl$1::call<axum::routing::Router<tuple$<>,hyper::body::body::Body>,ref$<hyper::server::tcp::addr_stream::AddrStream> >
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\axum-0.6.10\src\routing\into_make_service.rs:36
  22: hyper::service::make::impl$2::make_service_ref<axum::routing::into_make_service::IntoMakeService<axum::routing::Router<tuple$<>,hyper::body::body::Body> >,hyper::server::tcp::addr_stream::AddrStream,enum2$<core::convert::Infallible>,enum2$<core::convert::
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\hyper-0.14.24\src\service\make.rs:89
  23: hyper::server::server::Server<hyper::server::tcp::AddrIncoming,axum::routing::into_make_service::IntoMakeService<axum::routing::Router<tuple$<>,hyper::body::body::Body> >,enum2$<hyper::common::exec::Exec> >::poll_next_<hyper::server::tcp::AddrIncoming,hyp
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\hyper-0.14.24\src\server\server.rs:178
  24: hyper::server::server::Server<hyper::server::tcp::AddrIncoming,axum::routing::into_make_service::IntoMakeService<axum::routing::Router<tuple$<>,hyper::body::body::Body> >,enum2$<hyper::common::exec::Exec> >::poll_watch<hyper::server::tcp::AddrIncoming,hyp
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\hyper-0.14.24\src\server\server.rs:199
  25: hyper::server::server::impl$4::poll<hyper::server::tcp::AddrIncoming,hyper::server::tcp::addr_stream::AddrStream,std::io::error::Error,axum::routing::into_make_service::IntoMakeService<axum::routing::Router<tuple$<>,hyper::body::body::Body> >,http_body::c
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\hyper-0.14.24\src\server\server.rs:225
  26: axum_test::main::async_block$0
             at .\src\main.rs:25
  27: tokio::runtime::park::impl$4::block_on::closure$0<enum2$<axum_test::main::async_block_env$0> >
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\park.rs:283
  28: tokio::runtime::coop::with_budget
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\coop.rs:107
  29: tokio::runtime::coop::budget
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\coop.rs:73
  30: tokio::runtime::park::CachedParkThread::block_on<enum2$<axum_test::main::async_block_env$0> >
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\park.rs:283
  31: tokio::runtime::context::BlockingRegionGuard::block_on<enum2$<axum_test::main::async_block_env$0> >
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\context.rs:315
  32: tokio::runtime::scheduler::multi_thread::MultiThread::block_on<enum2$<axum_test::main::async_block_env$0> >
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\scheduler\multi_thread\mod.rs:66
  33: tokio::runtime::runtime::Runtime::block_on<enum2$<axum_test::main::async_block_env$0> >
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\runtime.rs:304
  34: axum_test::main
             at .\src\main.rs:23
  35: core::ops::function::FnOnce::call_once<void (*)(),tuple$<> >
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\core\src\ops\function.rs:507
  36: std::sys_common::backtrace::__rust_begin_short_backtrace<void (*)(),tuple$<> >
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\std\src\sys_common\backtrace.rs:121
  37: std::sys_common::backtrace::__rust_begin_short_backtrace<void (*)(),tuple$<> >
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\std\src\sys_common\backtrace.rs:121
  38: std::rt::lang_start::closure$0<tuple$<> >
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\std\src\rt.rs:166
  39: core::ops::function::impls::impl$2::call_once
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library\core\src\ops\function.rs:606
  40: std::panicking::try::do_call
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library\std\src\panicking.rs:483
  41: std::panicking::try
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library\std\src\panicking.rs:447
  42: std::panic::catch_unwind
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library\std\src\panic.rs:137
  43: std::rt::lang_start_internal::closure$2
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library\std\src\rt.rs:148
  44: std::panicking::try::do_call
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library\std\src\panicking.rs:483
  45: std::panicking::try
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library\std\src\panicking.rs:447
  46: std::panic::catch_unwind
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library\std\src\panic.rs:137
  47: std::rt::lang_start_internal
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library\std\src\rt.rs:148
  48: std::rt::lang_start<tuple$<> >
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\std\src\rt.rs:165
  49: main
  50: invoke_main
             at D:\a\_work\1\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:78
  51: __scrt_common_main_seh
             at D:\a\_work\1\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:288
  52: BaseThreadInitThunk
  53: RtlUserThreadStart

MyState::Clone:
===============
   0: std::backtrace_rs::backtrace::dbghelp::trace
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library\std\src\..\..\backtrace\src\backtrace\dbghelp.rs:98
   1: std::backtrace_rs::backtrace::trace_unsynchronized
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library\std\src\..\..\backtrace\src\backtrace\mod.rs:66
   2: std::backtrace::Backtrace::create
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library\std\src\backtrace.rs:332
   3: std::backtrace::Backtrace::force_capture
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library\std\src\backtrace.rs:314
   4: axum_test::impl$0::clone
             at .\src\main.rs:8
   5: axum::handler::service::impl$3::clone<enum2$<axum_test::read::async_fn_env$0> (*)(axum::extract::state::State<axum_test::MyState>),tuple$<enum2$<axum_core::extract::private::ViaParts>,axum::extract::state::State<axum_test::MyState> >,axum_test::MyState,hy
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\axum-0.6.10\src\handler\service.rs:143
   6: tower::util::map_response::impl$5::clone<axum::handler::service::HandlerService<enum2$<axum_test::read::async_fn_env$0> (*)(axum::extract::state::State<axum_test::MyState>),tuple$<enum2$<axum_core::extract::private::ViaParts>,axum::extract::state::State<a
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tower-0.4.13\src\util\map_response.rs:12
   7: tower::util::map_future::impl$6::clone<tower::util::map_response::MapResponse<axum::handler::service::HandlerService<enum2$<axum_test::read::async_fn_env$0> (*)(axum::extract::state::State<axum_test::MyState>),tuple$<enum2$<axum_core::extract::private::Vi
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tower-0.4.13\src\util\map_future.rs:14
   8: tower::util::boxed_clone::impl$3::clone_box<http::request::Request<hyper::body::body::Body>,tower::util::map_future::MapFuture<tower::util::map_response::MapResponse<axum::handler::service::HandlerService<enum2$<axum_test::read::async_fn_env$0> (*)(axum::
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tower-0.4.13\src\util\boxed_clone.rs:128
   9: tower::util::boxed_clone::impl$2::clone<http::request::Request<hyper::body::body::Body>,http::response::Response<http_body::combinators::box_body::UnsyncBoxBody<bytes::bytes::Bytes,axum_core::error::Error> >,enum2$<core::convert::Infallible> >
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tower-0.4.13\src\util\boxed_clone.rs:107
  10: axum::routing::route::impl$1::clone<hyper::body::body::Body,enum2$<core::convert::Infallible> >
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\axum-0.6.10\src\routing\route.rs:73
  11: axum::routing::method_routing::impl$8::clone<tuple$<>,hyper::body::body::Body,enum2$<core::convert::Infallible> >
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\axum-0.6.10\src\routing\method_routing.rs:1236
  12: axum::routing::method_routing::impl$5::clone<tuple$<>,hyper::body::body::Body,enum2$<core::convert::Infallible> >
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\axum-0.6.10\src\routing\method_routing.rs:1163
  13: axum::routing::impl$12::clone<tuple$<>,hyper::body::body::Body>
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\axum-0.6.10\src\routing\mod.rs:717
  14: axum::routing::Router<tuple$<>,hyper::body::body::Body>::call_route<tuple$<>,hyper::body::body::Body>
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\axum-0.6.10\src\routing\mod.rs:492
  15: axum::routing::Router<tuple$<>,hyper::body::body::Body>::call_with_state<tuple$<>,hyper::body::body::Body>
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\axum-0.6.10\src\routing\mod.rs:452
  16: axum::routing::impl$5::call<hyper::body::body::Body>
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\axum-0.6.10\src\routing\mod.rs:573
  17: hyper::service::http::impl$0::call<axum::routing::Router<tuple$<>,hyper::body::body::Body>,hyper::body::body::Body,http_body::combinators::box_body::UnsyncBoxBody<bytes::bytes::Bytes,axum_core::error::Error> >    
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\hyper-0.14.24\src\service\http.rs:45
  18: hyper::proto::h1::dispatch::impl$6::recv_msg<axum::routing::Router<tuple$<>,hyper::body::body::Body>,http_body::combinators::box_body::UnsyncBoxBody<bytes::bytes::Bytes,axum_core::error::Error> >
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\hyper-0.14.24\src\proto\h1\dispatch.rs:517
  19: hyper::proto::h1::dispatch::Dispatcher<hyper::proto::h1::dispatch::Server<axum::routing::Router<tuple$<>,hyper::body::body::Body>,hyper::body::body::Body>,http_body::combinators::box_body::UnsyncBoxBody<bytes::bytes::Bytes,axum_core::error::Error>,hyper::
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\hyper-0.14.24\src\proto\h1\dispatch.rs:264
  20: hyper::proto::h1::dispatch::Dispatcher<hyper::proto::h1::dispatch::Server<axum::routing::Router<tuple$<>,hyper::body::body::Body>,hyper::body::body::Body>,http_body::combinators::box_body::UnsyncBoxBody<bytes::bytes::Bytes,axum_core::error::Error>,hyper::
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\hyper-0.14.24\src\proto\h1\dispatch.rs:188
  21: hyper::proto::h1::dispatch::Dispatcher<hyper::proto::h1::dispatch::Server<axum::routing::Router<tuple$<>,hyper::body::body::Body>,hyper::body::body::Body>,http_body::combinators::box_body::UnsyncBoxBody<bytes::bytes::Bytes,axum_core::error::Error>,hyper::
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\hyper-0.14.24\src\proto\h1\dispatch.rs:160
  22: hyper::proto::h1::dispatch::Dispatcher<hyper::proto::h1::dispatch::Server<axum::routing::Router<tuple$<>,hyper::body::body::Body>,hyper::body::body::Body>,http_body::combinators::box_body::UnsyncBoxBody<bytes::bytes::Bytes,axum_core::error::Error>,hyper::
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\hyper-0.14.24\src\proto\h1\dispatch.rs:137
  23: hyper::proto::h1::dispatch::Dispatcher<hyper::proto::h1::dispatch::Server<axum::routing::Router<tuple$<>,hyper::body::body::Body>,hyper::body::body::Body>,http_body::combinators::box_body::UnsyncBoxBody<bytes::bytes::Bytes,axum_core::error::Error>,hyper::
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\hyper-0.14.24\src\proto\h1\dispatch.rs:120
  24: hyper::proto::h1::dispatch::impl$1::poll<hyper::proto::h1::dispatch::Server<axum::routing::Router<tuple$<>,hyper::body::body::Body>,hyper::body::body::Body>,http_body::combinators::box_body::UnsyncBoxBody<bytes::bytes::Bytes,axum_core::error::Error>,hyper
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\hyper-0.14.24\src\proto\h1\dispatch.rs:424
  25: hyper::server::conn::impl$6::poll<hyper::server::tcp::addr_stream::AddrStream,http_body::combinators::box_body::UnsyncBoxBody<bytes::bytes::Bytes,axum_core::error::Error>,axum::routing::Router<tuple$<>,hyper::body::body::Body>,enum2$<hyper::common::exec::
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\hyper-0.14.24\src\server\conn.rs:952
  26: hyper::server::conn::upgrades::impl$1::poll<hyper::server::tcp::addr_stream::AddrStream,http_body::combinators::box_body::UnsyncBoxBody<bytes::bytes::Bytes,axum_core::error::Error>,axum::routing::Router<tuple$<>,hyper::body::body::Body>,enum2$<hyper::comm
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\hyper-0.14.24\src\server\conn.rs:1012
  27: hyper::server::server::new_svc::impl$1::poll<hyper::server::tcp::addr_stream::AddrStream,axum::routing::into_make_service::IntoMakeServiceFuture<axum::routing::Router<tuple$<>,hyper::body::body::Body> >,axum::routing::Router<tuple$<>,hyper::body::body::Bo
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\hyper-0.14.24\src\server\server.rs:751
  28: tokio::runtime::task::core::impl$6::poll::closure$0<hyper::server::server::new_svc::NewSvcTask<hyper::server::tcp::addr_stream::AddrStream,axum::routing::into_make_service::IntoMakeServiceFuture<axum::routing::Router<tuple$<>,hyper::body::body::Body> >,ax
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\task\core.rs:223
  29: tokio::loom::std::unsafe_cell::UnsafeCell<enum2$<tokio::runtime::task::core::Stage<hyper::server::server::new_svc::NewSvcTask<hyper::server::tcp::addr_stream::AddrStream,axum::routing::into_make_service::IntoMakeServiceFuture<axum::routing::Router<tuple$<
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\loom\std\unsafe_cell.rs:14
  30: tokio::runtime::task::core::Core<hyper::server::server::new_svc::NewSvcTask<hyper::server::tcp::addr_stream::AddrStream,axum::routing::into_make_service::IntoMakeServiceFuture<axum::routing::Router<tuple$<>,hyper::body::body::Body> >,axum::routing::Router
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\task\core.rs:212
  31: tokio::runtime::task::harness::poll_future::closure$0<hyper::server::server::new_svc::NewSvcTask<hyper::server::tcp::addr_stream::AddrStream,axum::routing::into_make_service::IntoMakeServiceFuture<axum::routing::Router<tuple$<>,hyper::body::body::Body> >,
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\task\harness.rs:476
  32: core::panic::unwind_safe::impl$23::call_once<enum2$<core::task::poll::Poll<tuple$<> > >,tokio::runtime::task::harness::poll_future::closure_env$0<hyper::server::server::new_svc::NewSvcTask<hyper::server::tcp::addr_stream::AddrStream,axum::routing::into_ma
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\core\src\panic\unwind_safe.rs:271
  33: std::panicking::try::do_call<core::panic::unwind_safe::AssertUnwindSafe<tokio::runtime::task::harness::poll_future::closure_env$0<hyper::server::server::new_svc::NewSvcTask<hyper::server::tcp::addr_stream::AddrStream,axum::routing::into_make_service::Into
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\std\src\panicking.rs:483
  34: std::panicking::try::do_catch<core::panic::unwind_safe::AssertUnwindSafe<tokio::runtime::task::harness::poll_future::closure_env$0<hyper::server::server::new_svc::NewSvcTask<hyper::server::tcp::addr_stream::AddrStream,axum::routing::into_make_service::Int
  35: std::panicking::try<enum2$<core::task::poll::Poll<tuple$<> > >,core::panic::unwind_safe::AssertUnwindSafe<tokio::runtime::task::harness::poll_future::closure_env$0<hyper::server::server::new_svc::NewSvcTask<hyper::server::tcp::addr_stream::AddrStream,axum
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\std\src\panicking.rs:447
  36: std::panic::catch_unwind<core::panic::unwind_safe::AssertUnwindSafe<tokio::runtime::task::harness::poll_future::closure_env$0<hyper::server::server::new_svc::NewSvcTask<hyper::server::tcp::addr_stream::AddrStream,axum::routing::into_make_service::IntoMake
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\std\src\panic.rs:137
  37: tokio::runtime::task::harness::poll_future<hyper::server::server::new_svc::NewSvcTask<hyper::server::tcp::addr_stream::AddrStream,axum::routing::into_make_service::IntoMakeServiceFuture<axum::routing::Router<tuple$<>,hyper::body::body::Body> >,axum::routi
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\task\harness.rs:464
  38: tokio::runtime::task::harness::Harness<hyper::server::server::new_svc::NewSvcTask<hyper::server::tcp::addr_stream::AddrStream,axum::routing::into_make_service::IntoMakeServiceFuture<axum::routing::Router<tuple$<>,hyper::body::body::Body> >,axum::routing::
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\task\harness.rs:198
  39: tokio::runtime::task::harness::Harness<hyper::server::server::new_svc::NewSvcTask<hyper::server::tcp::addr_stream::AddrStream,axum::routing::into_make_service::IntoMakeServiceFuture<axum::routing::Router<tuple$<>,hyper::body::body::Body> >,axum::routing::
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\task\harness.rs:152
  40: tokio::runtime::task::raw::poll<hyper::server::server::new_svc::NewSvcTask<hyper::server::tcp::addr_stream::AddrStream,axum::routing::into_make_service::IntoMakeServiceFuture<axum::routing::Router<tuple$<>,hyper::body::body::Body> >,axum::routing::Router<
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\task\raw.rs:255
  41: tokio::runtime::task::raw::RawTask::poll
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\task\raw.rs:200
  42: tokio::runtime::task::LocalNotified<alloc::sync::Arc<tokio::runtime::scheduler::multi_thread::handle::Handle> >::run<alloc::sync::Arc<tokio::runtime::scheduler::multi_thread::handle::Handle> >
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\task\mod.rs:394
  43: tokio::runtime::scheduler::multi_thread::worker::impl$1::run_task::closure$0
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\scheduler\multi_thread\worker.rs:464
  44: tokio::runtime::coop::with_budget
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\coop.rs:107
  45: tokio::runtime::coop::budget
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\coop.rs:73
  46: tokio::runtime::scheduler::multi_thread::worker::Context::run_task
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\scheduler\multi_thread\worker.rs:463
  47: tokio::runtime::scheduler::multi_thread::worker::Context::run
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\scheduler\multi_thread\worker.rs:426
  48: tokio::runtime::scheduler::multi_thread::worker::run::closure$0
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\scheduler\multi_thread\worker.rs:406
  49: tokio::macros::scoped_tls::ScopedKey<tokio::runtime::scheduler::multi_thread::worker::Context>::set<tokio::runtime::scheduler::multi_thread::worker::Context,tokio::runtime::scheduler::multi_thread::worker::run::closure_env$0,tuple$<> >
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\macros\scoped_tls.rs:61
  50: tokio::runtime::scheduler::multi_thread::worker::run
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\scheduler\multi_thread\worker.rs:403
  51: tokio::runtime::scheduler::multi_thread::worker::impl$0::launch::closure$0
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\scheduler\multi_thread\worker.rs:365
  52: tokio::runtime::blocking::task::impl$2::poll<tokio::runtime::scheduler::multi_thread::worker::impl$0::launch::closure_env$0,tuple$<> >
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\blocking\task.rs:42
  53: tokio::runtime::task::core::impl$6::poll::closure$0<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::impl$0::launch::closure_env$0>,tokio::runtime::blocking::schedule::BlockingSchedule>
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\task\core.rs:223
  54: tokio::loom::std::unsafe_cell::UnsafeCell<enum2$<tokio::runtime::task::core::Stage<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::impl$0::launch::closure_env$0> > > >::with_mut<enum2$<tokio::runtime::task::co
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\loom\std\unsafe_cell.rs:14
  55: tokio::runtime::task::core::Core<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::impl$0::launch::closure_env$0>,tokio::runtime::blocking::schedule::BlockingSchedule>::poll<tokio::runtime::blocking::task::Block
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\task\core.rs:212
  56: tokio::runtime::task::harness::poll_future::closure$0<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::impl$0::launch::closure_env$0>,tokio::runtime::blocking::schedule::BlockingSchedule>
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\task\harness.rs:476
  57: core::panic::unwind_safe::impl$23::call_once<enum2$<core::task::poll::Poll<tuple$<> > >,tokio::runtime::task::harness::poll_future::closure_env$0<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::impl$0::launch:
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\core\src\panic\unwind_safe.rs:271
  58: std::panicking::try::do_call<core::panic::unwind_safe::AssertUnwindSafe<tokio::runtime::task::harness::poll_future::closure_env$0<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::impl$0::launch::closure_env$0>,
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\std\src\panicking.rs:483
  59: std::panicking::try::do_catch<core::panic::unwind_safe::AssertUnwindSafe<tokio::runtime::task::harness::cancel_task::closure_env$0<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::impl$0::launch::closure_env$0>
  60: std::panicking::try<enum2$<core::task::poll::Poll<tuple$<> > >,core::panic::unwind_safe::AssertUnwindSafe<tokio::runtime::task::harness::poll_future::closure_env$0<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worke
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\std\src\panicking.rs:447
  61: std::panic::catch_unwind<core::panic::unwind_safe::AssertUnwindSafe<tokio::runtime::task::harness::poll_future::closure_env$0<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::impl$0::launch::closure_env$0>,toki
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\std\src\panic.rs:137
  62: tokio::runtime::task::harness::poll_future<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::impl$0::launch::closure_env$0>,tokio::runtime::blocking::schedule::BlockingSchedule>
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\task\harness.rs:464
  63: tokio::runtime::task::harness::Harness<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::impl$0::launch::closure_env$0>,tokio::runtime::blocking::schedule::BlockingSchedule>::poll_inner<tokio::runtime::blocking:
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\task\harness.rs:198
  64: tokio::runtime::task::harness::Harness<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::impl$0::launch::closure_env$0>,tokio::runtime::blocking::schedule::BlockingSchedule>::poll<tokio::runtime::blocking::task:
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\task\harness.rs:152
  65: tokio::runtime::task::raw::poll<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::impl$0::launch::closure_env$0>,tokio::runtime::blocking::schedule::BlockingSchedule>   
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\task\raw.rs:255
  66: tokio::runtime::task::raw::RawTask::poll
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\task\raw.rs:200
  67: tokio::runtime::task::UnownedTask<tokio::runtime::blocking::schedule::BlockingSchedule>::run<tokio::runtime::blocking::schedule::BlockingSchedule>
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\task\mod.rs:431
  68: tokio::runtime::blocking::pool::Task::run
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\blocking\pool.rs:159
  69: tokio::runtime::blocking::pool::Inner::run
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\blocking\pool.rs:511
  70: tokio::runtime::blocking::pool::impl$6::spawn_thread::closure$0
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\blocking\pool.rs:469
  71: std::sys_common::backtrace::__rust_begin_short_backtrace<tokio::runtime::blocking::pool::impl$6::spawn_thread::closure_env$0,tuple$<> >
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\std\src\sys_common\backtrace.rs:121
  72: std::sys_common::backtrace::__rust_begin_short_backtrace<tokio::runtime::blocking::pool::impl$6::spawn_thread::closure_env$0,tuple$<> >
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\std\src\sys_common\backtrace.rs:121
  73: std::thread::impl$0::spawn_unchecked_::closure$1::closure$0<tokio::runtime::blocking::pool::impl$6::spawn_thread::closure_env$0,tuple$<> >
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\std\src\thread\mod.rs:550
  74: core::panic::unwind_safe::impl$23::call_once<tuple$<>,std::thread::impl$0::spawn_unchecked_::closure$1::closure_env$0<tokio::runtime::blocking::pool::impl$6::spawn_thread::closure_env$0,tuple$<> > >
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\core\src\panic\unwind_safe.rs:271
  75: std::panicking::try::do_call<core::panic::unwind_safe::AssertUnwindSafe<std::thread::impl$0::spawn_unchecked_::closure$1::closure_env$0<tokio::runtime::blocking::pool::impl$6::spawn_thread::closure_env$0,tuple$<> > >,tuple$<> >
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\std\src\panicking.rs:483
  76: std::panicking::try::do_catch<core::panic::unwind_safe::AssertUnwindSafe<tokio::runtime::task::harness::cancel_task::closure_env$0<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::impl$0::launch::closure_env$0>
  77: std::panicking::try<tuple$<>,core::panic::unwind_safe::AssertUnwindSafe<std::thread::impl$0::spawn_unchecked_::closure$1::closure_env$0<tokio::runtime::blocking::pool::impl$6::spawn_thread::closure_env$0,tuple$<> > > >
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\std\src\panicking.rs:447
  78: std::panic::catch_unwind<core::panic::unwind_safe::AssertUnwindSafe<std::thread::impl$0::spawn_unchecked_::closure$1::closure_env$0<tokio::runtime::blocking::pool::impl$6::spawn_thread::closure_env$0,tuple$<> > >,tuple$<> >
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\std\src\panic.rs:137
  79: std::thread::impl$0::spawn_unchecked_::closure$1<tokio::runtime::blocking::pool::impl$6::spawn_thread::closure_env$0,tuple$<> >
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\std\src\thread\mod.rs:549
  80: core::ops::function::FnOnce::call_once<std::thread::impl$0::spawn_unchecked_::closure_env$1<tokio::runtime::blocking::pool::impl$6::spawn_thread::closure_env$0,tuple$<> >,tuple$<> >
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\core\src\ops\function.rs:507
  81: alloc::boxed::impl$45::call_once
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library\alloc\src\boxed.rs:2000
  82: alloc::boxed::impl$45::call_once
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library\alloc\src\boxed.rs:2000
  83: std::sys::windows::thread::impl$0::new::thread_start
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library\std\src\sys\windows\thread.rs:56
  84: BaseThreadInitThunk
  85: RtlUserThreadStart

MyState::Clone:
===============
   0: std::backtrace_rs::backtrace::dbghelp::trace
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library\std\src\..\..\backtrace\src\backtrace\dbghelp.rs:98
   1: std::backtrace_rs::backtrace::trace_unsynchronized
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library\std\src\..\..\backtrace\src\backtrace\mod.rs:66
   2: std::backtrace::Backtrace::create
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library\std\src\backtrace.rs:332
   3: std::backtrace::Backtrace::force_capture
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library\std\src\backtrace.rs:314
   4: axum_test::impl$0::clone
             at .\src\main.rs:8
   5: axum::handler::service::impl$3::clone<enum2$<axum_test::read::async_fn_env$0> (*)(axum::extract::state::State<axum_test::MyState>),tuple$<enum2$<axum_core::extract::private::ViaParts>,axum::extract::state::State<axum_test::MyState> >,axum_test::MyState,hy
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\axum-0.6.10\src\handler\service.rs:143
   6: tower::util::map_response::impl$5::clone<axum::handler::service::HandlerService<enum2$<axum_test::read::async_fn_env$0> (*)(axum::extract::state::State<axum_test::MyState>),tuple$<enum2$<axum_core::extract::private::ViaParts>,axum::extract::state::State<a
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tower-0.4.13\src\util\map_response.rs:12
   7: tower::util::map_future::impl$6::clone<tower::util::map_response::MapResponse<axum::handler::service::HandlerService<enum2$<axum_test::read::async_fn_env$0> (*)(axum::extract::state::State<axum_test::MyState>),tuple$<enum2$<axum_core::extract::private::Vi
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tower-0.4.13\src\util\map_future.rs:14
   8: tower::util::boxed_clone::impl$3::clone_box<http::request::Request<hyper::body::body::Body>,tower::util::map_future::MapFuture<tower::util::map_response::MapResponse<axum::handler::service::HandlerService<enum2$<axum_test::read::async_fn_env$0> (*)(axum::
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tower-0.4.13\src\util\boxed_clone.rs:128
   9: tower::util::boxed_clone::impl$2::clone<http::request::Request<hyper::body::body::Body>,http::response::Response<http_body::combinators::box_body::UnsyncBoxBody<bytes::bytes::Bytes,axum_core::error::Error> >,enum2$<core::convert::Infallible> >
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tower-0.4.13\src\util\boxed_clone.rs:107
  10: axum::routing::route::Route<hyper::body::body::Body,enum2$<core::convert::Infallible> >::oneshot_inner<hyper::body::body::Body,enum2$<core::convert::Infallible> >
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\axum-0.6.10\src\routing\route.rs:48
  11: axum::routing::method_routing::MethodRouter<tuple$<>,hyper::body::body::Body,enum2$<core::convert::Infallible> >::call_with_state<tuple$<>,hyper::body::body::Body,enum2$<core::convert::Infallible> >
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\axum-0.6.10\src\routing\method_routing.rs:1114
  12: axum::routing::Router<tuple$<>,hyper::body::body::Body>::call_route<tuple$<>,hyper::body::body::Body>
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\axum-0.6.10\src\routing\mod.rs:499
  13: axum::routing::Router<tuple$<>,hyper::body::body::Body>::call_with_state<tuple$<>,hyper::body::body::Body>
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\axum-0.6.10\src\routing\mod.rs:452
  14: axum::routing::impl$5::call<hyper::body::body::Body>
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\axum-0.6.10\src\routing\mod.rs:573
  15: hyper::service::http::impl$0::call<axum::routing::Router<tuple$<>,hyper::body::body::Body>,hyper::body::body::Body,http_body::combinators::box_body::UnsyncBoxBody<bytes::bytes::Bytes,axum_core::error::Error> >    
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\hyper-0.14.24\src\service\http.rs:45
  16: hyper::proto::h1::dispatch::impl$6::recv_msg<axum::routing::Router<tuple$<>,hyper::body::body::Body>,http_body::combinators::box_body::UnsyncBoxBody<bytes::bytes::Bytes,axum_core::error::Error> >
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\hyper-0.14.24\src\proto\h1\dispatch.rs:517
  17: hyper::proto::h1::dispatch::Dispatcher<hyper::proto::h1::dispatch::Server<axum::routing::Router<tuple$<>,hyper::body::body::Body>,hyper::body::body::Body>,http_body::combinators::box_body::UnsyncBoxBody<bytes::bytes::Bytes,axum_core::error::Error>,hyper::
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\hyper-0.14.24\src\proto\h1\dispatch.rs:264
  18: hyper::proto::h1::dispatch::Dispatcher<hyper::proto::h1::dispatch::Server<axum::routing::Router<tuple$<>,hyper::body::body::Body>,hyper::body::body::Body>,http_body::combinators::box_body::UnsyncBoxBody<bytes::bytes::Bytes,axum_core::error::Error>,hyper::
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\hyper-0.14.24\src\proto\h1\dispatch.rs:188
  19: hyper::proto::h1::dispatch::Dispatcher<hyper::proto::h1::dispatch::Server<axum::routing::Router<tuple$<>,hyper::body::body::Body>,hyper::body::body::Body>,http_body::combinators::box_body::UnsyncBoxBody<bytes::bytes::Bytes,axum_core::error::Error>,hyper::
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\hyper-0.14.24\src\proto\h1\dispatch.rs:160
  20: hyper::proto::h1::dispatch::Dispatcher<hyper::proto::h1::dispatch::Server<axum::routing::Router<tuple$<>,hyper::body::body::Body>,hyper::body::body::Body>,http_body::combinators::box_body::UnsyncBoxBody<bytes::bytes::Bytes,axum_core::error::Error>,hyper::
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\hyper-0.14.24\src\proto\h1\dispatch.rs:137
  21: hyper::proto::h1::dispatch::Dispatcher<hyper::proto::h1::dispatch::Server<axum::routing::Router<tuple$<>,hyper::body::body::Body>,hyper::body::body::Body>,http_body::combinators::box_body::UnsyncBoxBody<bytes::bytes::Bytes,axum_core::error::Error>,hyper::
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\hyper-0.14.24\src\proto\h1\dispatch.rs:120
  22: hyper::proto::h1::dispatch::impl$1::poll<hyper::proto::h1::dispatch::Server<axum::routing::Router<tuple$<>,hyper::body::body::Body>,hyper::body::body::Body>,http_body::combinators::box_body::UnsyncBoxBody<bytes::bytes::Bytes,axum_core::error::Error>,hyper
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\hyper-0.14.24\src\proto\h1\dispatch.rs:424
  23: hyper::server::conn::impl$6::poll<hyper::server::tcp::addr_stream::AddrStream,http_body::combinators::box_body::UnsyncBoxBody<bytes::bytes::Bytes,axum_core::error::Error>,axum::routing::Router<tuple$<>,hyper::body::body::Body>,enum2$<hyper::common::exec::
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\hyper-0.14.24\src\server\conn.rs:952
  24: hyper::server::conn::upgrades::impl$1::poll<hyper::server::tcp::addr_stream::AddrStream,http_body::combinators::box_body::UnsyncBoxBody<bytes::bytes::Bytes,axum_core::error::Error>,axum::routing::Router<tuple$<>,hyper::body::body::Body>,enum2$<hyper::comm
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\hyper-0.14.24\src\server\conn.rs:1012
  25: hyper::server::server::new_svc::impl$1::poll<hyper::server::tcp::addr_stream::AddrStream,axum::routing::into_make_service::IntoMakeServiceFuture<axum::routing::Router<tuple$<>,hyper::body::body::Body> >,axum::routing::Router<tuple$<>,hyper::body::body::Bo
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\hyper-0.14.24\src\server\server.rs:751
  26: tokio::runtime::task::core::impl$6::poll::closure$0<hyper::server::server::new_svc::NewSvcTask<hyper::server::tcp::addr_stream::AddrStream,axum::routing::into_make_service::IntoMakeServiceFuture<axum::routing::Router<tuple$<>,hyper::body::body::Body> >,ax
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\task\core.rs:223
  27: tokio::loom::std::unsafe_cell::UnsafeCell<enum2$<tokio::runtime::task::core::Stage<hyper::server::server::new_svc::NewSvcTask<hyper::server::tcp::addr_stream::AddrStream,axum::routing::into_make_service::IntoMakeServiceFuture<axum::routing::Router<tuple$<
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\loom\std\unsafe_cell.rs:14
  28: tokio::runtime::task::core::Core<hyper::server::server::new_svc::NewSvcTask<hyper::server::tcp::addr_stream::AddrStream,axum::routing::into_make_service::IntoMakeServiceFuture<axum::routing::Router<tuple$<>,hyper::body::body::Body> >,axum::routing::Router
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\task\core.rs:212
  29: tokio::runtime::task::harness::poll_future::closure$0<hyper::server::server::new_svc::NewSvcTask<hyper::server::tcp::addr_stream::AddrStream,axum::routing::into_make_service::IntoMakeServiceFuture<axum::routing::Router<tuple$<>,hyper::body::body::Body> >,
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\task\harness.rs:476
  30: core::panic::unwind_safe::impl$23::call_once<enum2$<core::task::poll::Poll<tuple$<> > >,tokio::runtime::task::harness::poll_future::closure_env$0<hyper::server::server::new_svc::NewSvcTask<hyper::server::tcp::addr_stream::AddrStream,axum::routing::into_ma
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\core\src\panic\unwind_safe.rs:271
  31: std::panicking::try::do_call<core::panic::unwind_safe::AssertUnwindSafe<tokio::runtime::task::harness::poll_future::closure_env$0<hyper::server::server::new_svc::NewSvcTask<hyper::server::tcp::addr_stream::AddrStream,axum::routing::into_make_service::Into
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\std\src\panicking.rs:483
  32: std::panicking::try::do_catch<core::panic::unwind_safe::AssertUnwindSafe<tokio::runtime::task::harness::poll_future::closure_env$0<hyper::server::server::new_svc::NewSvcTask<hyper::server::tcp::addr_stream::AddrStream,axum::routing::into_make_service::Int
  33: std::panicking::try<enum2$<core::task::poll::Poll<tuple$<> > >,core::panic::unwind_safe::AssertUnwindSafe<tokio::runtime::task::harness::poll_future::closure_env$0<hyper::server::server::new_svc::NewSvcTask<hyper::server::tcp::addr_stream::AddrStream,axum
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\std\src\panicking.rs:447
  34: std::panic::catch_unwind<core::panic::unwind_safe::AssertUnwindSafe<tokio::runtime::task::harness::poll_future::closure_env$0<hyper::server::server::new_svc::NewSvcTask<hyper::server::tcp::addr_stream::AddrStream,axum::routing::into_make_service::IntoMake
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\std\src\panic.rs:137
  35: tokio::runtime::task::harness::poll_future<hyper::server::server::new_svc::NewSvcTask<hyper::server::tcp::addr_stream::AddrStream,axum::routing::into_make_service::IntoMakeServiceFuture<axum::routing::Router<tuple$<>,hyper::body::body::Body> >,axum::routi
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\task\harness.rs:464
  36: tokio::runtime::task::harness::Harness<hyper::server::server::new_svc::NewSvcTask<hyper::server::tcp::addr_stream::AddrStream,axum::routing::into_make_service::IntoMakeServiceFuture<axum::routing::Router<tuple$<>,hyper::body::body::Body> >,axum::routing::
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\task\harness.rs:198
  37: tokio::runtime::task::harness::Harness<hyper::server::server::new_svc::NewSvcTask<hyper::server::tcp::addr_stream::AddrStream,axum::routing::into_make_service::IntoMakeServiceFuture<axum::routing::Router<tuple$<>,hyper::body::body::Body> >,axum::routing::
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\task\harness.rs:152
  38: tokio::runtime::task::raw::poll<hyper::server::server::new_svc::NewSvcTask<hyper::server::tcp::addr_stream::AddrStream,axum::routing::into_make_service::IntoMakeServiceFuture<axum::routing::Router<tuple$<>,hyper::body::body::Body> >,axum::routing::Router<
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\task\raw.rs:255
  39: tokio::runtime::task::raw::RawTask::poll
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\task\raw.rs:200
  40: tokio::runtime::task::LocalNotified<alloc::sync::Arc<tokio::runtime::scheduler::multi_thread::handle::Handle> >::run<alloc::sync::Arc<tokio::runtime::scheduler::multi_thread::handle::Handle> >
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\task\mod.rs:394
  41: tokio::runtime::scheduler::multi_thread::worker::impl$1::run_task::closure$0
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\scheduler\multi_thread\worker.rs:464
  42: tokio::runtime::coop::with_budget
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\coop.rs:107
  43: tokio::runtime::coop::budget
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\coop.rs:73
  44: tokio::runtime::scheduler::multi_thread::worker::Context::run_task
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\scheduler\multi_thread\worker.rs:463
  45: tokio::runtime::scheduler::multi_thread::worker::Context::run
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\scheduler\multi_thread\worker.rs:426
  46: tokio::runtime::scheduler::multi_thread::worker::run::closure$0
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\scheduler\multi_thread\worker.rs:406
  47: tokio::macros::scoped_tls::ScopedKey<tokio::runtime::scheduler::multi_thread::worker::Context>::set<tokio::runtime::scheduler::multi_thread::worker::Context,tokio::runtime::scheduler::multi_thread::worker::run::closure_env$0,tuple$<> >
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\macros\scoped_tls.rs:61
  48: tokio::runtime::scheduler::multi_thread::worker::run
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\scheduler\multi_thread\worker.rs:403
  49: tokio::runtime::scheduler::multi_thread::worker::impl$0::launch::closure$0
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\scheduler\multi_thread\worker.rs:365
  50: tokio::runtime::blocking::task::impl$2::poll<tokio::runtime::scheduler::multi_thread::worker::impl$0::launch::closure_env$0,tuple$<> >
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\blocking\task.rs:42
  51: tokio::runtime::task::core::impl$6::poll::closure$0<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::impl$0::launch::closure_env$0>,tokio::runtime::blocking::schedule::BlockingSchedule>
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\task\core.rs:223
  52: tokio::loom::std::unsafe_cell::UnsafeCell<enum2$<tokio::runtime::task::core::Stage<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::impl$0::launch::closure_env$0> > > >::with_mut<enum2$<tokio::runtime::task::co
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\loom\std\unsafe_cell.rs:14
  53: tokio::runtime::task::core::Core<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::impl$0::launch::closure_env$0>,tokio::runtime::blocking::schedule::BlockingSchedule>::poll<tokio::runtime::blocking::task::Block
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\task\core.rs:212
  54: tokio::runtime::task::harness::poll_future::closure$0<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::impl$0::launch::closure_env$0>,tokio::runtime::blocking::schedule::BlockingSchedule>
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\task\harness.rs:476
  55: core::panic::unwind_safe::impl$23::call_once<enum2$<core::task::poll::Poll<tuple$<> > >,tokio::runtime::task::harness::poll_future::closure_env$0<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::impl$0::launch:
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\core\src\panic\unwind_safe.rs:271
  56: std::panicking::try::do_call<core::panic::unwind_safe::AssertUnwindSafe<tokio::runtime::task::harness::poll_future::closure_env$0<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::impl$0::launch::closure_env$0>,
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\std\src\panicking.rs:483
  57: std::panicking::try::do_catch<core::panic::unwind_safe::AssertUnwindSafe<tokio::runtime::task::harness::cancel_task::closure_env$0<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::impl$0::launch::closure_env$0>
  58: std::panicking::try<enum2$<core::task::poll::Poll<tuple$<> > >,core::panic::unwind_safe::AssertUnwindSafe<tokio::runtime::task::harness::poll_future::closure_env$0<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worke
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\std\src\panicking.rs:447
  59: std::panic::catch_unwind<core::panic::unwind_safe::AssertUnwindSafe<tokio::runtime::task::harness::poll_future::closure_env$0<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::impl$0::launch::closure_env$0>,toki
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\std\src\panic.rs:137
  60: tokio::runtime::task::harness::poll_future<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::impl$0::launch::closure_env$0>,tokio::runtime::blocking::schedule::BlockingSchedule>
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\task\harness.rs:464
  61: tokio::runtime::task::harness::Harness<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::impl$0::launch::closure_env$0>,tokio::runtime::blocking::schedule::BlockingSchedule>::poll_inner<tokio::runtime::blocking:
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\task\harness.rs:198
  62: tokio::runtime::task::harness::Harness<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::impl$0::launch::closure_env$0>,tokio::runtime::blocking::schedule::BlockingSchedule>::poll<tokio::runtime::blocking::task:
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\task\harness.rs:152
  63: tokio::runtime::task::raw::poll<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::impl$0::launch::closure_env$0>,tokio::runtime::blocking::schedule::BlockingSchedule>   
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\task\raw.rs:255
  64: tokio::runtime::task::raw::RawTask::poll
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\task\raw.rs:200
  65: tokio::runtime::task::UnownedTask<tokio::runtime::blocking::schedule::BlockingSchedule>::run<tokio::runtime::blocking::schedule::BlockingSchedule>
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\task\mod.rs:431
  66: tokio::runtime::blocking::pool::Task::run
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\blocking\pool.rs:159
  67: tokio::runtime::blocking::pool::Inner::run
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\blocking\pool.rs:511
  68: tokio::runtime::blocking::pool::impl$6::spawn_thread::closure$0
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\blocking\pool.rs:469
  69: std::sys_common::backtrace::__rust_begin_short_backtrace<tokio::runtime::blocking::pool::impl$6::spawn_thread::closure_env$0,tuple$<> >
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\std\src\sys_common\backtrace.rs:121
  70: std::sys_common::backtrace::__rust_begin_short_backtrace<tokio::runtime::blocking::pool::impl$6::spawn_thread::closure_env$0,tuple$<> >
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\std\src\sys_common\backtrace.rs:121
  71: std::thread::impl$0::spawn_unchecked_::closure$1::closure$0<tokio::runtime::blocking::pool::impl$6::spawn_thread::closure_env$0,tuple$<> >
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\std\src\thread\mod.rs:550
  72: core::panic::unwind_safe::impl$23::call_once<tuple$<>,std::thread::impl$0::spawn_unchecked_::closure$1::closure_env$0<tokio::runtime::blocking::pool::impl$6::spawn_thread::closure_env$0,tuple$<> > >
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\core\src\panic\unwind_safe.rs:271
  73: std::panicking::try::do_call<core::panic::unwind_safe::AssertUnwindSafe<std::thread::impl$0::spawn_unchecked_::closure$1::closure_env$0<tokio::runtime::blocking::pool::impl$6::spawn_thread::closure_env$0,tuple$<> > >,tuple$<> >
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\std\src\panicking.rs:483
  74: std::panicking::try::do_catch<core::panic::unwind_safe::AssertUnwindSafe<tokio::runtime::task::harness::cancel_task::closure_env$0<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::impl$0::launch::closure_env$0>
  75: std::panicking::try<tuple$<>,core::panic::unwind_safe::AssertUnwindSafe<std::thread::impl$0::spawn_unchecked_::closure$1::closure_env$0<tokio::runtime::blocking::pool::impl$6::spawn_thread::closure_env$0,tuple$<> > > >
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\std\src\panicking.rs:447
  76: std::panic::catch_unwind<core::panic::unwind_safe::AssertUnwindSafe<std::thread::impl$0::spawn_unchecked_::closure$1::closure_env$0<tokio::runtime::blocking::pool::impl$6::spawn_thread::closure_env$0,tuple$<> > >,tuple$<> >
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\std\src\panic.rs:137
  77: std::thread::impl$0::spawn_unchecked_::closure$1<tokio::runtime::blocking::pool::impl$6::spawn_thread::closure_env$0,tuple$<> >
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\std\src\thread\mod.rs:549
  78: core::ops::function::FnOnce::call_once<std::thread::impl$0::spawn_unchecked_::closure_env$1<tokio::runtime::blocking::pool::impl$6::spawn_thread::closure_env$0,tuple$<> >,tuple$<> >
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\core\src\ops\function.rs:507
  79: alloc::boxed::impl$45::call_once
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library\alloc\src\boxed.rs:2000
  80: alloc::boxed::impl$45::call_once
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library\alloc\src\boxed.rs:2000
  81: std::sys::windows::thread::impl$0::new::thread_start
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library\std\src\sys\windows\thread.rs:56
  82: BaseThreadInitThunk
  83: RtlUserThreadStart

MyState::Clone:
===============
   0: std::backtrace_rs::backtrace::dbghelp::trace
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library\std\src\..\..\backtrace\src\backtrace\dbghelp.rs:98
   1: std::backtrace_rs::backtrace::trace_unsynchronized
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library\std\src\..\..\backtrace\src\backtrace\mod.rs:66
   2: std::backtrace::Backtrace::create
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library\std\src\backtrace.rs:332
   3: std::backtrace::Backtrace::force_capture
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library\std\src\backtrace.rs:314
   4: axum_test::impl$0::clone
             at .\src\main.rs:8
   5: axum::handler::service::impl$4::call<enum2$<axum_test::read::async_fn_env$0> (*)(axum::extract::state::State<axum_test::MyState>),tuple$<enum2$<axum_core::extract::private::ViaParts>,axum::extract::state::State<axum_test::MyState> >,axum_test::MyState,hyp
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\axum-0.6.10\src\handler\service.rs:171
   6: tower::util::map_response::impl$2::call<axum::handler::service::HandlerService<enum2$<axum_test::read::async_fn_env$0> (*)(axum::extract::state::State<axum_test::MyState>),tuple$<enum2$<axum_core::extract::private::ViaParts>,axum::extract::state::State<ax
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tower-0.4.13\src\util\map_response.rs:75
   7: tower::util::map_future::impl$1::call<http::request::Request<hyper::body::body::Body>,tower::util::map_response::MapResponse<axum::handler::service::HandlerService<enum2$<axum_test::read::async_fn_env$0> (*)(axum::extract::state::State<axum_test::MyState>
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tower-0.4.13\src\util\map_future.rs:65
   8: tower_service::impl$1::call<dyn$<tower::util::boxed_clone::CloneService<http::request::Request<hyper::body::body::Body>,assoc$<Error,enum2$<core::convert::Infallible> >,assoc$<Future,core::pin::Pin<alloc::boxed::Box<dyn$<core::future::future::Future<assoc
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tower-service-0.3.2\src\lib.rs:387
   9: tower::util::boxed_clone::impl$1::call<http::request::Request<hyper::body::body::Body>,http::response::Response<http_body::combinators::box_body::UnsyncBoxBody<bytes::bytes::Bytes,axum_core::error::Error> >,enum2$<core::convert::Infallible> >
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tower-0.4.13\src\util\boxed_clone.rs:101
  10: tower::util::oneshot::impl$3::poll<tower::util::boxed_clone::BoxCloneService<http::request::Request<hyper::body::body::Body>,http::response::Response<http_body::combinators::box_body::UnsyncBoxBody<bytes::bytes::Bytes,axum_core::error::Error> >,enum2$<cor
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tower-0.4.13\src\util\oneshot.rs:93
  11: axum::routing::route::impl$5::poll<hyper::body::body::Body,enum2$<core::convert::Infallible> >
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\axum-0.6.10\src\routing\route.rs:161
  12: hyper::proto::h1::dispatch::impl$6::poll_msg<axum::routing::Router<tuple$<>,hyper::body::body::Body>,http_body::combinators::box_body::UnsyncBoxBody<bytes::bytes::Bytes,axum_core::error::Error> >
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\hyper-0.14.24\src\proto\h1\dispatch.rs:491
  13: hyper::proto::h1::dispatch::Dispatcher<hyper::proto::h1::dispatch::Server<axum::routing::Router<tuple$<>,hyper::body::body::Body>,hyper::body::body::Body>,http_body::combinators::box_body::UnsyncBoxBody<bytes::bytes::Bytes,axum_core::error::Error>,hyper::
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\hyper-0.14.24\src\proto\h1\dispatch.rs:297
  14: hyper::proto::h1::dispatch::Dispatcher<hyper::proto::h1::dispatch::Server<axum::routing::Router<tuple$<>,hyper::body::body::Body>,hyper::body::body::Body>,http_body::combinators::box_body::UnsyncBoxBody<bytes::bytes::Bytes,axum_core::error::Error>,hyper::
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\hyper-0.14.24\src\proto\h1\dispatch.rs:161
  15: hyper::proto::h1::dispatch::Dispatcher<hyper::proto::h1::dispatch::Server<axum::routing::Router<tuple$<>,hyper::body::body::Body>,hyper::body::body::Body>,http_body::combinators::box_body::UnsyncBoxBody<bytes::bytes::Bytes,axum_core::error::Error>,hyper::
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\hyper-0.14.24\src\proto\h1\dispatch.rs:137
  16: hyper::proto::h1::dispatch::Dispatcher<hyper::proto::h1::dispatch::Server<axum::routing::Router<tuple$<>,hyper::body::body::Body>,hyper::body::body::Body>,http_body::combinators::box_body::UnsyncBoxBody<bytes::bytes::Bytes,axum_core::error::Error>,hyper::
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\hyper-0.14.24\src\proto\h1\dispatch.rs:120
  17: hyper::proto::h1::dispatch::impl$1::poll<hyper::proto::h1::dispatch::Server<axum::routing::Router<tuple$<>,hyper::body::body::Body>,hyper::body::body::Body>,http_body::combinators::box_body::UnsyncBoxBody<bytes::bytes::Bytes,axum_core::error::Error>,hyper
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\hyper-0.14.24\src\proto\h1\dispatch.rs:424
  18: hyper::server::conn::impl$6::poll<hyper::server::tcp::addr_stream::AddrStream,http_body::combinators::box_body::UnsyncBoxBody<bytes::bytes::Bytes,axum_core::error::Error>,axum::routing::Router<tuple$<>,hyper::body::body::Body>,enum2$<hyper::common::exec::
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\hyper-0.14.24\src\server\conn.rs:952
  19: hyper::server::conn::upgrades::impl$1::poll<hyper::server::tcp::addr_stream::AddrStream,http_body::combinators::box_body::UnsyncBoxBody<bytes::bytes::Bytes,axum_core::error::Error>,axum::routing::Router<tuple$<>,hyper::body::body::Body>,enum2$<hyper::comm
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\hyper-0.14.24\src\server\conn.rs:1012
  20: hyper::server::server::new_svc::impl$1::poll<hyper::server::tcp::addr_stream::AddrStream,axum::routing::into_make_service::IntoMakeServiceFuture<axum::routing::Router<tuple$<>,hyper::body::body::Body> >,axum::routing::Router<tuple$<>,hyper::body::body::Bo
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\hyper-0.14.24\src\server\server.rs:751
  21: tokio::runtime::task::core::impl$6::poll::closure$0<hyper::server::server::new_svc::NewSvcTask<hyper::server::tcp::addr_stream::AddrStream,axum::routing::into_make_service::IntoMakeServiceFuture<axum::routing::Router<tuple$<>,hyper::body::body::Body> >,ax
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\task\core.rs:223
  22: tokio::loom::std::unsafe_cell::UnsafeCell<enum2$<tokio::runtime::task::core::Stage<hyper::server::server::new_svc::NewSvcTask<hyper::server::tcp::addr_stream::AddrStream,axum::routing::into_make_service::IntoMakeServiceFuture<axum::routing::Router<tuple$<
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\loom\std\unsafe_cell.rs:14
  23: tokio::runtime::task::core::Core<hyper::server::server::new_svc::NewSvcTask<hyper::server::tcp::addr_stream::AddrStream,axum::routing::into_make_service::IntoMakeServiceFuture<axum::routing::Router<tuple$<>,hyper::body::body::Body> >,axum::routing::Router
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\task\core.rs:212
  24: tokio::runtime::task::harness::poll_future::closure$0<hyper::server::server::new_svc::NewSvcTask<hyper::server::tcp::addr_stream::AddrStream,axum::routing::into_make_service::IntoMakeServiceFuture<axum::routing::Router<tuple$<>,hyper::body::body::Body> >,
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\task\harness.rs:476
  25: core::panic::unwind_safe::impl$23::call_once<enum2$<core::task::poll::Poll<tuple$<> > >,tokio::runtime::task::harness::poll_future::closure_env$0<hyper::server::server::new_svc::NewSvcTask<hyper::server::tcp::addr_stream::AddrStream,axum::routing::into_ma
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\core\src\panic\unwind_safe.rs:271
  26: std::panicking::try::do_call<core::panic::unwind_safe::AssertUnwindSafe<tokio::runtime::task::harness::poll_future::closure_env$0<hyper::server::server::new_svc::NewSvcTask<hyper::server::tcp::addr_stream::AddrStream,axum::routing::into_make_service::Into
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\std\src\panicking.rs:483
  27: std::panicking::try::do_catch<core::panic::unwind_safe::AssertUnwindSafe<tokio::runtime::task::harness::poll_future::closure_env$0<hyper::server::server::new_svc::NewSvcTask<hyper::server::tcp::addr_stream::AddrStream,axum::routing::into_make_service::Int
  28: std::panicking::try<enum2$<core::task::poll::Poll<tuple$<> > >,core::panic::unwind_safe::AssertUnwindSafe<tokio::runtime::task::harness::poll_future::closure_env$0<hyper::server::server::new_svc::NewSvcTask<hyper::server::tcp::addr_stream::AddrStream,axum
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\std\src\panicking.rs:447
  29: std::panic::catch_unwind<core::panic::unwind_safe::AssertUnwindSafe<tokio::runtime::task::harness::poll_future::closure_env$0<hyper::server::server::new_svc::NewSvcTask<hyper::server::tcp::addr_stream::AddrStream,axum::routing::into_make_service::IntoMake
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\std\src\panic.rs:137
  30: tokio::runtime::task::harness::poll_future<hyper::server::server::new_svc::NewSvcTask<hyper::server::tcp::addr_stream::AddrStream,axum::routing::into_make_service::IntoMakeServiceFuture<axum::routing::Router<tuple$<>,hyper::body::body::Body> >,axum::routi
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\task\harness.rs:464
  31: tokio::runtime::task::harness::Harness<hyper::server::server::new_svc::NewSvcTask<hyper::server::tcp::addr_stream::AddrStream,axum::routing::into_make_service::IntoMakeServiceFuture<axum::routing::Router<tuple$<>,hyper::body::body::Body> >,axum::routing::
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\task\harness.rs:198
  32: tokio::runtime::task::harness::Harness<hyper::server::server::new_svc::NewSvcTask<hyper::server::tcp::addr_stream::AddrStream,axum::routing::into_make_service::IntoMakeServiceFuture<axum::routing::Router<tuple$<>,hyper::body::body::Body> >,axum::routing::
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\task\harness.rs:152
  33: tokio::runtime::task::raw::poll<hyper::server::server::new_svc::NewSvcTask<hyper::server::tcp::addr_stream::AddrStream,axum::routing::into_make_service::IntoMakeServiceFuture<axum::routing::Router<tuple$<>,hyper::body::body::Body> >,axum::routing::Router<
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\task\raw.rs:255
  34: tokio::runtime::task::raw::RawTask::poll
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\task\raw.rs:200
  35: tokio::runtime::task::LocalNotified<alloc::sync::Arc<tokio::runtime::scheduler::multi_thread::handle::Handle> >::run<alloc::sync::Arc<tokio::runtime::scheduler::multi_thread::handle::Handle> >
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\task\mod.rs:394
  36: tokio::runtime::scheduler::multi_thread::worker::impl$1::run_task::closure$0
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\scheduler\multi_thread\worker.rs:464
  37: tokio::runtime::coop::with_budget
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\coop.rs:107
  38: tokio::runtime::coop::budget
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\coop.rs:73
  39: tokio::runtime::scheduler::multi_thread::worker::Context::run_task
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\scheduler\multi_thread\worker.rs:463
  40: tokio::runtime::scheduler::multi_thread::worker::Context::run
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\scheduler\multi_thread\worker.rs:426
  41: tokio::runtime::scheduler::multi_thread::worker::run::closure$0
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\scheduler\multi_thread\worker.rs:406
  42: tokio::macros::scoped_tls::ScopedKey<tokio::runtime::scheduler::multi_thread::worker::Context>::set<tokio::runtime::scheduler::multi_thread::worker::Context,tokio::runtime::scheduler::multi_thread::worker::run::closure_env$0,tuple$<> >
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\macros\scoped_tls.rs:61
  43: tokio::runtime::scheduler::multi_thread::worker::run
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\scheduler\multi_thread\worker.rs:403
  44: tokio::runtime::scheduler::multi_thread::worker::impl$0::launch::closure$0
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\scheduler\multi_thread\worker.rs:365
  45: tokio::runtime::blocking::task::impl$2::poll<tokio::runtime::scheduler::multi_thread::worker::impl$0::launch::closure_env$0,tuple$<> >
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\blocking\task.rs:42
  46: tokio::runtime::task::core::impl$6::poll::closure$0<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::impl$0::launch::closure_env$0>,tokio::runtime::blocking::schedule::BlockingSchedule>
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\task\core.rs:223
  47: tokio::loom::std::unsafe_cell::UnsafeCell<enum2$<tokio::runtime::task::core::Stage<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::impl$0::launch::closure_env$0> > > >::with_mut<enum2$<tokio::runtime::task::co
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\loom\std\unsafe_cell.rs:14
  48: tokio::runtime::task::core::Core<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::impl$0::launch::closure_env$0>,tokio::runtime::blocking::schedule::BlockingSchedule>::poll<tokio::runtime::blocking::task::Block
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\task\core.rs:212
  49: tokio::runtime::task::harness::poll_future::closure$0<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::impl$0::launch::closure_env$0>,tokio::runtime::blocking::schedule::BlockingSchedule>
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\task\harness.rs:476
  50: core::panic::unwind_safe::impl$23::call_once<enum2$<core::task::poll::Poll<tuple$<> > >,tokio::runtime::task::harness::poll_future::closure_env$0<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::impl$0::launch:
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\core\src\panic\unwind_safe.rs:271
  51: std::panicking::try::do_call<core::panic::unwind_safe::AssertUnwindSafe<tokio::runtime::task::harness::poll_future::closure_env$0<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::impl$0::launch::closure_env$0>,
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\std\src\panicking.rs:483
  52: std::panicking::try::do_catch<core::panic::unwind_safe::AssertUnwindSafe<tokio::runtime::task::harness::cancel_task::closure_env$0<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::impl$0::launch::closure_env$0>
  53: std::panicking::try<enum2$<core::task::poll::Poll<tuple$<> > >,core::panic::unwind_safe::AssertUnwindSafe<tokio::runtime::task::harness::poll_future::closure_env$0<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worke
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\std\src\panicking.rs:447
  54: std::panic::catch_unwind<core::panic::unwind_safe::AssertUnwindSafe<tokio::runtime::task::harness::poll_future::closure_env$0<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::impl$0::launch::closure_env$0>,toki
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\std\src\panic.rs:137
  55: tokio::runtime::task::harness::poll_future<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::impl$0::launch::closure_env$0>,tokio::runtime::blocking::schedule::BlockingSchedule>
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\task\harness.rs:464
  56: tokio::runtime::task::harness::Harness<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::impl$0::launch::closure_env$0>,tokio::runtime::blocking::schedule::BlockingSchedule>::poll_inner<tokio::runtime::blocking:
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\task\harness.rs:198
  57: tokio::runtime::task::harness::Harness<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::impl$0::launch::closure_env$0>,tokio::runtime::blocking::schedule::BlockingSchedule>::poll<tokio::runtime::blocking::task:
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\task\harness.rs:152
  58: tokio::runtime::task::raw::poll<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::impl$0::launch::closure_env$0>,tokio::runtime::blocking::schedule::BlockingSchedule>   
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\task\raw.rs:255
  59: tokio::runtime::task::raw::RawTask::poll
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\task\raw.rs:200
  60: tokio::runtime::task::UnownedTask<tokio::runtime::blocking::schedule::BlockingSchedule>::run<tokio::runtime::blocking::schedule::BlockingSchedule>
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\task\mod.rs:431
  61: tokio::runtime::blocking::pool::Task::run
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\blocking\pool.rs:159
  62: tokio::runtime::blocking::pool::Inner::run
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\blocking\pool.rs:511
  63: tokio::runtime::blocking::pool::impl$6::spawn_thread::closure$0
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\blocking\pool.rs:469
  64: std::sys_common::backtrace::__rust_begin_short_backtrace<tokio::runtime::blocking::pool::impl$6::spawn_thread::closure_env$0,tuple$<> >
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\std\src\sys_common\backtrace.rs:121
  65: std::sys_common::backtrace::__rust_begin_short_backtrace<tokio::runtime::blocking::pool::impl$6::spawn_thread::closure_env$0,tuple$<> >
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\std\src\sys_common\backtrace.rs:121
  66: std::thread::impl$0::spawn_unchecked_::closure$1::closure$0<tokio::runtime::blocking::pool::impl$6::spawn_thread::closure_env$0,tuple$<> >
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\std\src\thread\mod.rs:550
  67: core::panic::unwind_safe::impl$23::call_once<tuple$<>,std::thread::impl$0::spawn_unchecked_::closure$1::closure_env$0<tokio::runtime::blocking::pool::impl$6::spawn_thread::closure_env$0,tuple$<> > >
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\core\src\panic\unwind_safe.rs:271
  68: std::panicking::try::do_call<core::panic::unwind_safe::AssertUnwindSafe<std::thread::impl$0::spawn_unchecked_::closure$1::closure_env$0<tokio::runtime::blocking::pool::impl$6::spawn_thread::closure_env$0,tuple$<> > >,tuple$<> >
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\std\src\panicking.rs:483
  69: std::panicking::try::do_catch<core::panic::unwind_safe::AssertUnwindSafe<tokio::runtime::task::harness::cancel_task::closure_env$0<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::impl$0::launch::closure_env$0>
  70: std::panicking::try<tuple$<>,core::panic::unwind_safe::AssertUnwindSafe<std::thread::impl$0::spawn_unchecked_::closure$1::closure_env$0<tokio::runtime::blocking::pool::impl$6::spawn_thread::closure_env$0,tuple$<> > > >
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\std\src\panicking.rs:447
  71: std::panic::catch_unwind<core::panic::unwind_safe::AssertUnwindSafe<std::thread::impl$0::spawn_unchecked_::closure$1::closure_env$0<tokio::runtime::blocking::pool::impl$6::spawn_thread::closure_env$0,tuple$<> > >,tuple$<> >
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\std\src\panic.rs:137
  72: std::thread::impl$0::spawn_unchecked_::closure$1<tokio::runtime::blocking::pool::impl$6::spawn_thread::closure_env$0,tuple$<> >
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\std\src\thread\mod.rs:549
  73: core::ops::function::FnOnce::call_once<std::thread::impl$0::spawn_unchecked_::closure_env$1<tokio::runtime::blocking::pool::impl$6::spawn_thread::closure_env$0,tuple$<> >,tuple$<> >
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\core\src\ops\function.rs:507
  74: alloc::boxed::impl$45::call_once
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library\alloc\src\boxed.rs:2000
  75: alloc::boxed::impl$45::call_once
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library\alloc\src\boxed.rs:2000
  76: std::sys::windows::thread::impl$0::new::thread_start
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library\std\src\sys\windows\thread.rs:56
  77: BaseThreadInitThunk
  78: RtlUserThreadStart

MyState::Clone:
===============
   0: std::backtrace_rs::backtrace::dbghelp::trace
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library\std\src\..\..\backtrace\src\backtrace\dbghelp.rs:98
   1: std::backtrace_rs::backtrace::trace_unsynchronized
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library\std\src\..\..\backtrace\src\backtrace\mod.rs:66
   2: std::backtrace::Backtrace::create
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library\std\src\backtrace.rs:332
   3: std::backtrace::Backtrace::force_capture
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library\std\src\backtrace.rs:314
   4: axum_test::impl$0::clone
             at .\src\main.rs:8
   5: axum_core::extract::from_ref::impl$0::from_ref<axum_test::MyState>
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\axum-core-0.3.3\src\extract\from_ref.rs:23
   6: axum::extract::state::impl$6::from_request_parts::async_block$0<axum_test::MyState,axum_test::MyState>
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\axum-0.6.10\src\extract\state.rs:361
   7: core::future::future::impl$1::poll<alloc::boxed::Box<dyn$<core::future::future::Future<assoc$<Output,enum2$<core::result::Result<axum::extract::state::State<axum_test::MyState>,enum2$<core::convert::Infallible> > > > >,core::marker::Send>,alloc::alloc::Gl
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\core\src\future\future.rs:124
   8: axum_core::extract::impl$0::from_request::async_block$0<axum_test::MyState,hyper::body::body::Body,axum::extract::state::State<axum_test::MyState> >
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\axum-core-0.3.3\src\extract\mod.rs:137
   9: core::future::future::impl$1::poll<alloc::boxed::Box<dyn$<core::future::future::Future<assoc$<Output,enum2$<core::result::Result<axum::extract::state::State<axum_test::MyState>,enum2$<core::convert::Infallible> > > > >,core::marker::Send>,alloc::alloc::Gl
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\core\src\future\future.rs:124
  10: axum::handler::impl$5::call::async_block$0<enum2$<axum_test::read::async_fn_env$0> (*)(axum::extract::state::State<axum_test::MyState>),enum2$<axum_test::read::async_fn_env$0>,axum_test::MyState,hyper::body::body::Body,ref$<str$>,enum2$<axum_core::extract
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\axum-0.6.10\src\handler\mod.rs:208
  11: core::future::future::impl$1::poll<alloc::boxed::Box<dyn$<core::future::future::Future<assoc$<Output,http::response::Response<http_body::combinators::box_body::UnsyncBoxBody<bytes::bytes::Bytes,axum_core::error::Error> > > >,core::marker::Send>,alloc::all
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\core\src\future\future.rs:124
  12: futures_util::future::future::map::impl$2::poll<core::pin::Pin<alloc::boxed::Box<dyn$<core::future::future::Future<assoc$<Output,http::response::Response<http_body::combinators::box_body::UnsyncBoxBody<bytes::bytes::Bytes,axum_core::error::Error> > > >,co
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\futures-util-0.3.26\src\future\future\map.rs:55
  13: futures_util::future::future::impl$14::poll<core::pin::Pin<alloc::boxed::Box<dyn$<core::future::future::Future<assoc$<Output,http::response::Response<http_body::combinators::box_body::UnsyncBoxBody<bytes::bytes::Bytes,axum_core::error::Error> > > >,core::
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\futures-util-0.3.26\src\lib.rs:91
  14: axum::handler::future::impl$4::poll<core::pin::Pin<alloc::boxed::Box<dyn$<core::future::future::Future<assoc$<Output,http::response::Response<http_body::combinators::box_body::UnsyncBoxBody<bytes::bytes::Bytes,axum_core::error::Error> > > >,core::marker::
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\axum-0.6.10\src\macros.rs:42
  15: futures_core::future::impl$2::try_poll<axum::handler::future::IntoServiceFuture<core::pin::Pin<alloc::boxed::Box<dyn$<core::future::future::Future<assoc$<Output,http::response::Response<http_body::combinators::box_body::UnsyncBoxBody<bytes::bytes::Bytes,a
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\futures-core-0.3.26\src\future.rs:82
  16: futures_util::future::try_future::into_future::impl$2::poll<axum::handler::future::IntoServiceFuture<core::pin::Pin<alloc::boxed::Box<dyn$<core::future::future::Future<assoc$<Output,http::response::Response<http_body::combinators::box_body::UnsyncBoxBody<
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\futures-util-0.3.26\src\future\try_future\into_future.rs:34
  17: futures_util::future::future::map::impl$2::poll<futures_util::future::try_future::into_future::IntoFuture<axum::handler::future::IntoServiceFuture<core::pin::Pin<alloc::boxed::Box<dyn$<core::future::future::Future<assoc$<Output,http::response::Response<ht
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\futures-util-0.3.26\src\future\future\map.rs:55
  18: futures_util::future::future::impl$14::poll<futures_util::future::try_future::into_future::IntoFuture<axum::handler::future::IntoServiceFuture<core::pin::Pin<alloc::boxed::Box<dyn$<core::future::future::Future<assoc$<Output,http::response::Response<http_b
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\futures-util-0.3.26\src\lib.rs:91
  19: futures_util::future::try_future::impl$49::poll<axum::handler::future::IntoServiceFuture<core::pin::Pin<alloc::boxed::Box<dyn$<core::future::future::Future<assoc$<Output,http::response::Response<http_body::combinators::box_body::UnsyncBoxBody<bytes::bytes
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\futures-util-0.3.26\src\lib.rs:91
  20: tower::util::map_response::impl$10::poll<axum::handler::future::IntoServiceFuture<core::pin::Pin<alloc::boxed::Box<dyn$<core::future::future::Future<assoc$<Output,http::response::Response<http_body::combinators::box_body::UnsyncBoxBody<bytes::bytes::Bytes
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tower-0.4.13\src\macros.rs:38
  21: core::future::future::impl$1::poll<alloc::boxed::Box<dyn$<core::future::future::Future<assoc$<Output,enum2$<core::result::Result<http::response::Response<http_body::combinators::box_body::UnsyncBoxBody<bytes::bytes::Bytes,axum_core::error::Error> >,enum2$
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\core\src\future\future.rs:124
  22: tower::util::oneshot::impl$3::poll<tower::util::boxed_clone::BoxCloneService<http::request::Request<hyper::body::body::Body>,http::response::Response<http_body::combinators::box_body::UnsyncBoxBody<bytes::bytes::Bytes,axum_core::error::Error> >,enum2$<cor
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tower-0.4.13\src\util\oneshot.rs:97
  23: axum::routing::route::impl$5::poll<hyper::body::body::Body,enum2$<core::convert::Infallible> >
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\axum-0.6.10\src\routing\route.rs:161
  24: hyper::proto::h1::dispatch::impl$6::poll_msg<axum::routing::Router<tuple$<>,hyper::body::body::Body>,http_body::combinators::box_body::UnsyncBoxBody<bytes::bytes::Bytes,axum_core::error::Error> >
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\hyper-0.14.24\src\proto\h1\dispatch.rs:491
  25: hyper::proto::h1::dispatch::Dispatcher<hyper::proto::h1::dispatch::Server<axum::routing::Router<tuple$<>,hyper::body::body::Body>,hyper::body::body::Body>,http_body::combinators::box_body::UnsyncBoxBody<bytes::bytes::Bytes,axum_core::error::Error>,hyper::
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\hyper-0.14.24\src\proto\h1\dispatch.rs:297
  26: hyper::proto::h1::dispatch::Dispatcher<hyper::proto::h1::dispatch::Server<axum::routing::Router<tuple$<>,hyper::body::body::Body>,hyper::body::body::Body>,http_body::combinators::box_body::UnsyncBoxBody<bytes::bytes::Bytes,axum_core::error::Error>,hyper::
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\hyper-0.14.24\src\proto\h1\dispatch.rs:161
  27: hyper::proto::h1::dispatch::Dispatcher<hyper::proto::h1::dispatch::Server<axum::routing::Router<tuple$<>,hyper::body::body::Body>,hyper::body::body::Body>,http_body::combinators::box_body::UnsyncBoxBody<bytes::bytes::Bytes,axum_core::error::Error>,hyper::
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\hyper-0.14.24\src\proto\h1\dispatch.rs:137
  28: hyper::proto::h1::dispatch::Dispatcher<hyper::proto::h1::dispatch::Server<axum::routing::Router<tuple$<>,hyper::body::body::Body>,hyper::body::body::Body>,http_body::combinators::box_body::UnsyncBoxBody<bytes::bytes::Bytes,axum_core::error::Error>,hyper::
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\hyper-0.14.24\src\proto\h1\dispatch.rs:120
  29: hyper::proto::h1::dispatch::impl$1::poll<hyper::proto::h1::dispatch::Server<axum::routing::Router<tuple$<>,hyper::body::body::Body>,hyper::body::body::Body>,http_body::combinators::box_body::UnsyncBoxBody<bytes::bytes::Bytes,axum_core::error::Error>,hyper
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\hyper-0.14.24\src\proto\h1\dispatch.rs:424
  30: hyper::server::conn::impl$6::poll<hyper::server::tcp::addr_stream::AddrStream,http_body::combinators::box_body::UnsyncBoxBody<bytes::bytes::Bytes,axum_core::error::Error>,axum::routing::Router<tuple$<>,hyper::body::body::Body>,enum2$<hyper::common::exec::
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\hyper-0.14.24\src\server\conn.rs:952
  31: hyper::server::conn::upgrades::impl$1::poll<hyper::server::tcp::addr_stream::AddrStream,http_body::combinators::box_body::UnsyncBoxBody<bytes::bytes::Bytes,axum_core::error::Error>,axum::routing::Router<tuple$<>,hyper::body::body::Body>,enum2$<hyper::comm
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\hyper-0.14.24\src\server\conn.rs:1012
  32: hyper::server::server::new_svc::impl$1::poll<hyper::server::tcp::addr_stream::AddrStream,axum::routing::into_make_service::IntoMakeServiceFuture<axum::routing::Router<tuple$<>,hyper::body::body::Body> >,axum::routing::Router<tuple$<>,hyper::body::body::Bo
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\hyper-0.14.24\src\server\server.rs:751
  33: tokio::runtime::task::core::impl$6::poll::closure$0<hyper::server::server::new_svc::NewSvcTask<hyper::server::tcp::addr_stream::AddrStream,axum::routing::into_make_service::IntoMakeServiceFuture<axum::routing::Router<tuple$<>,hyper::body::body::Body> >,ax
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\task\core.rs:223
  34: tokio::loom::std::unsafe_cell::UnsafeCell<enum2$<tokio::runtime::task::core::Stage<hyper::server::server::new_svc::NewSvcTask<hyper::server::tcp::addr_stream::AddrStream,axum::routing::into_make_service::IntoMakeServiceFuture<axum::routing::Router<tuple$<
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\loom\std\unsafe_cell.rs:14
  35: tokio::runtime::task::core::Core<hyper::server::server::new_svc::NewSvcTask<hyper::server::tcp::addr_stream::AddrStream,axum::routing::into_make_service::IntoMakeServiceFuture<axum::routing::Router<tuple$<>,hyper::body::body::Body> >,axum::routing::Router
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\task\core.rs:212
  36: tokio::runtime::task::harness::poll_future::closure$0<hyper::server::server::new_svc::NewSvcTask<hyper::server::tcp::addr_stream::AddrStream,axum::routing::into_make_service::IntoMakeServiceFuture<axum::routing::Router<tuple$<>,hyper::body::body::Body> >,
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\task\harness.rs:476
  37: core::panic::unwind_safe::impl$23::call_once<enum2$<core::task::poll::Poll<tuple$<> > >,tokio::runtime::task::harness::poll_future::closure_env$0<hyper::server::server::new_svc::NewSvcTask<hyper::server::tcp::addr_stream::AddrStream,axum::routing::into_ma
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\core\src\panic\unwind_safe.rs:271
  38: std::panicking::try::do_call<core::panic::unwind_safe::AssertUnwindSafe<tokio::runtime::task::harness::poll_future::closure_env$0<hyper::server::server::new_svc::NewSvcTask<hyper::server::tcp::addr_stream::AddrStream,axum::routing::into_make_service::Into
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\std\src\panicking.rs:483
  39: std::panicking::try::do_catch<core::panic::unwind_safe::AssertUnwindSafe<tokio::runtime::task::harness::poll_future::closure_env$0<hyper::server::server::new_svc::NewSvcTask<hyper::server::tcp::addr_stream::AddrStream,axum::routing::into_make_service::Int
  40: std::panicking::try<enum2$<core::task::poll::Poll<tuple$<> > >,core::panic::unwind_safe::AssertUnwindSafe<tokio::runtime::task::harness::poll_future::closure_env$0<hyper::server::server::new_svc::NewSvcTask<hyper::server::tcp::addr_stream::AddrStream,axum
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\std\src\panicking.rs:447
  41: std::panic::catch_unwind<core::panic::unwind_safe::AssertUnwindSafe<tokio::runtime::task::harness::poll_future::closure_env$0<hyper::server::server::new_svc::NewSvcTask<hyper::server::tcp::addr_stream::AddrStream,axum::routing::into_make_service::IntoMake
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\std\src\panic.rs:137
  42: tokio::runtime::task::harness::poll_future<hyper::server::server::new_svc::NewSvcTask<hyper::server::tcp::addr_stream::AddrStream,axum::routing::into_make_service::IntoMakeServiceFuture<axum::routing::Router<tuple$<>,hyper::body::body::Body> >,axum::routi
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\task\harness.rs:464
  43: tokio::runtime::task::harness::Harness<hyper::server::server::new_svc::NewSvcTask<hyper::server::tcp::addr_stream::AddrStream,axum::routing::into_make_service::IntoMakeServiceFuture<axum::routing::Router<tuple$<>,hyper::body::body::Body> >,axum::routing::
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\task\harness.rs:198
  44: tokio::runtime::task::harness::Harness<hyper::server::server::new_svc::NewSvcTask<hyper::server::tcp::addr_stream::AddrStream,axum::routing::into_make_service::IntoMakeServiceFuture<axum::routing::Router<tuple$<>,hyper::body::body::Body> >,axum::routing::
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\task\harness.rs:152
  45: tokio::runtime::task::raw::poll<hyper::server::server::new_svc::NewSvcTask<hyper::server::tcp::addr_stream::AddrStream,axum::routing::into_make_service::IntoMakeServiceFuture<axum::routing::Router<tuple$<>,hyper::body::body::Body> >,axum::routing::Router<
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\task\raw.rs:255
  46: tokio::runtime::task::raw::RawTask::poll
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\task\raw.rs:200
  47: tokio::runtime::task::LocalNotified<alloc::sync::Arc<tokio::runtime::scheduler::multi_thread::handle::Handle> >::run<alloc::sync::Arc<tokio::runtime::scheduler::multi_thread::handle::Handle> >
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\task\mod.rs:394
  48: tokio::runtime::scheduler::multi_thread::worker::impl$1::run_task::closure$0
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\scheduler\multi_thread\worker.rs:464
  49: tokio::runtime::coop::with_budget
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\coop.rs:107
  50: tokio::runtime::coop::budget
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\coop.rs:73
  51: tokio::runtime::scheduler::multi_thread::worker::Context::run_task
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\scheduler\multi_thread\worker.rs:463
  52: tokio::runtime::scheduler::multi_thread::worker::Context::run
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\scheduler\multi_thread\worker.rs:426
  53: tokio::runtime::scheduler::multi_thread::worker::run::closure$0
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\scheduler\multi_thread\worker.rs:406
  54: tokio::macros::scoped_tls::ScopedKey<tokio::runtime::scheduler::multi_thread::worker::Context>::set<tokio::runtime::scheduler::multi_thread::worker::Context,tokio::runtime::scheduler::multi_thread::worker::run::closure_env$0,tuple$<> >
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\macros\scoped_tls.rs:61
  55: tokio::runtime::scheduler::multi_thread::worker::run
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\scheduler\multi_thread\worker.rs:403
  56: tokio::runtime::scheduler::multi_thread::worker::impl$0::launch::closure$0
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\scheduler\multi_thread\worker.rs:365
  57: tokio::runtime::blocking::task::impl$2::poll<tokio::runtime::scheduler::multi_thread::worker::impl$0::launch::closure_env$0,tuple$<> >
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\blocking\task.rs:42
  58: tokio::runtime::task::core::impl$6::poll::closure$0<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::impl$0::launch::closure_env$0>,tokio::runtime::blocking::schedule::BlockingSchedule>
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\task\core.rs:223
  59: tokio::loom::std::unsafe_cell::UnsafeCell<enum2$<tokio::runtime::task::core::Stage<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::impl$0::launch::closure_env$0> > > >::with_mut<enum2$<tokio::runtime::task::co
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\loom\std\unsafe_cell.rs:14
  60: tokio::runtime::task::core::Core<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::impl$0::launch::closure_env$0>,tokio::runtime::blocking::schedule::BlockingSchedule>::poll<tokio::runtime::blocking::task::Block
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\task\core.rs:212
  61: tokio::runtime::task::harness::poll_future::closure$0<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::impl$0::launch::closure_env$0>,tokio::runtime::blocking::schedule::BlockingSchedule>
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\task\harness.rs:476
  62: core::panic::unwind_safe::impl$23::call_once<enum2$<core::task::poll::Poll<tuple$<> > >,tokio::runtime::task::harness::poll_future::closure_env$0<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::impl$0::launch:
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\core\src\panic\unwind_safe.rs:271
  63: std::panicking::try::do_call<core::panic::unwind_safe::AssertUnwindSafe<tokio::runtime::task::harness::poll_future::closure_env$0<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::impl$0::launch::closure_env$0>,
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\std\src\panicking.rs:483
  64: std::panicking::try::do_catch<core::panic::unwind_safe::AssertUnwindSafe<tokio::runtime::task::harness::cancel_task::closure_env$0<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::impl$0::launch::closure_env$0>
  65: std::panicking::try<enum2$<core::task::poll::Poll<tuple$<> > >,core::panic::unwind_safe::AssertUnwindSafe<tokio::runtime::task::harness::poll_future::closure_env$0<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worke
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\std\src\panicking.rs:447
  66: std::panic::catch_unwind<core::panic::unwind_safe::AssertUnwindSafe<tokio::runtime::task::harness::poll_future::closure_env$0<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::impl$0::launch::closure_env$0>,toki
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\std\src\panic.rs:137
  67: tokio::runtime::task::harness::poll_future<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::impl$0::launch::closure_env$0>,tokio::runtime::blocking::schedule::BlockingSchedule>
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\task\harness.rs:464
  68: tokio::runtime::task::harness::Harness<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::impl$0::launch::closure_env$0>,tokio::runtime::blocking::schedule::BlockingSchedule>::poll_inner<tokio::runtime::blocking:
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\task\harness.rs:198
  69: tokio::runtime::task::harness::Harness<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::impl$0::launch::closure_env$0>,tokio::runtime::blocking::schedule::BlockingSchedule>::poll<tokio::runtime::blocking::task:
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\task\harness.rs:152
  70: tokio::runtime::task::raw::poll<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::impl$0::launch::closure_env$0>,tokio::runtime::blocking::schedule::BlockingSchedule>   
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\task\raw.rs:255
  71: tokio::runtime::task::raw::RawTask::poll
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\task\raw.rs:200
  72: tokio::runtime::task::UnownedTask<tokio::runtime::blocking::schedule::BlockingSchedule>::run<tokio::runtime::blocking::schedule::BlockingSchedule>
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\task\mod.rs:431
  73: tokio::runtime::blocking::pool::Task::run
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\blocking\pool.rs:159
  74: tokio::runtime::blocking::pool::Inner::run
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\blocking\pool.rs:511
  75: tokio::runtime::blocking::pool::impl$6::spawn_thread::closure$0
             at C:\Users\michael\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.26.0\src\runtime\blocking\pool.rs:469
  76: std::sys_common::backtrace::__rust_begin_short_backtrace<tokio::runtime::blocking::pool::impl$6::spawn_thread::closure_env$0,tuple$<> >
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\std\src\sys_common\backtrace.rs:121
  77: std::sys_common::backtrace::__rust_begin_short_backtrace<tokio::runtime::blocking::pool::impl$6::spawn_thread::closure_env$0,tuple$<> >
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\std\src\sys_common\backtrace.rs:121
  78: std::thread::impl$0::spawn_unchecked_::closure$1::closure$0<tokio::runtime::blocking::pool::impl$6::spawn_thread::closure_env$0,tuple$<> >
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\std\src\thread\mod.rs:550
  79: core::panic::unwind_safe::impl$23::call_once<tuple$<>,std::thread::impl$0::spawn_unchecked_::closure$1::closure_env$0<tokio::runtime::blocking::pool::impl$6::spawn_thread::closure_env$0,tuple$<> > >
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\core\src\panic\unwind_safe.rs:271
  80: std::panicking::try::do_call<core::panic::unwind_safe::AssertUnwindSafe<std::thread::impl$0::spawn_unchecked_::closure$1::closure_env$0<tokio::runtime::blocking::pool::impl$6::spawn_thread::closure_env$0,tuple$<> > >,tuple$<> >
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\std\src\panicking.rs:483
  81: std::panicking::try::do_catch<core::panic::unwind_safe::AssertUnwindSafe<tokio::runtime::task::harness::cancel_task::closure_env$0<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::impl$0::launch::closure_env$0>
  82: std::panicking::try<tuple$<>,core::panic::unwind_safe::AssertUnwindSafe<std::thread::impl$0::spawn_unchecked_::closure$1::closure_env$0<tokio::runtime::blocking::pool::impl$6::spawn_thread::closure_env$0,tuple$<> > > >
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\std\src\panicking.rs:447
  83: std::panic::catch_unwind<core::panic::unwind_safe::AssertUnwindSafe<std::thread::impl$0::spawn_unchecked_::closure$1::closure_env$0<tokio::runtime::blocking::pool::impl$6::spawn_thread::closure_env$0,tuple$<> > >,tuple$<> >
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\std\src\panic.rs:137
  84: std::thread::impl$0::spawn_unchecked_::closure$1<tokio::runtime::blocking::pool::impl$6::spawn_thread::closure_env$0,tuple$<> >
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\std\src\thread\mod.rs:549
  85: core::ops::function::FnOnce::call_once<std::thread::impl$0::spawn_unchecked_::closure_env$1<tokio::runtime::blocking::pool::impl$6::spawn_thread::closure_env$0,tuple$<> >,tuple$<> >
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483\library\core\src\ops\function.rs:507
  86: alloc::boxed::impl$45::call_once
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library\alloc\src\boxed.rs:2000
  87: alloc::boxed::impl$45::call_once
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library\alloc\src\boxed.rs:2000
  88: std::sys::windows::thread::impl$0::new::thread_start
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library\std\src\sys\windows\thread.rs:56
  89: BaseThreadInitThunk
  90: RtlUserThreadStart

read route called!
jplatte commented 1 year ago

Many of the line are cut off unfortunately. Something seems to be wrong with way you captured the output 🥲

mcatanzariti commented 1 year ago

Even with a redirection to a file, I got the same problem. It is a limitation of the call stack width when using Backtrace::force_capture()?

Anyway, can't you get away with the source code file names and line numbers ?

jplatte commented 1 year ago

Hm, maybe. If not, we could also run this ourselves, maybe the backtrace isn't cut off on Linux 🤷🏼

davidpdrsn commented 1 year ago

Oh yeah that's indeed a lot of clones. I thought https://github.com/tokio-rs/axum/pull/1745 would help but seems there are still areas that could be improved. Should probably also have a test for this.

jplatte commented 1 year ago

I moved the into_make_service before the printing of "listening", this is what I get for one request on linux:

MyState::Clone:
===============
   0: <axum_test::MyState as core::clone::Clone>::clone
             at ./src/main.rs:8:58
   1: axum::routing::Router<S,B>::with_state::{{closure}}
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/axum-0.6.10/src/routing/mod.rs:399:73
   2: core::iter::adapters::map::map_fold::{{closure}}
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/core/src/iter/adapters/map.rs:84:28
   3: core::iter::traits::iterator::Iterator::fold
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/core/src/iter/traits/iterator.rs:2414:21
   4: <core::iter::adapters::map::Map<I,F> as core::iter::traits::iterator::Iterator>::fold
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/core/src/iter/adapters/map.rs:124:9
   5: core::iter::traits::iterator::Iterator::for_each
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/core/src/iter/traits/iterator.rs:831:9
   6: <hashbrown::map::HashMap<K,V,S,A> as core::iter::traits::collect::Extend<(K,V)>>::extend
             at /cargo/registry/src/github.com-1ecc6299db9ec823/hashbrown-0.12.3/src/map.rs:6407:9
   7: <std::collections::hash::map::HashMap<K,V,S> as core::iter::traits::collect::Extend<(K,V)>>::extend
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/collections/hash/map.rs:3037:9
   8: <std::collections::hash::map::HashMap<K,V,S> as core::iter::traits::collect::FromIterator<(K,V)>>::from_iter
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/collections/hash/map.rs:3022:9
   9: core::iter::traits::iterator::Iterator::collect
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/core/src/iter/traits/iterator.rs:1836:9
  10: axum::routing::Router<S,B>::with_state
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/axum-0.6.10/src/routing/mod.rs:393:22
  11: axum_test::main::{{closure}}
             at ./src/main.rs:18:15
  12: tokio::runtime::park::CachedParkThread::block_on::{{closure}}
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/park.rs:283:63
  13: tokio::runtime::coop::with_budget
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/coop.rs:107:5
  14: tokio::runtime::coop::budget
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/coop.rs:73:5
  15: tokio::runtime::park::CachedParkThread::block_on
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/park.rs:283:31
  16: tokio::runtime::context::BlockingRegionGuard::block_on
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/context.rs:315:13
  17: tokio::runtime::scheduler::multi_thread::MultiThread::block_on
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/scheduler/multi_thread/mod.rs:66:9
  18: tokio::runtime::runtime::Runtime::block_on
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/runtime.rs:304:45
  19: axum_test::main
             at ./src/main.rs:23:5
  20: core::ops::function::FnOnce::call_once
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/core/src/ops/function.rs:507:5
  21: std::sys_common::backtrace::__rust_begin_short_backtrace
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/sys_common/backtrace.rs:121:18
  22: std::rt::lang_start::{{closure}}
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/rt.rs:166:18
  23: core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &F>::call_once
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/core/src/ops/function.rs:606:13
  24: std::panicking::try::do_call
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/panicking.rs:483:40
  25: std::panicking::try
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/panicking.rs:447:19
  26: std::panic::catch_unwind
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/panic.rs:137:14
  27: std::rt::lang_start_internal::{{closure}}
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/rt.rs:148:48
  28: std::panicking::try::do_call
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/panicking.rs:483:40
  29: std::panicking::try
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/panicking.rs:447:19
  30: std::panic::catch_unwind
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/panic.rs:137:14
  31: std::rt::lang_start_internal
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/rt.rs:148:20
  32: std::rt::lang_start
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/rt.rs:165:17
  33: main
  34: <unknown>
  35: __libc_start_main
  36: _start

MyState::Clone:
===============
   0: <axum_test::MyState as core::clone::Clone>::clone
             at ./src/main.rs:8:58
   1: axum::routing::method_routing::MethodEndpoint<S,B,E>::with_state
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/axum-0.6.10/src/routing/method_routing.rs:1226:58
   2: axum::routing::method_routing::MethodRouter<S,B,E>::with_state
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/axum-0.6.10/src/routing/method_routing.rs:738:18
   3: axum::routing::Router<S,B>::with_state::{{closure}}
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/axum-0.6.10/src/routing/mod.rs:399:48
   4: core::iter::adapters::map::map_fold::{{closure}}
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/core/src/iter/adapters/map.rs:84:28
   5: core::iter::traits::iterator::Iterator::fold
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/core/src/iter/traits/iterator.rs:2414:21
   6: <core::iter::adapters::map::Map<I,F> as core::iter::traits::iterator::Iterator>::fold
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/core/src/iter/adapters/map.rs:124:9
   7: core::iter::traits::iterator::Iterator::for_each
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/core/src/iter/traits/iterator.rs:831:9
   8: <hashbrown::map::HashMap<K,V,S,A> as core::iter::traits::collect::Extend<(K,V)>>::extend
             at /cargo/registry/src/github.com-1ecc6299db9ec823/hashbrown-0.12.3/src/map.rs:6407:9
   9: <std::collections::hash::map::HashMap<K,V,S> as core::iter::traits::collect::Extend<(K,V)>>::extend
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/collections/hash/map.rs:3037:9
  10: <std::collections::hash::map::HashMap<K,V,S> as core::iter::traits::collect::FromIterator<(K,V)>>::from_iter
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/collections/hash/map.rs:3022:9
  11: core::iter::traits::iterator::Iterator::collect
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/core/src/iter/traits/iterator.rs:1836:9
  12: axum::routing::Router<S,B>::with_state
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/axum-0.6.10/src/routing/mod.rs:393:22
  13: axum_test::main::{{closure}}
             at ./src/main.rs:18:15
  14: tokio::runtime::park::CachedParkThread::block_on::{{closure}}
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/park.rs:283:63
  15: tokio::runtime::coop::with_budget
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/coop.rs:107:5
  16: tokio::runtime::coop::budget
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/coop.rs:73:5
  17: tokio::runtime::park::CachedParkThread::block_on
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/park.rs:283:31
  18: tokio::runtime::context::BlockingRegionGuard::block_on
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/context.rs:315:13
  19: tokio::runtime::scheduler::multi_thread::MultiThread::block_on
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/scheduler/multi_thread/mod.rs:66:9
  20: tokio::runtime::runtime::Runtime::block_on
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/runtime.rs:304:45
  21: axum_test::main
             at ./src/main.rs:23:5
  22: core::ops::function::FnOnce::call_once
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/core/src/ops/function.rs:507:5
  23: std::sys_common::backtrace::__rust_begin_short_backtrace
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/sys_common/backtrace.rs:121:18
  24: std::rt::lang_start::{{closure}}
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/rt.rs:166:18
  25: core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &F>::call_once
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/core/src/ops/function.rs:606:13
  26: std::panicking::try::do_call
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/panicking.rs:483:40
  27: std::panicking::try
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/panicking.rs:447:19
  28: std::panic::catch_unwind
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/panic.rs:137:14
  29: std::rt::lang_start_internal::{{closure}}
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/rt.rs:148:48
  30: std::panicking::try::do_call
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/panicking.rs:483:40
  31: std::panicking::try
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/panicking.rs:447:19
  32: std::panic::catch_unwind
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/panic.rs:137:14
  33: std::rt::lang_start_internal
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/rt.rs:148:20
  34: std::rt::lang_start
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/rt.rs:165:17
  35: main
  36: <unknown>
  37: __libc_start_main
  38: _start

listening on 127.0.0.1:3000
MyState::Clone:
===============
   0: <axum_test::MyState as core::clone::Clone>::clone
             at ./src/main.rs:8:58
   1: <axum::handler::service::HandlerService<H,T,S,B> as core::clone::Clone>::clone
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/axum-0.6.10/src/handler/service.rs:143:20
   2: <tower::util::map_response::MapResponse<S,F> as core::clone::Clone>::clone
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tower-0.4.13/src/util/map_response.rs:12:5
   3: <tower::util::map_future::MapFuture<S,F> as core::clone::Clone>::clone
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tower-0.4.13/src/util/map_future.rs:14:5
   4: <T as tower::util::boxed_clone::CloneService<R>>::clone_box
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tower-0.4.13/src/util/boxed_clone.rs:128:18
   5: <tower::util::boxed_clone::BoxCloneService<T,U,E> as core::clone::Clone>::clone
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tower-0.4.13/src/util/boxed_clone.rs:107:14
   6: <axum::routing::route::Route<B,E> as core::clone::Clone>::clone
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/axum-0.6.10/src/routing/route.rs:73:14
   7: <axum::routing::method_routing::MethodEndpoint<S,B,E> as core::clone::Clone>::clone
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/axum-0.6.10/src/routing/method_routing.rs:1236:47
   8: <axum::routing::method_routing::MethodRouter<S,B,E> as core::clone::Clone>::clone
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/axum-0.6.10/src/routing/method_routing.rs:1163:18
   9: <axum::routing::Endpoint<S,B> as core::clone::Clone>::clone
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/axum-0.6.10/src/routing/mod.rs:717:61
  10: core::clone::Clone::clone
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/core/src/clone.rs:123:5
  11: hashbrown::raw::RawTable<T,A>::clone_from_impl
             at /cargo/registry/src/github.com-1ecc6299db9ec823/hashbrown-0.12.3/src/raw/mod.rs:1735:22
  12: <hashbrown::raw::RawTable<T,A> as hashbrown::raw::RawTableClone>::clone_from_spec
             at /cargo/registry/src/github.com-1ecc6299db9ec823/hashbrown-0.12.3/src/raw/mod.rs:1685:13
  13: <hashbrown::raw::RawTable<T,A> as core::clone::Clone>::clone
             at /cargo/registry/src/github.com-1ecc6299db9ec823/hashbrown-0.12.3/src/raw/mod.rs:1622:17
  14: <hashbrown::map::HashMap<K,V,S,A> as core::clone::Clone>::clone
             at /cargo/registry/src/github.com-1ecc6299db9ec823/hashbrown-0.12.3/src/map.rs:197:20
  15: <std::collections::hash::map::HashMap<K,V,S> as core::clone::Clone>::clone
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/collections/hash/map.rs:1267:22
  16: <axum::routing::Router<S,B> as core::clone::Clone>::clone
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/axum-0.6.10/src/routing/mod.rs:62:21
  17: <axum::routing::into_make_service::IntoMakeService<S> as tower_service::Service<T>>::call
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/axum-0.6.10/src/routing/into_make_service.rs:36:45
  18: <T as hyper::service::make::MakeServiceRef<Target,IB>>::make_service_ref
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/hyper-0.14.24/src/service/make.rs:89:9
  19: hyper::server::server::Server<I,S,E>::poll_next_
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/hyper-0.14.24/src/server/server.rs:178:27
  20: hyper::server::server::Server<I,S,E>::poll_watch
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/hyper-0.14.24/src/server/server.rs:199:46
  21: <hyper::server::server::Server<I,S,E> as core::future::future::Future>::poll
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/hyper-0.14.24/src/server/server.rs:225:9
  22: axum_test::main::{{closure}}
             at ./src/main.rs:25:9
  23: tokio::runtime::park::CachedParkThread::block_on::{{closure}}
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/park.rs:283:63
  24: tokio::runtime::coop::with_budget
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/coop.rs:107:5
  25: tokio::runtime::coop::budget
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/coop.rs:73:5
  26: tokio::runtime::park::CachedParkThread::block_on
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/park.rs:283:31
  27: tokio::runtime::context::BlockingRegionGuard::block_on
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/context.rs:315:13
  28: tokio::runtime::scheduler::multi_thread::MultiThread::block_on
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/scheduler/multi_thread/mod.rs:66:9
  29: tokio::runtime::runtime::Runtime::block_on
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/runtime.rs:304:45
  30: axum_test::main
             at ./src/main.rs:23:5
  31: core::ops::function::FnOnce::call_once
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/core/src/ops/function.rs:507:5
  32: std::sys_common::backtrace::__rust_begin_short_backtrace
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/sys_common/backtrace.rs:121:18
  33: std::rt::lang_start::{{closure}}
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/rt.rs:166:18
  34: core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &F>::call_once
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/core/src/ops/function.rs:606:13
  35: std::panicking::try::do_call
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/panicking.rs:483:40
  36: std::panicking::try
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/panicking.rs:447:19
  37: std::panic::catch_unwind
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/panic.rs:137:14
  38: std::rt::lang_start_internal::{{closure}}
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/rt.rs:148:48
  39: std::panicking::try::do_call
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/panicking.rs:483:40
  40: std::panicking::try
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/panicking.rs:447:19
  41: std::panic::catch_unwind
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/panic.rs:137:14
  42: std::rt::lang_start_internal
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/rt.rs:148:20
  43: std::rt::lang_start
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/rt.rs:165:17
  44: main
  45: <unknown>
  46: __libc_start_main
  47: _start

MyState::Clone:
===============
   0: <axum_test::MyState as core::clone::Clone>::clone
             at ./src/main.rs:8:58
   1: <axum::handler::service::HandlerService<H,T,S,B> as core::clone::Clone>::clone
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/axum-0.6.10/src/handler/service.rs:143:20
   2: <tower::util::map_response::MapResponse<S,F> as core::clone::Clone>::clone
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tower-0.4.13/src/util/map_response.rs:12:5
   3: <tower::util::map_future::MapFuture<S,F> as core::clone::Clone>::clone
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tower-0.4.13/src/util/map_future.rs:14:5
   4: <T as tower::util::boxed_clone::CloneService<R>>::clone_box
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tower-0.4.13/src/util/boxed_clone.rs:128:18
   5: <tower::util::boxed_clone::BoxCloneService<T,U,E> as core::clone::Clone>::clone
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tower-0.4.13/src/util/boxed_clone.rs:107:14
   6: <axum::routing::route::Route<B,E> as core::clone::Clone>::clone
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/axum-0.6.10/src/routing/route.rs:73:14
   7: <axum::routing::method_routing::MethodEndpoint<S,B,E> as core::clone::Clone>::clone
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/axum-0.6.10/src/routing/method_routing.rs:1236:47
   8: <axum::routing::method_routing::MethodRouter<S,B,E> as core::clone::Clone>::clone
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/axum-0.6.10/src/routing/method_routing.rs:1163:18
   9: <axum::routing::Endpoint<S,B> as core::clone::Clone>::clone
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/axum-0.6.10/src/routing/mod.rs:717:61
  10: axum::routing::Router<S,B>::call_route
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/axum-0.6.10/src/routing/mod.rs:492:23
  11: axum::routing::Router<S,B>::call_with_state
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/axum-0.6.10/src/routing/mod.rs:452:17
  12: <axum::routing::Router<(),B> as tower_service::Service<http::request::Request<B>>>::call
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/axum-0.6.10/src/routing/mod.rs:573:9
  13: <T as hyper::service::http::HttpService<B1>>::call
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/hyper-0.14.24/src/service/http.rs:45:9
  14: <hyper::proto::h1::dispatch::Server<S,hyper::body::body::Body> as hyper::proto::h1::dispatch::Dispatch>::recv_msg
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/hyper-0.14.24/src/proto/h1/dispatch.rs:517:23
  15: hyper::proto::h1::dispatch::Dispatcher<D,Bs,I,T>::poll_read_head
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/hyper-0.14.24/src/proto/h1/dispatch.rs:264:17
  16: hyper::proto::h1::dispatch::Dispatcher<D,Bs,I,T>::poll_read
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/hyper-0.14.24/src/proto/h1/dispatch.rs:188:24
  17: hyper::proto::h1::dispatch::Dispatcher<D,Bs,I,T>::poll_loop
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/hyper-0.14.24/src/proto/h1/dispatch.rs:160:21
  18: hyper::proto::h1::dispatch::Dispatcher<D,Bs,I,T>::poll_inner
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/hyper-0.14.24/src/proto/h1/dispatch.rs:137:16
  19: hyper::proto::h1::dispatch::Dispatcher<D,Bs,I,T>::poll_catch
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/hyper-0.14.24/src/proto/h1/dispatch.rs:120:28
  20: <hyper::proto::h1::dispatch::Dispatcher<D,Bs,I,T> as core::future::future::Future>::poll
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/hyper-0.14.24/src/proto/h1/dispatch.rs:424:9
  21: <hyper::server::conn::ProtoServer<T,B,S,E> as core::future::future::Future>::poll
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/hyper-0.14.24/src/server/conn.rs:952:47
  22: <hyper::server::conn::upgrades::UpgradeableConnection<I,S,E> as core::future::future::Future>::poll
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/hyper-0.14.24/src/server/conn.rs:1012:30
  23: <hyper::server::server::new_svc::NewSvcTask<I,N,S,E,W> as core::future::future::Future>::poll
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/hyper-0.14.24/src/server/server.rs:751:36
  24: tokio::runtime::task::core::Core<T,S>::poll::{{closure}}
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/task/core.rs:223:17
  25: tokio::loom::std::unsafe_cell::UnsafeCell<T>::with_mut
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/loom/std/unsafe_cell.rs:14:9
  26: tokio::runtime::task::core::Core<T,S>::poll
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/task/core.rs:212:13
  27: tokio::runtime::task::harness::poll_future::{{closure}}
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/task/harness.rs:476:19
  28: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/core/src/panic/unwind_safe.rs:271:9
  29: std::panicking::try::do_call
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/panicking.rs:483:40
  30: __rust_try
  31: std::panicking::try
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/panicking.rs:447:19
  32: std::panic::catch_unwind
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/panic.rs:137:14
  33: tokio::runtime::task::harness::poll_future
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/task/harness.rs:464:18
  34: tokio::runtime::task::harness::Harness<T,S>::poll_inner
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/task/harness.rs:198:27
  35: tokio::runtime::task::harness::Harness<T,S>::poll
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/task/harness.rs:152:15
  36: tokio::runtime::task::raw::poll
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/task/raw.rs:255:5
  37: tokio::runtime::task::raw::RawTask::poll
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/task/raw.rs:200:18
  38: tokio::runtime::task::LocalNotified<S>::run
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/task/mod.rs:394:9
  39: tokio::runtime::scheduler::multi_thread::worker::Context::run_task::{{closure}}
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/scheduler/multi_thread/worker.rs:464:13
  40: tokio::runtime::coop::with_budget
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/coop.rs:107:5
  41: tokio::runtime::coop::budget
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/coop.rs:73:5
  42: tokio::runtime::scheduler::multi_thread::worker::Context::run_task
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/scheduler/multi_thread/worker.rs:463:9
  43: tokio::runtime::scheduler::multi_thread::worker::Context::run
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/scheduler/multi_thread/worker.rs:426:24
  44: tokio::runtime::scheduler::multi_thread::worker::run::{{closure}}
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/scheduler/multi_thread/worker.rs:406:17
  45: tokio::macros::scoped_tls::ScopedKey<T>::set
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/macros/scoped_tls.rs:61:9
  46: tokio::runtime::scheduler::multi_thread::worker::run
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/scheduler/multi_thread/worker.rs:403:5
  47: tokio::runtime::scheduler::multi_thread::worker::Launch::launch::{{closure}}
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/scheduler/multi_thread/worker.rs:365:45
  48: <tokio::runtime::blocking::task::BlockingTask<T> as core::future::future::Future>::poll
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/blocking/task.rs:42:21
  49: tokio::runtime::task::core::Core<T,S>::poll::{{closure}}
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/task/core.rs:223:17
  50: tokio::loom::std::unsafe_cell::UnsafeCell<T>::with_mut
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/loom/std/unsafe_cell.rs:14:9
  51: tokio::runtime::task::core::Core<T,S>::poll
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/task/core.rs:212:13
  52: tokio::runtime::task::harness::poll_future::{{closure}}
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/task/harness.rs:476:19
  53: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/core/src/panic/unwind_safe.rs:271:9
  54: std::panicking::try::do_call
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/panicking.rs:483:40
  55: __rust_try
  56: std::panicking::try
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/panicking.rs:447:19
  57: std::panic::catch_unwind
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/panic.rs:137:14
  58: tokio::runtime::task::harness::poll_future
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/task/harness.rs:464:18
  59: tokio::runtime::task::harness::Harness<T,S>::poll_inner
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/task/harness.rs:198:27
  60: tokio::runtime::task::harness::Harness<T,S>::poll
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/task/harness.rs:152:15
  61: tokio::runtime::task::raw::poll
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/task/raw.rs:255:5
  62: tokio::runtime::task::raw::RawTask::poll
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/task/raw.rs:200:18
  63: tokio::runtime::task::UnownedTask<S>::run
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/task/mod.rs:431:9
  64: tokio::runtime::blocking::pool::Task::run
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/blocking/pool.rs:159:9
  65: tokio::runtime::blocking::pool::Inner::run
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/blocking/pool.rs:511:17
  66: tokio::runtime::blocking::pool::Spawner::spawn_thread::{{closure}}
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/blocking/pool.rs:469:13
  67: std::sys_common::backtrace::__rust_begin_short_backtrace
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/sys_common/backtrace.rs:121:18
  68: std::thread::Builder::spawn_unchecked_::{{closure}}::{{closure}}
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/thread/mod.rs:550:17
  69: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/core/src/panic/unwind_safe.rs:271:9
  70: std::panicking::try::do_call
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/panicking.rs:483:40
  71: __rust_try
  72: std::panicking::try
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/panicking.rs:447:19
  73: std::panic::catch_unwind
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/panic.rs:137:14
  74: std::thread::Builder::spawn_unchecked_::{{closure}}
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/thread/mod.rs:549:30
  75: core::ops::function::FnOnce::call_once{{vtable.shim}}
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/core/src/ops/function.rs:507:5
  76: <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/alloc/src/boxed.rs:2000:9
  77: <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/alloc/src/boxed.rs:2000:9
  78: std::sys::unix::thread::Thread::new::thread_start
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/sys/unix/thread.rs:108:17
  79: <unknown>
  80: <unknown>

MyState::Clone:
===============
   0: <axum_test::MyState as core::clone::Clone>::clone
             at ./src/main.rs:8:58
   1: <axum::handler::service::HandlerService<H,T,S,B> as core::clone::Clone>::clone
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/axum-0.6.10/src/handler/service.rs:143:20
   2: <tower::util::map_response::MapResponse<S,F> as core::clone::Clone>::clone
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tower-0.4.13/src/util/map_response.rs:12:5
   3: <tower::util::map_future::MapFuture<S,F> as core::clone::Clone>::clone
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tower-0.4.13/src/util/map_future.rs:14:5
   4: <T as tower::util::boxed_clone::CloneService<R>>::clone_box
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tower-0.4.13/src/util/boxed_clone.rs:128:18
   5: <tower::util::boxed_clone::BoxCloneService<T,U,E> as core::clone::Clone>::clone
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tower-0.4.13/src/util/boxed_clone.rs:107:14
   6: axum::routing::route::Route<B,E>::oneshot_inner
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/axum-0.6.10/src/routing/route.rs:48:9
   7: axum::routing::method_routing::MethodRouter<S,B,E>::call_with_state
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/axum-0.6.10/src/routing/method_routing.rs:1114:9
   8: axum::routing::Router<S,B>::call_route
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/axum-0.6.10/src/routing/mod.rs:499:58
   9: axum::routing::Router<S,B>::call_with_state
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/axum-0.6.10/src/routing/mod.rs:452:17
  10: <axum::routing::Router<(),B> as tower_service::Service<http::request::Request<B>>>::call
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/axum-0.6.10/src/routing/mod.rs:573:9
  11: <T as hyper::service::http::HttpService<B1>>::call
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/hyper-0.14.24/src/service/http.rs:45:9
  12: <hyper::proto::h1::dispatch::Server<S,hyper::body::body::Body> as hyper::proto::h1::dispatch::Dispatch>::recv_msg
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/hyper-0.14.24/src/proto/h1/dispatch.rs:517:23
  13: hyper::proto::h1::dispatch::Dispatcher<D,Bs,I,T>::poll_read_head
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/hyper-0.14.24/src/proto/h1/dispatch.rs:264:17
  14: hyper::proto::h1::dispatch::Dispatcher<D,Bs,I,T>::poll_read
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/hyper-0.14.24/src/proto/h1/dispatch.rs:188:24
  15: hyper::proto::h1::dispatch::Dispatcher<D,Bs,I,T>::poll_loop
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/hyper-0.14.24/src/proto/h1/dispatch.rs:160:21
  16: hyper::proto::h1::dispatch::Dispatcher<D,Bs,I,T>::poll_inner
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/hyper-0.14.24/src/proto/h1/dispatch.rs:137:16
  17: hyper::proto::h1::dispatch::Dispatcher<D,Bs,I,T>::poll_catch
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/hyper-0.14.24/src/proto/h1/dispatch.rs:120:28
  18: <hyper::proto::h1::dispatch::Dispatcher<D,Bs,I,T> as core::future::future::Future>::poll
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/hyper-0.14.24/src/proto/h1/dispatch.rs:424:9
  19: <hyper::server::conn::ProtoServer<T,B,S,E> as core::future::future::Future>::poll
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/hyper-0.14.24/src/server/conn.rs:952:47
  20: <hyper::server::conn::upgrades::UpgradeableConnection<I,S,E> as core::future::future::Future>::poll
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/hyper-0.14.24/src/server/conn.rs:1012:30
  21: <hyper::server::server::new_svc::NewSvcTask<I,N,S,E,W> as core::future::future::Future>::poll
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/hyper-0.14.24/src/server/server.rs:751:36
  22: tokio::runtime::task::core::Core<T,S>::poll::{{closure}}
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/task/core.rs:223:17
  23: tokio::loom::std::unsafe_cell::UnsafeCell<T>::with_mut
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/loom/std/unsafe_cell.rs:14:9
  24: tokio::runtime::task::core::Core<T,S>::poll
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/task/core.rs:212:13
  25: tokio::runtime::task::harness::poll_future::{{closure}}
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/task/harness.rs:476:19
  26: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/core/src/panic/unwind_safe.rs:271:9
  27: std::panicking::try::do_call
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/panicking.rs:483:40
  28: __rust_try
  29: std::panicking::try
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/panicking.rs:447:19
  30: std::panic::catch_unwind
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/panic.rs:137:14
  31: tokio::runtime::task::harness::poll_future
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/task/harness.rs:464:18
  32: tokio::runtime::task::harness::Harness<T,S>::poll_inner
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/task/harness.rs:198:27
  33: tokio::runtime::task::harness::Harness<T,S>::poll
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/task/harness.rs:152:15
  34: tokio::runtime::task::raw::poll
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/task/raw.rs:255:5
  35: tokio::runtime::task::raw::RawTask::poll
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/task/raw.rs:200:18
  36: tokio::runtime::task::LocalNotified<S>::run
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/task/mod.rs:394:9
  37: tokio::runtime::scheduler::multi_thread::worker::Context::run_task::{{closure}}
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/scheduler/multi_thread/worker.rs:464:13
  38: tokio::runtime::coop::with_budget
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/coop.rs:107:5
  39: tokio::runtime::coop::budget
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/coop.rs:73:5
  40: tokio::runtime::scheduler::multi_thread::worker::Context::run_task
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/scheduler/multi_thread/worker.rs:463:9
  41: tokio::runtime::scheduler::multi_thread::worker::Context::run
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/scheduler/multi_thread/worker.rs:426:24
  42: tokio::runtime::scheduler::multi_thread::worker::run::{{closure}}
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/scheduler/multi_thread/worker.rs:406:17
  43: tokio::macros::scoped_tls::ScopedKey<T>::set
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/macros/scoped_tls.rs:61:9
  44: tokio::runtime::scheduler::multi_thread::worker::run
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/scheduler/multi_thread/worker.rs:403:5
  45: tokio::runtime::scheduler::multi_thread::worker::Launch::launch::{{closure}}
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/scheduler/multi_thread/worker.rs:365:45
  46: <tokio::runtime::blocking::task::BlockingTask<T> as core::future::future::Future>::poll
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/blocking/task.rs:42:21
  47: tokio::runtime::task::core::Core<T,S>::poll::{{closure}}
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/task/core.rs:223:17
  48: tokio::loom::std::unsafe_cell::UnsafeCell<T>::with_mut
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/loom/std/unsafe_cell.rs:14:9
  49: tokio::runtime::task::core::Core<T,S>::poll
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/task/core.rs:212:13
  50: tokio::runtime::task::harness::poll_future::{{closure}}
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/task/harness.rs:476:19
  51: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/core/src/panic/unwind_safe.rs:271:9
  52: std::panicking::try::do_call
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/panicking.rs:483:40
  53: __rust_try
  54: std::panicking::try
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/panicking.rs:447:19
  55: std::panic::catch_unwind
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/panic.rs:137:14
  56: tokio::runtime::task::harness::poll_future
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/task/harness.rs:464:18
  57: tokio::runtime::task::harness::Harness<T,S>::poll_inner
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/task/harness.rs:198:27
  58: tokio::runtime::task::harness::Harness<T,S>::poll
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/task/harness.rs:152:15
  59: tokio::runtime::task::raw::poll
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/task/raw.rs:255:5
  60: tokio::runtime::task::raw::RawTask::poll
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/task/raw.rs:200:18
  61: tokio::runtime::task::UnownedTask<S>::run
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/task/mod.rs:431:9
  62: tokio::runtime::blocking::pool::Task::run
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/blocking/pool.rs:159:9
  63: tokio::runtime::blocking::pool::Inner::run
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/blocking/pool.rs:511:17
  64: tokio::runtime::blocking::pool::Spawner::spawn_thread::{{closure}}
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/blocking/pool.rs:469:13
  65: std::sys_common::backtrace::__rust_begin_short_backtrace
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/sys_common/backtrace.rs:121:18
  66: std::thread::Builder::spawn_unchecked_::{{closure}}::{{closure}}
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/thread/mod.rs:550:17
  67: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/core/src/panic/unwind_safe.rs:271:9
  68: std::panicking::try::do_call
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/panicking.rs:483:40
  69: __rust_try
  70: std::panicking::try
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/panicking.rs:447:19
  71: std::panic::catch_unwind
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/panic.rs:137:14
  72: std::thread::Builder::spawn_unchecked_::{{closure}}
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/thread/mod.rs:549:30
  73: core::ops::function::FnOnce::call_once{{vtable.shim}}
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/core/src/ops/function.rs:507:5
  74: <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/alloc/src/boxed.rs:2000:9
  75: <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/alloc/src/boxed.rs:2000:9
  76: std::sys::unix::thread::Thread::new::thread_start
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/sys/unix/thread.rs:108:17
  77: <unknown>
  78: <unknown>

MyState::Clone:
===============
   0: <axum_test::MyState as core::clone::Clone>::clone
             at ./src/main.rs:8:58
   1: <axum::handler::service::HandlerService<H,T,S,B> as tower_service::Service<http::request::Request<B>>>::call
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/axum-0.6.10/src/handler/service.rs:171:50
   2: <tower::util::map_response::MapResponse<S,F> as tower_service::Service<Request>>::call
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tower-0.4.13/src/util/map_response.rs:75:32
   3: <tower::util::map_future::MapFuture<S,F> as tower_service::Service<R>>::call
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tower-0.4.13/src/util/map_future.rs:65:18
   4: <alloc::boxed::Box<S> as tower_service::Service<Request>>::call
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tower-service-0.3.2/src/lib.rs:387:9
   5: <tower::util::boxed_clone::BoxCloneService<T,U,E> as tower_service::Service<T>>::call
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tower-0.4.13/src/util/boxed_clone.rs:101:9
   6: <tower::util::oneshot::Oneshot<S,Req> as core::future::future::Future>::poll
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tower-0.4.13/src/util/oneshot.rs:93:29
   7: <axum::routing::route::RouteFuture<B,E> as core::future::future::Future>::poll
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/axum-0.6.10/src/routing/route.rs:161:61
   8: <hyper::proto::h1::dispatch::Server<S,hyper::body::body::Body> as hyper::proto::h1::dispatch::Dispatch>::poll_msg
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/hyper-0.14.24/src/proto/h1/dispatch.rs:491:35
   9: hyper::proto::h1::dispatch::Dispatcher<D,Bs,I,T>::poll_write
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/hyper-0.14.24/src/proto/h1/dispatch.rs:297:43
  10: hyper::proto::h1::dispatch::Dispatcher<D,Bs,I,T>::poll_loop
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/hyper-0.14.24/src/proto/h1/dispatch.rs:161:21
  11: hyper::proto::h1::dispatch::Dispatcher<D,Bs,I,T>::poll_inner
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/hyper-0.14.24/src/proto/h1/dispatch.rs:137:16
  12: hyper::proto::h1::dispatch::Dispatcher<D,Bs,I,T>::poll_catch
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/hyper-0.14.24/src/proto/h1/dispatch.rs:120:28
  13: <hyper::proto::h1::dispatch::Dispatcher<D,Bs,I,T> as core::future::future::Future>::poll
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/hyper-0.14.24/src/proto/h1/dispatch.rs:424:9
  14: <hyper::server::conn::ProtoServer<T,B,S,E> as core::future::future::Future>::poll
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/hyper-0.14.24/src/server/conn.rs:952:47
  15: <hyper::server::conn::upgrades::UpgradeableConnection<I,S,E> as core::future::future::Future>::poll
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/hyper-0.14.24/src/server/conn.rs:1012:30
  16: <hyper::server::server::new_svc::NewSvcTask<I,N,S,E,W> as core::future::future::Future>::poll
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/hyper-0.14.24/src/server/server.rs:751:36
  17: tokio::runtime::task::core::Core<T,S>::poll::{{closure}}
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/task/core.rs:223:17
  18: tokio::loom::std::unsafe_cell::UnsafeCell<T>::with_mut
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/loom/std/unsafe_cell.rs:14:9
  19: tokio::runtime::task::core::Core<T,S>::poll
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/task/core.rs:212:13
  20: tokio::runtime::task::harness::poll_future::{{closure}}
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/task/harness.rs:476:19
  21: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/core/src/panic/unwind_safe.rs:271:9
  22: std::panicking::try::do_call
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/panicking.rs:483:40
  23: __rust_try
  24: std::panicking::try
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/panicking.rs:447:19
  25: std::panic::catch_unwind
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/panic.rs:137:14
  26: tokio::runtime::task::harness::poll_future
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/task/harness.rs:464:18
  27: tokio::runtime::task::harness::Harness<T,S>::poll_inner
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/task/harness.rs:198:27
  28: tokio::runtime::task::harness::Harness<T,S>::poll
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/task/harness.rs:152:15
  29: tokio::runtime::task::raw::poll
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/task/raw.rs:255:5
  30: tokio::runtime::task::raw::RawTask::poll
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/task/raw.rs:200:18
  31: tokio::runtime::task::LocalNotified<S>::run
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/task/mod.rs:394:9
  32: tokio::runtime::scheduler::multi_thread::worker::Context::run_task::{{closure}}
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/scheduler/multi_thread/worker.rs:464:13
  33: tokio::runtime::coop::with_budget
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/coop.rs:107:5
  34: tokio::runtime::coop::budget
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/coop.rs:73:5
  35: tokio::runtime::scheduler::multi_thread::worker::Context::run_task
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/scheduler/multi_thread/worker.rs:463:9
  36: tokio::runtime::scheduler::multi_thread::worker::Context::run
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/scheduler/multi_thread/worker.rs:426:24
  37: tokio::runtime::scheduler::multi_thread::worker::run::{{closure}}
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/scheduler/multi_thread/worker.rs:406:17
  38: tokio::macros::scoped_tls::ScopedKey<T>::set
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/macros/scoped_tls.rs:61:9
  39: tokio::runtime::scheduler::multi_thread::worker::run
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/scheduler/multi_thread/worker.rs:403:5
  40: tokio::runtime::scheduler::multi_thread::worker::Launch::launch::{{closure}}
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/scheduler/multi_thread/worker.rs:365:45
  41: <tokio::runtime::blocking::task::BlockingTask<T> as core::future::future::Future>::poll
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/blocking/task.rs:42:21
  42: tokio::runtime::task::core::Core<T,S>::poll::{{closure}}
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/task/core.rs:223:17
  43: tokio::loom::std::unsafe_cell::UnsafeCell<T>::with_mut
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/loom/std/unsafe_cell.rs:14:9
  44: tokio::runtime::task::core::Core<T,S>::poll
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/task/core.rs:212:13
  45: tokio::runtime::task::harness::poll_future::{{closure}}
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/task/harness.rs:476:19
  46: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/core/src/panic/unwind_safe.rs:271:9
  47: std::panicking::try::do_call
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/panicking.rs:483:40
  48: __rust_try
  49: std::panicking::try
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/panicking.rs:447:19
  50: std::panic::catch_unwind
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/panic.rs:137:14
  51: tokio::runtime::task::harness::poll_future
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/task/harness.rs:464:18
  52: tokio::runtime::task::harness::Harness<T,S>::poll_inner
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/task/harness.rs:198:27
  53: tokio::runtime::task::harness::Harness<T,S>::poll
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/task/harness.rs:152:15
  54: tokio::runtime::task::raw::poll
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/task/raw.rs:255:5
  55: tokio::runtime::task::raw::RawTask::poll
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/task/raw.rs:200:18
  56: tokio::runtime::task::UnownedTask<S>::run
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/task/mod.rs:431:9
  57: tokio::runtime::blocking::pool::Task::run
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/blocking/pool.rs:159:9
  58: tokio::runtime::blocking::pool::Inner::run
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/blocking/pool.rs:511:17
  59: tokio::runtime::blocking::pool::Spawner::spawn_thread::{{closure}}
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/blocking/pool.rs:469:13
  60: std::sys_common::backtrace::__rust_begin_short_backtrace
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/sys_common/backtrace.rs:121:18
  61: std::thread::Builder::spawn_unchecked_::{{closure}}::{{closure}}
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/thread/mod.rs:550:17
  62: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/core/src/panic/unwind_safe.rs:271:9
  63: std::panicking::try::do_call
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/panicking.rs:483:40
  64: __rust_try
  65: std::panicking::try
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/panicking.rs:447:19
  66: std::panic::catch_unwind
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/panic.rs:137:14
  67: std::thread::Builder::spawn_unchecked_::{{closure}}
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/thread/mod.rs:549:30
  68: core::ops::function::FnOnce::call_once{{vtable.shim}}
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/core/src/ops/function.rs:507:5
  69: <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/alloc/src/boxed.rs:2000:9
  70: <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/alloc/src/boxed.rs:2000:9
  71: std::sys::unix::thread::Thread::new::thread_start
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/sys/unix/thread.rs:108:17
  72: <unknown>
  73: <unknown>

MyState::Clone:
===============
   0: <axum_test::MyState as core::clone::Clone>::clone
             at ./src/main.rs:8:58
   1: <T as axum_core::extract::from_ref::FromRef<T>>::from_ref
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/axum-core-0.3.3/src/extract/from_ref.rs:23:9
   2: <axum::extract::state::State<InnerState> as axum_core::extract::FromRequestParts<OuterState>>::from_request_parts::{{closure}}
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/axum-0.6.10/src/extract/state.rs:361:27
   3: <core::pin::Pin<P> as core::future::future::Future>::poll
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/core/src/future/future.rs:124:9
   4: <T as axum_core::extract::FromRequest<S,B,axum_core::extract::private::ViaParts>>::from_request::{{closure}}
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/axum-core-0.3.3/src/extract/mod.rs:137:52
   5: <core::pin::Pin<P> as core::future::future::Future>::poll
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/core/src/future/future.rs:124:9
   6: <F as axum::handler::Handler<(M,T1),S,B>>::call::{{closure}}
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/axum-0.6.10/src/handler/mod.rs:208:70
   7: <core::pin::Pin<P> as core::future::future::Future>::poll
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/core/src/future/future.rs:124:9
   8: <futures_util::future::future::map::Map<Fut,F> as core::future::future::Future>::poll
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-util-0.3.26/src/future/future/map.rs:55:37
   9: <futures_util::future::future::Map<Fut,F> as core::future::future::Future>::poll
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-util-0.3.26/src/lib.rs:91:13
  10: <axum::handler::future::IntoServiceFuture<F> as core::future::future::Future>::poll
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/axum-0.6.10/src/macros.rs:42:17
  11: <F as futures_core::future::TryFuture>::try_poll
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-core-0.3.26/src/future.rs:82:9
  12: <futures_util::future::try_future::into_future::IntoFuture<Fut> as core::future::future::Future>::poll
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-util-0.3.26/src/future/try_future/into_future.rs:34:9
  13: <futures_util::future::future::map::Map<Fut,F> as core::future::future::Future>::poll
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-util-0.3.26/src/future/future/map.rs:55:37
  14: <futures_util::future::future::Map<Fut,F> as core::future::future::Future>::poll
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-util-0.3.26/src/lib.rs:91:13
  15: <futures_util::future::try_future::MapOk<Fut,F> as core::future::future::Future>::poll
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-util-0.3.26/src/lib.rs:91:13
  16: <tower::util::map_response::MapResponseFuture<F,N> as core::future::future::Future>::poll
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tower-0.4.13/src/macros.rs:38:17
  17: <core::pin::Pin<P> as core::future::future::Future>::poll
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/core/src/future/future.rs:124:9
  18: <tower::util::oneshot::Oneshot<S,Req> as core::future::future::Future>::poll
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tower-0.4.13/src/util/oneshot.rs:97:38
  19: <axum::routing::route::RouteFuture<B,E> as core::future::future::Future>::poll
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/axum-0.6.10/src/routing/route.rs:161:61
  20: <hyper::proto::h1::dispatch::Server<S,hyper::body::body::Body> as hyper::proto::h1::dispatch::Dispatch>::poll_msg
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/hyper-0.14.24/src/proto/h1/dispatch.rs:491:35
  21: hyper::proto::h1::dispatch::Dispatcher<D,Bs,I,T>::poll_write
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/hyper-0.14.24/src/proto/h1/dispatch.rs:297:43
  22: hyper::proto::h1::dispatch::Dispatcher<D,Bs,I,T>::poll_loop
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/hyper-0.14.24/src/proto/h1/dispatch.rs:161:21
  23: hyper::proto::h1::dispatch::Dispatcher<D,Bs,I,T>::poll_inner
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/hyper-0.14.24/src/proto/h1/dispatch.rs:137:16
  24: hyper::proto::h1::dispatch::Dispatcher<D,Bs,I,T>::poll_catch
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/hyper-0.14.24/src/proto/h1/dispatch.rs:120:28
  25: <hyper::proto::h1::dispatch::Dispatcher<D,Bs,I,T> as core::future::future::Future>::poll
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/hyper-0.14.24/src/proto/h1/dispatch.rs:424:9
  26: <hyper::server::conn::ProtoServer<T,B,S,E> as core::future::future::Future>::poll
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/hyper-0.14.24/src/server/conn.rs:952:47
  27: <hyper::server::conn::upgrades::UpgradeableConnection<I,S,E> as core::future::future::Future>::poll
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/hyper-0.14.24/src/server/conn.rs:1012:30
  28: <hyper::server::server::new_svc::NewSvcTask<I,N,S,E,W> as core::future::future::Future>::poll
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/hyper-0.14.24/src/server/server.rs:751:36
  29: tokio::runtime::task::core::Core<T,S>::poll::{{closure}}
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/task/core.rs:223:17
  30: tokio::loom::std::unsafe_cell::UnsafeCell<T>::with_mut
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/loom/std/unsafe_cell.rs:14:9
  31: tokio::runtime::task::core::Core<T,S>::poll
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/task/core.rs:212:13
  32: tokio::runtime::task::harness::poll_future::{{closure}}
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/task/harness.rs:476:19
  33: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/core/src/panic/unwind_safe.rs:271:9
  34: std::panicking::try::do_call
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/panicking.rs:483:40
  35: __rust_try
  36: std::panicking::try
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/panicking.rs:447:19
  37: std::panic::catch_unwind
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/panic.rs:137:14
  38: tokio::runtime::task::harness::poll_future
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/task/harness.rs:464:18
  39: tokio::runtime::task::harness::Harness<T,S>::poll_inner
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/task/harness.rs:198:27
  40: tokio::runtime::task::harness::Harness<T,S>::poll
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/task/harness.rs:152:15
  41: tokio::runtime::task::raw::poll
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/task/raw.rs:255:5
  42: tokio::runtime::task::raw::RawTask::poll
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/task/raw.rs:200:18
  43: tokio::runtime::task::LocalNotified<S>::run
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/task/mod.rs:394:9
  44: tokio::runtime::scheduler::multi_thread::worker::Context::run_task::{{closure}}
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/scheduler/multi_thread/worker.rs:464:13
  45: tokio::runtime::coop::with_budget
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/coop.rs:107:5
  46: tokio::runtime::coop::budget
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/coop.rs:73:5
  47: tokio::runtime::scheduler::multi_thread::worker::Context::run_task
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/scheduler/multi_thread/worker.rs:463:9
  48: tokio::runtime::scheduler::multi_thread::worker::Context::run
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/scheduler/multi_thread/worker.rs:426:24
  49: tokio::runtime::scheduler::multi_thread::worker::run::{{closure}}
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/scheduler/multi_thread/worker.rs:406:17
  50: tokio::macros::scoped_tls::ScopedKey<T>::set
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/macros/scoped_tls.rs:61:9
  51: tokio::runtime::scheduler::multi_thread::worker::run
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/scheduler/multi_thread/worker.rs:403:5
  52: tokio::runtime::scheduler::multi_thread::worker::Launch::launch::{{closure}}
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/scheduler/multi_thread/worker.rs:365:45
  53: <tokio::runtime::blocking::task::BlockingTask<T> as core::future::future::Future>::poll
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/blocking/task.rs:42:21
  54: tokio::runtime::task::core::Core<T,S>::poll::{{closure}}
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/task/core.rs:223:17
  55: tokio::loom::std::unsafe_cell::UnsafeCell<T>::with_mut
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/loom/std/unsafe_cell.rs:14:9
  56: tokio::runtime::task::core::Core<T,S>::poll
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/task/core.rs:212:13
  57: tokio::runtime::task::harness::poll_future::{{closure}}
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/task/harness.rs:476:19
  58: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/core/src/panic/unwind_safe.rs:271:9
  59: std::panicking::try::do_call
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/panicking.rs:483:40
  60: __rust_try
  61: std::panicking::try
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/panicking.rs:447:19
  62: std::panic::catch_unwind
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/panic.rs:137:14
  63: tokio::runtime::task::harness::poll_future
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/task/harness.rs:464:18
  64: tokio::runtime::task::harness::Harness<T,S>::poll_inner
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/task/harness.rs:198:27
  65: tokio::runtime::task::harness::Harness<T,S>::poll
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/task/harness.rs:152:15
  66: tokio::runtime::task::raw::poll
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/task/raw.rs:255:5
  67: tokio::runtime::task::raw::RawTask::poll
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/task/raw.rs:200:18
  68: tokio::runtime::task::UnownedTask<S>::run
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/task/mod.rs:431:9
  69: tokio::runtime::blocking::pool::Task::run
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/blocking/pool.rs:159:9
  70: tokio::runtime::blocking::pool::Inner::run
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/blocking/pool.rs:511:17
  71: tokio::runtime::blocking::pool::Spawner::spawn_thread::{{closure}}
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.26.0/src/runtime/blocking/pool.rs:469:13
  72: std::sys_common::backtrace::__rust_begin_short_backtrace
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/sys_common/backtrace.rs:121:18
  73: std::thread::Builder::spawn_unchecked_::{{closure}}::{{closure}}
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/thread/mod.rs:550:17
  74: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/core/src/panic/unwind_safe.rs:271:9
  75: std::panicking::try::do_call
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/panicking.rs:483:40
  76: __rust_try
  77: std::panicking::try
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/panicking.rs:447:19
  78: std::panic::catch_unwind
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/panic.rs:137:14
  79: std::thread::Builder::spawn_unchecked_::{{closure}}
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/thread/mod.rs:549:30
  80: core::ops::function::FnOnce::call_once{{vtable.shim}}
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/core/src/ops/function.rs:507:5
  81: <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/alloc/src/boxed.rs:2000:9
  82: <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/alloc/src/boxed.rs:2000:9
  83: std::sys::unix::thread::Thread::new::thread_start
             at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/sys/unix/thread.rs:108:17
  84: <unknown>
  85: <unknown>

read route called!
davidpdrsn commented 1 year ago

https://github.com/tokio-rs/axum/pull/1837 will reduce the counts from 5 to 4. Not much but better than nothing. It also includes a list of where the remaining clones happen. Its not clear that possible to fix them but worth investigating if someone is up for it.

ttys3 commented 1 year ago

What's the performance impact of those clones? Has anyone ran benchmarks?

davidpdrsn commented 1 year ago

That depends entirely on which state type you use. If you use an Arc (or things which are Arc'ed internally such as connection pools) which can be cheaply cloned then the impact is negligible.

alexpyattaev commented 1 year ago

I've done some benchmarks... This is for using Arc<Vec> as State, all requests are reading elements from the Vec. At first latency goes down (as we are serving more requests faster) then it starts to go up quite a bit. Replacing Arc with & 'static Vec solves the issue. image

PS: After 27 the CPU's are from hyperthreading, so it may not behave quite the same as with "real" cores. Do not read much into that. The concerning part is from 8 to 27 cores, where latency goes up even though I am giving more resources. Tested with latest stable Axum.

commonsensesoftware commented 10 months ago

Honestly, I'm not sure axum can solve this problem. It's a systemic issue with Clone. Without creating your own crafty setup, using Arc makes something a singleton and implementing Clone makes it transient with the potential cost of additional, unnecessary initializations. I'm not sure it's really possible to do per-request lifetimes with Clone. For simple state - sure - it will have that appearance. For more complex scenarios, not so much.

We clearly need, or at least want, DI in applications. axum provides a facility, but it's not a fully-fledged DI framework and it probably should never provide one. That's not its job. There aren't that many comprehensive DI solutions for Rust, which is why started the more-di project a couple of years back. Now that I'm starting to build web APIs in axum, I found myself running into the same challenges with State. Though I'm still new to axum, it was surprisingly simple to connect DI to axum a la more-di-axum. This approach does not use State and isn't subject to the limitations of Clone.

Reworking the OP:

#[injectable]
struct SharedState;

#[injectable]
struct MyStateA(Ref<SharedState>);

#[injectable]
struct MyStateB(Ref<SharedState>);

// SharedState is registered as 'scoped'; therefore, MyStateA and MyStateB
// hold their own reference count to a single instance per HTTP request
//
// Inject implements Clone so if cloning does occur, it's only cloning the
// reference count (e.g. Arc). instantiation and lifetime is managed by the
// underlying container
async fn read(
    Inject(_a): Inject<MyStateA>,
    Inject(_b): Inject<MyStateB>) -> &'static str {
    println!("handler::read:\n===============\n{}", Backtrace::force_capture());
    "OK"
}

#[tokio::main]
async fn main() {
    let provider = ServiceCollection::new()
        .add(SharedState::scoped())
        .add(MyStateA::transient())
        .add(MyStateB::transient())
        .build_provider()
        .unwrap();

    // build our application with a route
    let app = Router::new().route("/", get(read)).with_provider(provider);

    // run it
    let listener = TcpListener::bind("127.0.0.1:3000").await.unwrap();
    println!("listening on {}", listener.local_addr().unwrap());
    axum::serve(listener, app).await.unwrap();
}

@mcatanzariti , in the event that it is useful or interesting to you, here is a working example:

issue-1827-with-di.zip

alexpyattaev commented 10 months ago

I am not sure how this addresses the issue. The only reason State implies Arc is because it requires the inner struct to be Send+Sync, and also 'static/owned. I could have a State with 3 const integers and would still need to wrap it in Arc to keep borrow checker happy.

Now the real solution that might work, in my opinion, is for State to be owned by Router. This way Router can give shared references to handlers as needed, and once Router is dismantled it Drop the state to clean up. It would effectively achieve the same thing as Box::leak trick, but without the unsafe needed to clean up in the end. Unless I'm missing something obvious =) I can make a PR if the maintainers deem it a viable approach.

davidpdrsn commented 10 months ago

Not fully sure I understand what you mean but I don't think we're looking to change state management in that way. How the state is carried, in a type safe way, from the router into the handlers is already complicated enough and not worth rewriting just to save a few clones of an Arc.

If you need the absolute most performance then you can use other tactics, like Box::leak, yourself.

alexpyattaev commented 10 months ago

I'm happy to Box::leak my state all day =) the only issue I see is that it is unsafe and real easy to get wrong in case of a panic. So if main thread panics you end up with files/sockets behind State object in some sort of danglng state, which is scary if you expect axum process to get autorestarted by systemd (as it may fail to restart due to stuck ports).

Maybe there is some kind of pattern that can produce a "scoped" 'static reference that is somehow bound to the lifetime of the router and that would get correctly dropped during unwinding, but I have no clue what that would look like. Maybe using something a la defer crate?

davidpdrsn commented 10 months ago

I'm happy to Box::leak my state all day =) the only issue I see is that it is unsafe and real easy to get wrong in case of a panic.

Why can't you just use an Arc?

Maybe there is some kind of pattern that can produce a "scoped" 'static reference that is somehow bound to the lifetime of the router and that would get correctly dropped during unwinding, but I have no clue what that would look like. Maybe using something a la defer crate?

Dunno 🤷

commonsensesoftware commented 10 months ago

Maybe there is some kind of pattern that can produce a "scoped" 'static reference that is somehow bound to the lifetime of the router and that would get correctly dropped during unwinding...

@alexpyattaev I hate to shamelessly plug myself, but that is exactly what the example above that I provided does. Transient and singleton services are not constructed until they are used by the route handler, which means you get pay for play without having to craft a bunch of things yourself. Scoped services are bound to the current request. When the request ends, they are dropped automatically. In this context, Arc seems more than a straight forward way to share a reference. I understand if you don't what to do it that way, but it is possible. It's admittedly an overkill solution for something as simple as Arc<Vec>, but most applications are much more complex. If you can live with Arc, I'm happy to provide an example that might be an acceptable solution to you.

alexpyattaev commented 10 months ago

I am probably not explaining the problem with Arc well enough, sorry.

The whole point of not cloning Arc too much is that every Clone on Arc (does not matter which Arc) causes atomic increments, which creates additional broadcast traffic on interconnects between CPU cores, reducing their ability to fetch data from memory to do actual work substantially. This effect is far more pronounced on machines with many cores, as events related to atomic operations need to be delivered to more cores at a time.

Essentially, cloning Arc (or any other atomic operation for that matter) is far more expensive than one would expect by simply benchmarking "how long it takes in criterion" on a single core. In cases where State holds objects like semi-static tables that can be safely read from multiple handlers without a Mutex, the performance impact of Arc becomes non-negligible. If Arc was cloned once per request, it would not be a big problem, but since Axum tends to clone State quite a bit per request, we get into trouble. Thus, whatever solution is proposed, it needs to avoid touching atomics during handling of requests.

In fact, I've noticed that having Arc as a State object actually gets largest performance hit from Arc and not from mutexes within DashMap, as long as ~90% of requests are indeed reads and not writes. This is likely because Arc gets poked a whole bunch of times per request, while DashMap itself is poked only once per request.

davidpdrsn commented 10 months ago

My stance is still that using Arc is totally fine for the vast majority of applications (which are gonna be IO bound anyway) and I don't think we're looking to change axum's state handling to totally avoid atomics.

In fact axum doesn't even require Arc so if you wanna use State<&'static SomeLeakedThing> then you can still do that, with the caveat that you need to manage the clean up yourself. I still think that is the right trade-off.

fwiw https://github.com/tokio-rs/axum/pull/2483 will decrease the number of clones per request by 1, from 5 to 4, and in 0.8 it'll be further decreased to 3 (https://github.com/tokio-rs/axum/pull/2476/files#diff-23bc1aa0df75595e41c45d9ae3c442bd4eedfaf30352c4bf007045a2aeba74bcR954)

alexpyattaev commented 10 months ago

With 3 clones per request it should become a non-issue if it used to be 5 or more. It would push the inflection point on that plot I've measured to ~16 CPUs instead of 8 (assuming it scales linearly), and very few deployments would use >16 cores for one process anyway.