dndx / phantun

Transforms UDP stream into (fake) TCP streams that can go through Layer 3 & Layer 4 (NAPT) firewalls/NATs.
Apache License 2.0
1.59k stars 131 forks source link

phantun always panicked when used in a few days #79

Open lzfmars opened 1 year ago

lzfmars commented 1 year ago

INFO client > New UDP client from 127.0.0.1:15497 INFO fake_tcp > Sent SYN to server INFO fake_tcp > Connection to xxx.xxx.xxx.xxx:xxx established INFO client > New UDP client from 127.0.0.1:63470 thread 'tokio-runtime-worker' panicked at 'assertion failed: tuples.insert(tuple, incoming.clone()).is_none()', /project/fake-tcp/src/lib.rs:425:13 note: run with RUST_BACKTRACE=1 environment variable to display a backtrace thread 'tokio-runtime-worker' panicked at 'called Result::unwrap() on an Err value: PoisonError { .. }', fake-tcp/src/lib.rs:325:44 stack backtrace: 0: 0x56c79d - std::backtrace_rs::backtrace::libunwind::trace::h081201764674ef17 at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/std/src/../../backtrace/src/backtrace/libunwind.rs:93:5 1: 0x56c79d - std::backtrace_rs::backtrace::trace_unsynchronized::hebab37398c391bd7 at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5 2: 0x56c79d - std::sys_common::backtrace::_print_fmt::h301516df68ed24f9 at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/std/src/sys_common/backtrace.rs:66:5 3: 0x56c79d - ::fmt::h8f5170f4f03a12c0 at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/std/src/sys_common/backtrace.rs:45:22 4: 0x5a4cdc - core::fmt::write::h5dc5601e8d9f6367 at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/core/src/fmt/mod.rs:1190:17 5: 0x565e88 - std::io::Write::write_fmt::h5b19302eb99d9acf at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/std/src/io/mod.rs:1657:15 6: 0x56ebe7 - std::sys_common::backtrace::_print::hd81cf53a75c8ae6a at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/std/src/sys_common/backtrace.rs:48:5 7: 0x56ebe7 - std::sys_common::backtrace::print::hb5aa882e87c2a0dc at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/std/src/sys_common/backtrace.rs:35:9 8: 0x56ebe7 - std::panicking::default_hook::{{closure}}::had913369af61b326 at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/std/src/panicking.rs:295:22 9: 0x56e8b0 - std::panicking::default_hook::h37b06af9ee965447 at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/std/src/panicking.rs:314:9 10: 0x56f339 - std::panicking::rust_panic_with_hook::hf2019958d21362cc at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/std/src/panicking.rs:698:17 11: 0x56f027 - std::panicking::begin_panic_handler::{{closure}}::he9c06fdd592f8785 at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/std/src/panicking.rs:588:13 12: 0x56cc64 - std::sys_common::backtrace::__rust_end_short_backtrace::ha521b96560789310 at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/std/src/sys_common/backtrace.rs:138:18 13: 0x56ed39 - rust_begin_unwind at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/std/src/panicking.rs:584:5 14: 0x40c363 - core::panicking::panic_fmt::h28f1697d4e9394b4 at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/core/src/panicking.rs:143:14 15: 0x40c3f3 - core::result::unwrap_failed::hea43ef43a7a8c801 at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/core/src/result.rs:1749:5 16: 0x4c36b6 - ::drop::h1a82aa2a762e6663 17: 0x42769d - core::ptr::drop_in_place::h9642bb5034846e49 18: 0x42d1d6 - client::main::{{closure}}::{{closure}}::hedb1c4db73446abe 19: 0x4108dd - tokio::runtime::task::harness::poll_future::h71e4629bb72ebffa 20: 0x40ec41 - tokio::runtime::task::harness::Harness<T,S>::poll::hd78d455ce817a0c4 21: 0x4fc9f0 - std::thread::local::LocalKey::with::h25d46ec2bd26319e 22: 0x4f3c0d - tokio::runtime::thread_pool::worker::Context::run_task::h89a72a6001d042f4 23: 0x4f2d8f - tokio::runtime::thread_pool::worker::Context::run::h902a2ffa1a9bd051 24: 0x4dea1b - tokio::macros::scoped_tls::ScopedKey::set::h60ccb18cf4ba637e 25: 0x4f280c - tokio::runtime::thread_pool::worker::run::hdaf5742befe30ba0 26: 0x4d80e1 - <tokio::runtime::blocking::task::BlockingTask as core::future::future::Future>::poll::h100ed8d5c3bf1cbf 27: 0x4d9b41 - tokio::runtime::task::harness::Harness<T,S>::poll::hbd3606d2e34d7e90 28: 0x4e2e83 - tokio::runtime::blocking::pool::Inner::run::ha389bc77b977d94f 29: 0x4e096c - std::sys_common::backtrace::__rust_begin_short_backtrace::he288b5cbf780ffc1 30: 0x4e987f - core::ops::function::FnOnce::call_once{{vtable.shim}}::he75549d8b045b827 31: 0x573043 - <alloc::boxed::Box<F,A> as core::ops::function::FnOnce>::call_once::hcf019fa04facec20 at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/alloc/src/boxed.rs:1853:9 32: 0x573043 - <alloc::boxed::Box<F,A> as core::ops::function::FnOnce>::call_once::h5cc542c486f8c3f4 at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/alloc/src/boxed.rs:1853:9 33: 0x573043 - std::sys::unix::thread::Thread::new::thread_start::h5d35ae99cebafa12 at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/std/src/sys/unix/thread.rs:108:17 thread panicked while panicking. aborting.

lzfmars commented 1 year ago

phantun_client side

dndx commented 1 year ago

What kind of client do you have? I have never seen this error before and it will be helpful if you can provide more info.

lzfmars commented 1 year ago

client system run in a kvm,Openwrt 21.02.3 from official. uname -a: Linux OpenWrt 5.4.188 #0 SMP Sat Apr 16 12:59:34 2022 x86_64 GNU/Linux

phantun_client version: 0.5.0,x86_64,musl

start command: /usr/bin/phantun_client -l 127.0.0.1:1081 -r xxx:xxx --tun tun.trojan --tun-local 192.168.100.1 --tun-local6 fcc1::1 --tun-peer 192.168.100.2 --tun-peer6 fcc1::2

use case: trojan-go ---> kcptun ---> phantun_client --------------------> phantun_server ---> kcptun ---> trojan-go

lzfmars commented 1 year ago

And I found ipv6 address is not set on tun in openwrt

mmexport1664929385166
dndx commented 1 year ago

Do you have high number of new UDP connections all the time? There is a small chance that Phantun might reuse an existing port number and that could cause this assertion failure you showed. How often does these New UDP client from logs appear?

lzfmars commented 1 year ago

Do you have high number of new UDP connections all the time? There is a small chance that Phantun might reuse an existing port number and that could cause this assertion failure you showed. How often does these New UDP client from logs appear?

About 2-3 New UDP client from per minute.

Theo-Silfa commented 2 months ago

Hi! Got same stack tonight:

panicked at 'assertion failed: tuples.insert(tuple, incoming.clone()).is_none()', /project/fake-tcp/src/lib.rs:425:13
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
thread 'tokio-runtime-worker' panicked at 'called `Result::unwrap()` on an `Err` value: PoisonError { .. }', fake-tcp/src/lib.rs:325:44
stack backtrace:
   0:     0x5615095c583d - std::backtrace_rs::backtrace::libunwind::trace::h9135f25bc195152c
                               at /rustc/a55dd71d5fb0ec5a6a3a9e8c27b2127ba491ce52/library/std/src/../../backtrace/src/backtrace/libunwind.rs:93:5
   1:     0x5615095c583d - std::backtrace_rs::backtrace::trace_unsynchronized::h015ee85be510df51
                               at /rustc/a55dd71d5fb0ec5a6a3a9e8c27b2127ba491ce52/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5
   2:     0x5615095c583d - std::sys_common::backtrace::_print_fmt::h5fad03caa9652a2c
                               at /rustc/a55dd71d5fb0ec5a6a3a9e8c27b2127ba491ce52/library/std/src/sys_common/backtrace.rs:66:5
   3:     0x5615095c583d - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::h2b42ca28d244e5c7
                               at /rustc/a55dd71d5fb0ec5a6a3a9e8c27b2127ba491ce52/library/std/src/sys_common/backtrace.rs:45:22
   4:     0x5615095e2f8c - core::fmt::write::h401e827d053130ed
                               at /rustc/a55dd71d5fb0ec5a6a3a9e8c27b2127ba491ce52/library/core/src/fmt/mod.rs:1198:17
   5:     0x5615095c0831 - std::io::Write::write_fmt::hffec93268f5cde32
                               at /rustc/a55dd71d5fb0ec5a6a3a9e8c27b2127ba491ce52/library/std/src/io/mod.rs:1672:15
   6:     0x5615095c7005 - std::sys_common::backtrace::_print::h180c4c706ee1d3fb
                               at /rustc/a55dd71d5fb0ec5a6a3a9e8c27b2127ba491ce52/library/std/src/sys_common/backtrace.rs:48:5
   7:     0x5615095c7005 - std::sys_common::backtrace::print::hd0c35d18765761c9
                               at /rustc/a55dd71d5fb0ec5a6a3a9e8c27b2127ba491ce52/library/std/src/sys_common/backtrace.rs:35:9
   8:     0x5615095c7005 - std::panicking::default_hook::{{closure}}::h1f023310983bc730
                               at /rustc/a55dd71d5fb0ec5a6a3a9e8c27b2127ba491ce52/library/std/src/panicking.rs:295:22
   9:     0x5615095c6d21 - std::panicking::default_hook::h188fec3334afd5be
                               at /rustc/a55dd71d5fb0ec5a6a3a9e8c27b2127ba491ce52/library/std/src/panicking.rs:314:9
  10:     0x5615095c7596 - std::panicking::rust_panic_with_hook::hf26e9d4f97b40096
                               at /rustc/a55dd71d5fb0ec5a6a3a9e8c27b2127ba491ce52/library/std/src/panicking.rs:698:17
  11:     0x5615095c7487 - std::panicking::begin_panic_handler::{{closure}}::hfab912107608087a
                               at /rustc/a55dd71d5fb0ec5a6a3a9e8c27b2127ba491ce52/library/std/src/panicking.rs:588:13
  12:     0x5615095c5d34 - std::sys_common::backtrace::__rust_end_short_backtrace::h434b685ce8d9965b
                               at /rustc/a55dd71d5fb0ec5a6a3a9e8c27b2127ba491ce52/library/std/src/sys_common/backtrace.rs:138:18
  13:     0x5615095c71b9 - rust_begin_unwind
                               at /rustc/a55dd71d5fb0ec5a6a3a9e8c27b2127ba491ce52/library/std/src/panicking.rs:584:5
  14:     0x56150945f663 - core::panicking::panic_fmt::ha6dc7f2ab2479463
                               at /rustc/a55dd71d5fb0ec5a6a3a9e8c27b2127ba491ce52/library/core/src/panicking.rs:142:14
  15:     0x56150945f753 - core::result::unwrap_failed::h42ad8e915aa0a906
                               at /rustc/a55dd71d5fb0ec5a6a3a9e8c27b2127ba491ce52/library/core/src/result.rs:1814:5
  16:     0x56150950cae1 - <fake_tcp::Socket as core::ops::drop::Drop>::drop::h19e2e075513180a0
  17:     0x56150947d43d - core::ptr::drop_in_place<fake_tcp::Socket>::ha967b200bc38b151
  18:     0x5615094806e6 - <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll::h03c38917dad1c848
  19:     0x56150946812c - <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once::hc8b45a1f819e7f31
  20:     0x561509462b8a - tokio::runtime::task::harness::Harness<T,S>::poll::hf9aa08af6be609ae
  21:     0x561509552f25 - std::thread::local::LocalKey<T>::with::ha7b80a901bf9e81b
  22:     0x561509546d0d - tokio::runtime::scheduler::multi_thread::worker::Context::run_task::h650b7cb2bf218561
  23:     0x561509545fc0 - tokio::runtime::scheduler::multi_thread::worker::Context::run::hfa733f6a2e0eb109
  24:     0x56150954b487 - tokio::macros::scoped_tls::ScopedKey<T>::set::hc0a769405f79e976
  25:     0x561509545a89 - tokio::runtime::scheduler::multi_thread::worker::run::h7a48bbe89036a0a7
  26:     0x5615095433f1 - <tokio::runtime::blocking::task::BlockingTask<T> as core::future::future::Future>::poll::h4f1627be177f482f
  27:     0x56150952ef09 - tokio::runtime::task::harness::Harness<T,S>::poll::h978af0f4f4ebe3e9
  28:     0x561509542c55 - tokio::runtime::blocking::pool::Inner::run::h781931038d2c6d58
  29:     0x5615095353c2 - std::sys_common::backtrace::__rust_begin_short_backtrace::hddca8cca672b70a3
  30:     0x56150953849f - core::ops::function::FnOnce::call_once{{vtable.shim}}::h3de090dace688788
  31:     0x5615095ca1f3 - <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once::h56d5fc072706762b
                               at /rustc/a55dd71d5fb0ec5a6a3a9e8c27b2127ba491ce52/library/alloc/src/boxed.rs:1935:9
  32:     0x5615095ca1f3 - <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once::h41deef8e33b824bb
                               at /rustc/a55dd71d5fb0ec5a6a3a9e8c27b2127ba491ce52/library/alloc/src/boxed.rs:1935:9
  33:     0x5615095ca1f3 - std::sys::unix::thread::Thread::new::thread_start::ha6436304a1170bba
                               at /rustc/a55dd71d5fb0ec5a6a3a9e8c27b2127ba491ce52/library/std/src/sys/unix/thread.rs:108:17
  34:     0x7f7d2efd4ac3 - <unknown>
  35:     0x7f7d2f066850 - <unknown>
  36:                0x0 - <unknown>
thread panicked while panicking. aborting.
phantun.service: Main process exited, code=killed, status=6/ABRT
phantun.service: Failed with result 'signal'.
phantun.service: Consumed 1h 59min 36.441s CPU time.

OS is the Ubuntu 22.04.4 LTS:

Linux 1893537-sh48322.twc1.net 5.15.0-112-generic #122-Ubuntu SMP Thu May 23 07:48:21 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Client version:

Phantun Client 0.6.0

Start command:

/usr/local/bin/phantun_client --local 0.0.0.0:<localport> --remote <remoteip>:<remoteport>

Usage: I do not use anything special, just redirecting Wireguard traffic to remote server Reproducibility: I've updated to 0.6.0 in April. This crash is a first time in three months.

dndx commented 3 weeks ago

Please give https://github.com/dndx/phantun/pull/162 a try, I think this will address your issue.

If it works well, I will publish a new release with the fix.