angt / glorytun

Multipath UDP tunnel
BSD 2-Clause "Simplified" License
675 stars 103 forks source link

Client Shutdown #75

Closed echolimazulu closed 3 years ago

echolimazulu commented 3 years ago

Hello Adrien,

Today I encountered glorytun client crashing twice. Periodically, the glorytun client process exits itself. Is it possible to track the reason for the closure of the process and what could be the reason for what is happening? The client is launched as a regular user with setcap cap_net_admin = ep.

Startup command: glorytun bind 0.0.0.0 to $SERVER_IP keyfile gt.key &

Similar issue: https://github.com/angt/glorytun/issues/61 But I am not using systemd yet. OS Version: Debian 10 (kernel-4.19.126)

angt commented 3 years ago

Hi, Do you use glorytun in OverTheBox/openMPTCProuter ? Are you using the last version 0.3.4 ?

echolimazulu commented 3 years ago

Hi, Do you use glorytun in OverTheBox/openMPTCProuter ? Are you using the last version 0.3.4 ?

No, i use clean Debian 10 installation with implemented kernel 4.19.126-mptcp and net.mptcp.mptcp_enabled=0. Version of Glorytun is 0.3.4 binary build from releases on github.

echolimazulu commented 3 years ago

The strange thing is that it works for several hours (some time), and then it closes itself for no reason.

angt commented 3 years ago

Do you have a coredump or any information ? The only way to stop the client without printing in stderr is to send it a signal. Now we need to know what signal it was..

echolimazulu commented 3 years ago

At the moment, there is no file with information. But if you tell me how to assemble it, I will definitely do it as soon as possible.

angt commented 3 years ago

With the exception of signals that cannot be ignored, we stop the client on: SIGHUP, SIGTERM, SIGQUIT, SIGINT.

angt commented 3 years ago

do you have coredumpctl or something similar ?

angt commented 3 years ago

On debian, I think you can run the systemd-coredump service. Next time, just run coredumpctl list If you see nothing about glorytun, then you need track the process that sends signal to it.

echolimazulu commented 3 years ago

For now, I have installed the systemd-coredump package. Is it enough to just install the package, or is there something else to do while the command prints:

sudo coredumpctl list No coredumps found.

angt commented 3 years ago

This is too late, you need to start/enable the service systemd-coredump. Next time you see glorytun going down you can coredumpctl. If there is an entry for glorytun then you can send it to me by email so that I can find the problem. But, there is no memory allocation while running and the 0.3.4 binary from github is compiled with -fstack-protector-strong, so in the case of a stack overflow there is a (very) good chance that a message will follow the bug.

echolimazulu commented 3 years ago

Maybe we have a preference for kernel.core_pattern? Or startup options for systemd-coredump?

echolimazulu commented 3 years ago

sudo service systemd-coredump@.service status ● systemd-coredump@.service.service - Process Core Dump Loaded: loaded (/lib/systemd/system/systemd-coredump@.service; static; vendor preset: enabled) Active: inactive (dead) Docs: man:systemd-coredump(8)

Oct 29 10:18:23 superrouter systemd[1]: Started Process Core Dump. Oct 29 10:18:23 superrouter systemd-coredump[4361]: Not enough arguments passed by the kernel (0, expected 7).

angt commented 3 years ago

I have no idea about debian, I've never used this OS. But I would expect that if you install and start something it works out of the box.. But yes, maybe you need to tweak debian's systcl about cordeump to make it works ?

echolimazulu commented 3 years ago

Yes, after installing a separate package. The service apparently requires it to be configured in a certain way before starting to be monitored. Would you recommend the startup options for the service systemd-coredump? What OS do you prefer to use for glorytun in which it is most stable?

angt commented 3 years ago

I use archlinux personally but I don't necessarily recommend it because every OS has its advantages and disadvantages. However I can recommend OpenSUSE simply because it is the only distribution that had the will and took the time to distribute glorytun by itself :)

angt commented 3 years ago

If you don't see anything in your logs (dmesg/journald/etc) and no coredump-like files, most likely another process has sent a signal to glorytun.

angt commented 3 years ago

I'm closing, but don't hesitate to reopen with a coredump if the problem comes back.