wstunnel armv7l crashes on startup (but not on gdb) after it has been running for months

[AlphaJack]

I discovered only today that since a week wstunnel on my server causes a segmentation fault and crashes when launched. It has worked properly for 3 months, but since the 28 of September it is giving me this problem. The server is a Raspberry Pi 2 running Arch Arm (armv7l), the client is a 64 bit Manjaro laptop (wstunnel here works fine). Running gdb --args wstunnel --server --restrictTo=127.0.0.1:yyyyy ws://127.0.0.1:xxxxx, or even time wstunnel make wstunnel run correctly, while simply running wstunnel with or without arguments returns:

[1]    29440 segmentation fault (core dumped)  wstunnel

If it may help, these are the steps I've used to install it back in July:

wget https://github.com/erebe/wstunnel/releases/download/2.0/wstunnel-armv7l
chmod +x wstunnel-armv7l
sudo mv wstunnel-armv7l /usr/bin/wstunnel

This is the service that I use to run it at startup: wstunnel.service

[Unit]
Description=Tunnel WG UDP over websocket
After=network.target

[Service]
Type=simple
User=nobody
ExecStart=/usr/bin/wstunnel --server --restrictTo=127.0.0.1:yyyyy ws://127.0.0.1:xxxxx
Restart=no

[Install]
WantedBy=multi-user.target

This is the journalctl log: journalctl -u wstunnel.service

[...]
-- Reboot --
Sep 23 04:00:09 hostname systemd[1]: Started Tunnel WG UDP over websocket.
Sep 23 04:00:15 hostname wstunnel[326]: WAIT for connection on 127.0.0.1:xxxxx
Sep 23 15:30:15 hostname systemd[1]: Stopping Tunnel WG UDP over websocket...
Sep 23 15:30:15 hostname systemd[1]: wstunnel.service: Succeeded.
Sep 23 15:30:15 hostname systemd[1]: Stopped Tunnel WG UDP over websocket.
-- Reboot --
Sep 23 15:30:50 hostname systemd[1]: Started Tunnel WG UDP over websocket.
Sep 23 15:30:53 hostname wstunnel[272]: WAIT for connection on 127.0.0.1:xxxxx
Sep 25 12:24:20 hostname wstunnel[272]: SENDING datagrammes to 127.0.0.1:yyyyy
Sep 25 13:43:30 hostname wstunnel[272]: CLOSE udp connection to 127.0.0.1:yyyyy
-- Reboot --
Sep 27 09:29:46 hostname systemd[1]: Started Tunnel WG UDP over websocket.
Sep 27 09:29:49 hostname wstunnel[314]: WAIT for connection on 127.0.0.1:xxxxx
Sep 27 14:47:37 hostname wstunnel[314]: SENDING datagrammes to 127.0.0.1:yyyyy
Sep 27 17:25:41 hostname wstunnel[314]: CLOSE udp connection to 127.0.0.1:yyyyy
-- Reboot --
Sep 28 13:33:25 hostname systemd[1]: Started Tunnel WG UDP over websocket.
Sep 28 13:33:31 hostname systemd[1]: wstunnel.service: Main process exited, code=dumped, status=11/SEGV
Sep 28 13:33:31 hostname systemd[1]: wstunnel.service: Failed with result 'core-dump'.
-- Reboot --
Sep 28 20:42:15 hostname systemd[1]: Started Tunnel WG UDP over websocket.
Sep 28 20:42:23 hostname systemd[1]: wstunnel.service: Main process exited, code=dumped, status=11/SEGV
Sep 28 20:42:23 hostname systemd[1]: wstunnel.service: Failed with result 'core-dump'.
-- Reboot --
[...]

Of course I have checked if the file has been altered and no, it has the same hashes of the one I downloaded in July.

[erebe]

lol strange case.

Can you try to install upx and do on the binary upx -d wstunnel. After that, try to start wstunnel with the new binary and let me know if it crashes also

[AlphaJack]

It gives me the same error, even if the files now weighs 27MB.
Running with sudo doesn't help, and neither does journalctl -xe | grep wstunnel

[1]   30070 segmentation fault (core dumped)  ./wstunnel-armv7l

[erebe]

ok, will try to have a look at it this week-end if I have time :)

[erebe]

Well, I tried on an armv7 ubuntu bionic, but sadly for you, wstunnel is working as expected. I will give a try with archlinux, but it will take me time to find armv7 machine and install it. No cloud/hosting provider propose archlinux with armv7 machines :(

[AlphaJack]

Thank you for checking it, however I suppose that something on my system must be in conflict with wstunnel since the 28th of September, the day it stopped working, or maybe since the 27th, when I had last updated my packages before the logs showed that wstunnel was crashing at startup.
After saying this, I have no idea about what is causing this conflict (if this is the culprit), maybe you can help me debugging it

---

EDIT: a temporary fix that seems to work is replacing the "ExecStart" line in my startup script with

ExecStart=gdb -ex run --args /usr/bin/wstunnel --server --restrictTo=127.0.0.1:yyyyyy ws://127.0.0.1:xxxxx

[mghadam]

I have a similar issue on armv7l (raspberry pi 3), wstunnel-armv7l returns Segmentation fault most of the times but also runs sometimes:

root@raspberrypi:~# uname -a
Linux raspberrypi 4.19.75-v7+ #1270 SMP Tue Sep 24 18:45:11 BST 2019 armv7l GNU/Linux
root@raspberrypi:~# /usr/local/bin/wstunnel 
Segmentation fault
root@raspberrypi:~# /usr/local/bin/wstunnel 
Segmentation fault
root@raspberrypi:~# /usr/local/bin/wstunnel 
Segmentation fault
root@raspberrypi:~# /usr/local/bin/wstunnel 
Segmentation fault
root@raspberrypi:~# /usr/local/bin/wstunnel 
Segmentation fault
root@raspberrypi:~# /usr/local/bin/wstunnel 
Segmentation fault
root@raspberrypi:~# /usr/local/bin/wstunnel 
Use the websockets protocol to tunnel {TCP,UDP} traffic
wsTunnelClient <---> wsTunnelServer <---> RemoteHost
Use secure connection (wss://) to bypass proxies

wstunnel [OPTIONS] ws[s]://wstunnelServer[:port]

Client options:
  -L --localToRemote=[BIND:]PORT:HOST:PORT  Listen on local and forwards
                                            traffic from remote
  -D --dynamicToRemote=[BIND:]PORT          Listen on local and dynamically
                                            (with socks5 proxy) forwards
                                            traffic from remote
  -u --udp                                  forward UDP traffic instead of
                                            TCP
     --udpTimeoutSec=INT                    When using udp forwarding,
                                            timeout in seconds after when the
                                            tunnel connection is closed.
                                            Default 30sec, -1 means no timeout
  -p --httpProxy=USER:PASS@HOST:PORT        If set, will use this proxy to
                                            connect to the server
     --upgradePathPrefix=String             Use a specific prefix that will
                                            show up in the http path in the
                                            upgrade request. Useful if you need
                                            to route requests server side but
                                            don't have vhosts
  -h --help                                 Display help message
  -V --version                              Print version information
Server options:
     --server                               Start a server that will forward
                                            traffic for you
  -r --restrictTo=HOST:PORT                 Accept traffic to be forwarded
                                            only to this service
Common options:
  -v --verbose                              Print debug information
  -q --quiet                                Print only errors
root@raspberrypi:~# /usr/local/bin/wstunnel 
Segmentation fault
root@raspberrypi:~# /usr/local/bin/wstunnel 
Segmentation fault
root@raspberrypi:~#

this happens with both the packed and unpacked versions (upx -d wstunnel)

[AlphaJack]

It seems to work again now

[erebe]

Have you changed something ? Or just updating the system made it works again ?

[AlphaJack]

Have you changed something ? Or just updating the system made it works again ?

Today I've just tried to run by chance wstunnel normally instead of launching it via gdb and it just worked. Since October I haven't touched anything regarding wstunnel, I have only updated my machine regularly.

[mghadam]

I have a similar issue on armv7l (raspberry pi 3), wstunnel-armv7l returns Segmentation fault most of the times but also runs sometimes:

root@raspberrypi:~# uname -a
Linux raspberrypi 4.19.75-v7+ #1270 SMP Tue Sep 24 18:45:11 BST 2019 armv7l GNU/Linux
root@raspberrypi:~# /usr/local/bin/wstunnel 
Segmentation fault
root@raspberrypi:~# /usr/local/bin/wstunnel 
Segmentation fault
root@raspberrypi:~# /usr/local/bin/wstunnel 
Segmentation fault
root@raspberrypi:~# /usr/local/bin/wstunnel 
Segmentation fault
root@raspberrypi:~# /usr/local/bin/wstunnel 
Segmentation fault
root@raspberrypi:~# /usr/local/bin/wstunnel 
Segmentation fault
root@raspberrypi:~# /usr/local/bin/wstunnel 
Use the websockets protocol to tunnel {TCP,UDP} traffic
wsTunnelClient <---> wsTunnelServer <---> RemoteHost
Use secure connection (wss://) to bypass proxies

wstunnel [OPTIONS] ws[s]://wstunnelServer[:port]

Client options:
  -L --localToRemote=[BIND:]PORT:HOST:PORT  Listen on local and forwards
                                            traffic from remote
  -D --dynamicToRemote=[BIND:]PORT          Listen on local and dynamically
                                            (with socks5 proxy) forwards
                                            traffic from remote
  -u --udp                                  forward UDP traffic instead of
                                            TCP
     --udpTimeoutSec=INT                    When using udp forwarding,
                                            timeout in seconds after when the
                                            tunnel connection is closed.
                                            Default 30sec, -1 means no timeout
  -p --httpProxy=USER:PASS@HOST:PORT        If set, will use this proxy to
                                            connect to the server
     --upgradePathPrefix=String             Use a specific prefix that will
                                            show up in the http path in the
                                            upgrade request. Useful if you need
                                            to route requests server side but
                                            don't have vhosts
  -h --help                                 Display help message
  -V --version                              Print version information
Server options:
     --server                               Start a server that will forward
                                            traffic for you
  -r --restrictTo=HOST:PORT                 Accept traffic to be forwarded
                                            only to this service
Common options:
  -v --verbose                              Print debug information
  -q --quiet                                Print only errors
root@raspberrypi:~# /usr/local/bin/wstunnel 
Segmentation fault
root@raspberrypi:~# /usr/local/bin/wstunnel 
Segmentation fault
root@raspberrypi:~#

this happens with both the packed and unpacked versions (upx -d wstunnel)

this issue still exists on my armv7l raspberry pi

[mghadam]

I have a similar issue on armv7l (raspberry pi 3), wstunnel-armv7l returns Segmentation fault most of the times but also runs sometimes:

I got a new raspberry 4 today and confirm this issue still exists on raspberry pi. The v7l binary returns Segmentation fault most of the times root@raspbx:~# uname -a Linux raspbx 4.19.97-v7l+ #1294 SMP Thu Jan 30 13:21:14 GMT 2020 armv7l GNU/Linux root@raspbx:~#