search

tobychui / zoraxy

A general purpose HTTP reverse proxy and forwarding tool. Now written in Go!
https://zoraxy.aroz.org
GNU Affero General Public License v3.0
2.91k stars 176 forks source link

[BUG] Stream Proxies are lost when process restarts #320

Open jwgn opened 1 week ago

jwgn commented 1 week ago

Describe the bug I have set up multiple stream proxies. When I restart the Zoraxy process, all of them are gone and I have to re-setup them. It's not a cosmetic issue, they stop working.

To Reproduce Steps to reproduce the behavior:

  1. Create a stream proxy and enable it
  2. Restart Zoraxy
  3. Stream proxy is gone

Expected behavior I expected the settings to not vanish upon restarting the process.

Screenshots

Browser (if it is a bug appears on the UI section of the system):

Host Environment (please complete the following information):

Additional context Add any other context about the problem here.

tobychui commented 1 week ago

圖片

Cannot reproduce on local instance (without docker). I tried to recreate the above bug with the following steps

  1. Create a proxy rule with TCP :8080 to another IP address and port
  2. Start the proxy rule
  3. Check if HTTP connection (HTTP is on top of TCP) is reachable to the other proxied IP (and it did)
  4. Stop Zoraxy with Ctrl + C
  5. Start Zoraxy again
  6. The proxy rule is started following with the Zoraxy startup
  7. Connect to localhost:8080 again and everything seems working

@PassiveLemon could you try to see if this issue is only related to the docker build of Zoraxy?

PassiveLemon commented 1 week ago

I am also unable to recreate this issue. I setup a TCP proxy to forward :6754 to localhost:8005 (my Zoraxy dashboard) and I am still able to access the dashboard over :6754 after restarting the container a few times.

ViperOGrind commented 6 days ago

Confirm. Have the same problem - Zoraxy does not save stream proxy forwarding rules. OS: Armbian 24.8.4 noble Kernel: Linux 5.15.137-legacy-meson-s4t7 Platform: Khadas VIM4 SBC Zoraxy version: 3.1.1r3

Reproduced several times. Steps to reproduce (considering Zoraxy installed according to the guide in /opt/zoraxy folder and systemd unit configured to launch zoraxy withoput script):

  1. Boot SBC, confirm Zoraxy is running
  2. Login to web-panel
  3. Switch to stream proxy rules page
  4. Create stream proxy rules
  5. Enable proxy rules
  6. Reboot platform
  7. Confirm Zoraxy is running
  8. Login to web-panel
  9. Switch to stream proxy rules
  10. Confirm no rules are present.

image

Just now after reboot Zoraxy deny access from other hosts on my network. Moreover - i lost access by ssh. All services return "Connection refused". Including Zoraxy. Systemctl logs show zoraxy uptime-monitor system:error Ping upstream timeout for zoraxy itself. "dial tcp [zoraxy-address:port]: connect: connection refused" Can't make screenshots - connected to the host directly.

[2024-10-20` 14:49:02.614811] [netstat] [system:info] Netstats listener stopped

[2024-10-20 14:49:02.614964] [internal] [system:info] Closing Netstats Listener

[2024-10-20 14:49:02.615025] [internal] [system:info] Shutting down Zoraxy

[2024-10-20 14:49:02.916104] [internal] [system:info] Closing Statistic Collector

[2024-10-20 14:49:02.965165] [internal] [system:info] Stopping mDNS Discoverer (might take a few minutes)

[2024-10-20 14:49:02.965830] [internal] [system:info] Shutting down load balancer

[2024-10-20 14:49:02.965915] [internal] [system:info] Closing Certificates Auto Renewer

[2024-10-20 14:49:02.965992] [internal] [system:info] Cleaning up tmp files

[2024-10-20 14:49:03.126293] [auth] [system:info] Authentication session key loaded from database

[2024-10-20 14:49:08.186103] [internal] [system:info] Failed to load ZeroTier controller API authtoken

[2024-10-20 14:49:08.198085] [internal] [system:info] Starting ACME handler

[2024-10-20 14:49:08.199324] [internal] [system:info] Serving inbound port 443

[2024-10-20 14:49:08.199387] [internal] [system:info] TLS mode enabled. Serving proxxy request with TLS

[2024-10-20 14:49:08.199430] [internal] [system:info] Development mode enabled. Using no-store Cache Control policy

[2024-10-20 14:49:08.199499] [internal] [system:info] Force latest TLS mode enabled. Minimum TLS LS version is set to v1.2

[2024-10-20 14:49:08.199527] [internal] [system:info] Port 80 listener disabled

[2024-10-20 14:49:08.199660] [internal] [system:info] Force HTTPS mode disabled

[2024-10-20 14:49:08.200722] [proxy-config] [system:info] [some_local_site] -> 127.0.0.1:39081 routing rule loaded

[2024-10-20 14:49:08.201000] [proxy-config] [system:info] [some_local_site] -> 127.0.0.1:39080 routing rule loaded

[2024-10-20 14:49:08.201083] [proxy-config] [system:info] [some_local_site] -> 127.0.0.1:9443 routing rule loaded

[2024-10-20 14:49:08.201264] [proxy-config] [system:info] / -> [zoraxy_host_ip]:38000 routing rule loaded

[2024-10-20 14:49:08.201416] [proxy-config] [system:info] [zoraxy_domain_name] -> [zoraxy_host_ip]:38000 routing rule loaded

[2024-10-20 14:49:08.503100] [dprouter] [system:info] Reverse proxy service started in the background (TLS mode)

[2024-10-20 14:49:08.503517] [internal] [system:info] Dynamic Reverse Proxy service started

[2024-10-20 14:49:08.506275] [uptime-monitor] [system:error] Ping upstream timeout. Assume offline: Get "https://[zoraxy_domain_name]:38000": dial tcp [zoraxy_host_ip]:38000: connect: connection refused

[2024-10-20 14:49:08.693320] [internal] [system:info] Uptime Monitor background service started

[2024-10-20 14:49:08.699421] [internal] [system:info] Assigned temporary port:60188

[2024-10-20 14:49:08.699526] [internal] [system:info] Zoraxy started. Visit control panel at http://localhost:38000

[2024-10-20 14:49:38.165230] [internal] [system:info] mDNS Startup scan completed

Have to stop all containers and ssh and zoraxy itself and restart them all.

Have to stop all containers and ssh and zoraxy itself and restart them all.

Didn't help. Installation is totally screwed. Redeployed OS to my SBC.

Almost forgot - SSH access to my SBC is not related to the issue. That was my Windows 11 host problem.

ViperOGrind commented 6 days ago

I can curl zoraxy web-server locally though. It responds "Temporary redirect" even for http requests. Though i configured https only. And it also responds to https requests. Also with "Temporary redirect".

MedPlex98 commented 4 days ago

Can confirm it I use version v.3.1.1 on Docker and my stream proxy entries are randomly deleted after the restart. Sometimes they all stay there and sometimes they are all deleted.

What I also notice is that my connections generally don't work. I enter a stream proxy and yet the port is closed. No matter what I do. Sometimes Zoraxy doesn't even save the variant, whether TCP or UDP.

jwgn commented 4 days ago

I have Zoraxy installed to /opt without docker.

root@zoraxy:~# systemctl cat zoraxy.service
# /etc/systemd/system/zoraxy.service
[Unit]
Description=General purpose request proxy and forwarding tool
After=syslog.target network-online.target

[Service]
ExecStart=/opt/zoraxy/src/./zoraxy
WorkingDirectory=/opt/zoraxy/src/
Restart=always

[Install]
WantedBy=multi-user.target

# /etc/systemd/system/zoraxy.service.d/override.conf
[Service]
ExecStart=
ExecStart=/opt/zoraxy/src/./zoraxy -fastgeoip=true

Immediately after restarting it with systemctl restart zoraxy.service, all stream proxies are lost and I have to re-setup them.

PassiveLemon commented 4 days ago

Since it appears that it happens on both Docker and native installs of Zoraxy, it's most likely from Zoraxy itself, I can't really help much. Have any of you created a brand new config or has your config been updated through releases? Because it happens only after restarting Zoraxy, it makes me think it's something to do with the database.

MedPlex98 commented 4 days ago

I installed Zoraxy yesterday, so it is a new install without updates

ViperOGrind commented 4 days ago

Hello. Mine was a fresh install. Also checked the same version but x86_64 arch at my work - it saves tcp/udp stream proxy rules correctly and they are available even after host reboot. I can't check udp stream proxy rule correct operation - just need to configure some service behind this rule (DNS or maybe NTP). I'll check when ready.

ViperOGrind commented 4 days ago

Ok, some more info: deployed new Zoraxy appliance on my SBC, and configured a udp stream proxy for port 53 on eth interface ip to 127.0.0.1:39053. Noticed there are no config files in /opt/.zoraxy/conf/proxy image

So, service restart did not broke anything this time. After service restart i logged in to web console and my udp stream proxy rule was allright. Then i decided to reboot my SBC aaaand... Bang! No rules... image

My new appliance now has tis config (everything is fresh as a fresh): OS: Ubuntu 24.04.1 LTS Kernel: Linux 5.15.137 Architecture: arm64 Platform: Khadas VIM4 SBC Zoraxy version: 3.1.1r3

ViperOGrind commented 4 days ago

Confirm the same problem with 3.1.1 Arm64 version. Well... 3.1.0 the same.

tomcatcw1980 commented 1 day ago

Hello,

I can confirm that streams vanish after restart. This is not a docker problem. I installed zoraxy natively on ubuntu.

Streams are even not backed up. I used the backup and restore function to migrate from docker to native. All other was restored except the streams.

Greetings.

tobychui commented 1 day ago

Well I cannot reproduce the bug so there isn't much I can do. But I do see there are some places where the implementation can be optimized.

As the added commit above, I have tried to update some stuffs in the development build and moved the stream proxy config store from database based to file based. If anyone is interested, feel free to try build it yourself to see if the problem is fixed.