search

OpenVPN / openvpn

OpenVPN is an open source VPN daemon
http://openvpn.net
Other
10.94k stars 3.01k forks source link

VPN with local HTTP proxy doesn't work on MacOS Sonoma #456

Open vader-sama opened 11 months ago

vader-sama commented 11 months ago

Describe the bug I've tried the same .ovpn config on Windows and it works, I don't know why on my mac it just doesn't. The VPN connects but it just gets restarted constantly. Proxy is valid and working, I can connect to it using system proxy, etc. I've also tried GUI clients like tunnelblick, connect, etc. none of them works either.

Logs:

2023-11-16 00:27:00 add_route_ipv6(::/3 -> :: metric -1) dev utun4
route: writing to routing socket: Network is unreachable
add net ::: gateway utun4: Network is unreachable
2023-11-16 00:27:00 add_route_ipv6(2000::/4 -> :: metric -1) dev utun4
route: writing to routing socket: Network is unreachable
add net 2000::: gateway utun4: Network is unreachable
2023-11-16 00:27:00 add_route_ipv6(3000::/4 -> :: metric -1) dev utun4
route: writing to routing socket: Network is unreachable
add net 3000::: gateway utun4: Network is unreachable
2023-11-16 00:27:00 add_route_ipv6(fc00::/7 -> :: metric -1) dev utun4
route: writing to routing socket: Network is unreachable
add net fc00::: gateway utun4: Network is unreachable
2023-11-16 00:27:00 Initialization Sequence Completed
2023-11-16 00:27:31 Connection reset, restarting [0]
2023-11-16 00:27:31 SIGUSR1[soft,connection-reset] received, process restarting
2023-11-16 00:27:32 TCP/UDP: Preserving recently used remote address: [AF_INET]127.0.0.1:8001
2023-11-16 00:27:32 Attempting to establish TCP connection with [AF_INET]127.0.0.1:8001
2023-11-16 00:27:32 TCP connection established with [AF_INET]127.0.0.1:8001
2023-11-16 00:27:34 TCPv4_CLIENT link local: (not bound)
2023-11-16 00:27:34 TCPv4_CLIENT link remote: [AF_INET]127.0.0.1:8001
2023-11-16 00:27:39 Connection reset, restarting [0]
2023-11-16 00:27:39 SIGUSR1[soft,connection-reset] received, process restarting
2023-11-16 00:27:40 TCP/UDP: Preserving recently used remote address: [AF_INET]127.0.0.1:8001
2023-11-16 00:27:40 Attempting to establish TCP connection with [AF_INET]127.0.0.1:8001
2023-11-16 00:27:40 TCP connection established with [AF_INET]127.0.0.1:8001
2023-11-16 00:27:42 TCPv4_CLIENT link local: (not bound)
2023-11-16 00:27:42 TCPv4_CLIENT link remote: [AF_INET]127.0.0.1:8001
2023-11-16 00:27:42 Connection reset, restarting [-1]

To Reproduce My .ovpn config

dev tun
persist-tun
persist-key
data-ciphers AES-256-GCM:AES-128-GCM:CHACHA20-POLY1305:AES-256-CBC
data-ciphers-fallback AES-256-CBC
auth SHA256
tls-client
client
resolv-retry infinite
remote x.x.x.x 1111 tcp4
nobind
http-proxy 127.0.0.1 8001
auth-user-pass
remote-cert-tls server

<ca>
-----BEGIN CERTIFICATE-----
XXXXXX
-----END CERTIFICATE-----
</ca>
setenv CLIENT_CERT 0
key-direction 1
<tls-auth>
-----BEGIN OpenVPN Static key V1-----
XXXXXX
-----END OpenVPN Static key V1-----
</tls-auth>

I use following command to connect:

sudo openvpn --config ~/path/to/config.ovpn

Expected behavior It should connect to the server and work just like the Windows version

Version information

cron2 commented 11 months ago

The log file is missing most of the important parts - what did it try to ifconfig, and what did the server send. Please include a full log with verb 3.

vader-sama commented 11 months ago

The log file is missing most of the important parts - what did it try to ifconfig, and what did the server send. Please include a full log with verb 3.

Here is the full log:

2023-11-16 01:05:53 OpenVPN 2.6.7 aarch64-apple-darwin23.0.0 [SSL (OpenSSL)] [LZO] [LZ4] [PKCS11] [MH/RECVDA] [AEAD]
2023-11-16 01:05:53 library versions: OpenSSL 3.1.4 24 Oct 2023, LZO 2.10
Enter Auth Username:
Enter Auth Password:
2023-11-16 01:05:58 TCP/UDP: Preserving recently used remote address: [AF_INET]127.0.0.1:8001
2023-11-16 01:05:58 Socket Buffers: R=[131072->131072] S=[131072->131072]
2023-11-16 01:05:58 Attempting to establish TCP connection with [AF_INET]127.0.0.1:8001
2023-11-16 01:05:58 TCP connection established with [AF_INET]127.0.0.1:8001
2023-11-16 01:05:58 Send to HTTP proxy: 'CONNECT x.x.x.x:1111 HTTP/1.0'
2023-11-16 01:05:58 Send to HTTP proxy: 'Host: x.x.x.x'
2023-11-16 01:05:58 HTTP proxy returned: 'HTTP/1.1 200 Connection established'
2023-11-16 01:06:00 TCPv4_CLIENT link local: (not bound)
2023-11-16 01:06:00 TCPv4_CLIENT link remote: [AF_INET]127.0.0.1:8001
2023-11-16 01:06:00 TLS: Initial packet from [AF_INET]127.0.0.1:8001, sid=668b2823 6aa87ec8
2023-11-16 01:06:00 WARNING: this configuration may cache passwords in memory -- use the auth-nocache option to prevent this
2023-11-16 01:06:00 VERIFY OK: depth=1, CN=Hetzner CA, C=FI, ST=Uusimaa, L=Tuusula, O=ADF
2023-11-16 01:06:00 VERIFY KU OK
2023-11-16 01:06:00 Validating certificate extended key usage
2023-11-16 01:06:00 ++ Certificate has EKU (str) TLS Web Server Authentication, expects TLS Web Server Authentication
2023-11-16 01:06:00 VERIFY EKU OK
2023-11-16 01:06:00 VERIFY OK: depth=0, CN=gw-htz.xino.local, C=FI, ST=Uusimaa, L=Tuusula, O=ADF
2023-11-16 01:06:00 Control Channel: TLSv1.3, cipher TLSv1.3 TLS_AES_256_GCM_SHA384, peer certificate: 2048 bits RSA, signature: RSA-SHA256, peer temporary key: 253 bits X25519
2023-11-16 01:06:00 [example.com] Peer Connection Initiated with [AF_INET]127.0.0.1:8001
2023-11-16 01:06:00 TLS: move_session: dest=TM_ACTIVE src=TM_INITIAL reinit_src=1
2023-11-16 01:06:00 TLS: tls_multi_process: initial untrusted session promoted to trusted
2023-11-16 01:06:02 SENT CONTROL [gw-htz.xino.local]: 'PUSH_REQUEST' (status=1)
2023-11-16 01:06:02 PUSH: Received control message: 'PUSH_REPLY,dhcp-option DNS 172.16.11.7,redirect-gateway def1,redirect-gateway ipv6,route 168.119.183.246 255.255.255.255 net_gateway,route 5.182.44.236 255.255.255.255 net_gateway,route 152.89.45.65 255.255.255.255 net_gateway,route 193.151.134.172 255.255.255.255 net_gateway,route 178.62.78.139 255.255.255.255 net_gateway,route 172.16.24.1,topology net30,ping 10,ping-restart 60,ifconfig 172.16.24.10 172.16.24.9,peer-id 1,cipher AES-256-GCM,protocol-flags cc-exit tls-ekm dyn-tls-crypt,tun-mtu 1500'
2023-11-16 01:06:02 WARNING: You have specified redirect-gateway and redirect-private at the same time (or the same option multiple times). This is not well supported and may lead to unexpected results
2023-11-16 01:06:02 OPTIONS IMPORT: --ifconfig/up options modified
2023-11-16 01:06:02 OPTIONS IMPORT: route options modified
2023-11-16 01:06:02 OPTIONS IMPORT: --ip-win32 and/or --dhcp-option options modified
2023-11-16 01:06:02 OPTIONS IMPORT: tun-mtu set to 1500
2023-11-16 01:06:02 GDG6: remote_host_ipv6=n/a
2023-11-16 01:06:02 GDG6: problem writing to routing socket: No such process (errno=3)
2023-11-16 01:06:02 Opened utun device utun4
2023-11-16 01:06:02 /sbin/ifconfig utun4 delete
ifconfig: ioctl (SIOCDIFADDR): Can't assign requested address
2023-11-16 01:06:02 NOTE: Tried to delete pre-existing tun/tap instance -- No Problem if failure
2023-11-16 01:06:02 /sbin/ifconfig utun4 172.16.24.10 172.16.24.9 mtu 1500 netmask 255.255.255.255 up
2023-11-16 01:06:02 /sbin/route add -net 127.0.0.1 192.168.204.124 255.255.255.255
add net 127.0.0.1: gateway 192.168.204.124
2023-11-16 01:06:02 /sbin/route add -net 0.0.0.0 172.16.24.9 128.0.0.0
add net 0.0.0.0: gateway 172.16.24.9
2023-11-16 01:06:02 /sbin/route add -net 128.0.0.0 172.16.24.9 128.0.0.0
add net 128.0.0.0: gateway 172.16.24.9
2023-11-16 01:06:02 /sbin/route add -net 168.119.183.246 192.168.204.124 255.255.255.255
add net 168.119.183.246: gateway 192.168.204.124
2023-11-16 01:06:02 /sbin/route add -net 5.182.44.236 192.168.204.124 255.255.255.255
add net 5.182.44.236: gateway 192.168.204.124
2023-11-16 01:06:02 /sbin/route add -net 152.89.45.65 192.168.204.124 255.255.255.255
add net 152.89.45.65: gateway 192.168.204.124
2023-11-16 01:06:02 /sbin/route add -net 193.151.134.172 192.168.204.124 255.255.255.255
add net 193.151.134.172: gateway 192.168.204.124
2023-11-16 01:06:02 /sbin/route add -net 178.62.78.139 192.168.204.124 255.255.255.255
add net 178.62.78.139: gateway 192.168.204.124
2023-11-16 01:06:02 /sbin/route add -net 172.16.24.1 172.16.24.9 255.255.255.255
add net 172.16.24.1: gateway 172.16.24.9
2023-11-16 01:06:02 WARNING: OpenVPN was configured to add an IPv6 route. However, no IPv6 has been configured for utun4, therefore the route installation may fail or may not work as expected.
2023-11-16 01:06:02 add_route_ipv6(::/3 -> :: metric -1) dev utun4
2023-11-16 01:06:02 /sbin/route add -inet6 :: -prefixlen 3 -iface utun4
route: writing to routing socket: Network is unreachable
add net ::: gateway utun4: Network is unreachable
2023-11-16 01:06:02 add_route_ipv6(2000::/4 -> :: metric -1) dev utun4
2023-11-16 01:06:02 /sbin/route add -inet6 2000:: -prefixlen 4 -iface utun4
route: writing to routing socket: Network is unreachable
add net 2000::: gateway utun4: Network is unreachable
2023-11-16 01:06:02 add_route_ipv6(3000::/4 -> :: metric -1) dev utun4
2023-11-16 01:06:02 /sbin/route add -inet6 3000:: -prefixlen 4 -iface utun4
route: writing to routing socket: Network is unreachable
add net 3000::: gateway utun4: Network is unreachable
2023-11-16 01:06:02 add_route_ipv6(fc00::/7 -> :: metric -1) dev utun4
2023-11-16 01:06:02 /sbin/route add -inet6 fc00:: -prefixlen 7 -iface utun4
route: writing to routing socket: Network is unreachable
add net fc00::: gateway utun4: Network is unreachable
2023-11-16 01:06:02 Initialization Sequence Completed
2023-11-16 01:06:02 Data Channel: cipher 'AES-256-GCM', peer-id: 1
2023-11-16 01:06:02 Timers: ping 10, ping-restart 60
2023-11-16 01:06:02 Protocol options: protocol-flags cc-exit tls-ekm dyn-tls-crypt
2023-11-16 01:07:02 [gw-htz.xino.local] Inactivity timeout (--ping-restart), restarting
2023-11-16 01:07:02 SIGUSR1[soft,ping-restart] received, process restarting
2023-11-16 01:07:02 Restart pause, 1 second(s)
2023-11-16 01:07:03 TCP/UDP: Preserving recently used remote address: [AF_INET]127.0.0.1:8001
2023-11-16 01:07:03 Socket Buffers: R=[131072->131072] S=[131072->131072]
2023-11-16 01:07:03 Attempting to establish TCP connection with [AF_INET]127.0.0.1:8001
2023-11-16 01:07:03 TCP connection established with [AF_INET]127.0.0.1:8001
2023-11-16 01:07:03 Send to HTTP proxy: 'CONNECT x.x.x.x:1111 HTTP/1.0'
2023-11-16 01:07:03 Send to HTTP proxy: 'Host: x.x.x.x'
2023-11-16 01:07:03 HTTP proxy returned: 'HTTP/1.1 200 Connection established'
2023-11-16 01:07:05 TCPv4_CLIENT link local: (not bound)
2023-11-16 01:07:05 TCPv4_CLIENT link remote: [AF_INET]127.0.0.1:8001
cron2 commented 11 months ago

So, the reason for all these errors about IPv6 routing are because your PUSH_REPLY message has redirect-gateway ipv6, but no ifconfig-ipv6 - and putting an IPv6 route to an interface without an IPv6 address is not allowed by MacOS. It's telling you about this...

2023-11-16 01:06:02 WARNING: OpenVPN was configured to add an IPv6 route. However, no IPv6 has been configured for utun4, therefore the route installation may fail or may not work as expected.

besides this, it looks okayish - the connection is established and the log looks good, until ping-timeout hits (which can be mismatch between client and server config). Does the VPN work in this minute (ping to the server IP works, etc) or does it not work at all?

vader-sama commented 11 months ago

So, the reason for all these errors about IPv6 routing are because your PUSH_REPLY message has redirect-gateway ipv6, but no ifconfig-ipv6 - and putting an IPv6 route to an interface without an IPv6 address is not allowed by MacOS. It's telling you about this...

2023-11-16 01:06:02 WARNING: OpenVPN was configured to add an IPv6 route. However, no IPv6 has been configured for utun4, therefore the route installation may fail or may not work as expected.

besides this, it looks okayish - the connection is established and the log looks good, until ping-timeout hits (which can be mismatch between client and server config). Does the VPN work in this minute (ping to the server IP works, etc) or does it not work at all?

No VPN does not work at all, Its like I don't have internet access anymore. But the same config works on a Windows machine

I've also tried OpenVPN connect the error it gives is a bit different:

[Nov 13, 2023, 16:29:54] Transport Error: Transport error on 'x.x.x.x' via HTTP proxy 127.0.0.1:8001 : NETWORK_EOF_ERROR
[Nov 13, 2023, 16:29:54] EVENT: TRANSPORT_ERROR Transport error on 'x.x.x.x' via HTTP proxy 127.0.0.1:8001 : NETWORK_EOF_ERROR
[Nov 13, 2023, 16:29:54] Client terminated, restarting in 5000 ms...
cron2 commented 11 months ago

What sort of proxy is this, running locally on the same machine (127.0.0.1)? Since this will most certainly be different between Windows and MacOS - maybe there are logs on the proxy side showing what is failing?

vader-sama commented 11 months ago

What sort of proxy is this, running locally on the same machine (127.0.0.1)? Since this will most certainly be different between Windows and MacOS - maybe there are logs on the proxy side showing what is failing?

Its a VLESS and its running by Nekoray (I've tried other V2Ray clients as well), Unfortunately there are no logs on the proxy side, Nothing is being sent to the proxy, The app shows proxy is sending some data but It's not receiving anything

cron2 commented 11 months ago

At least 2.x is successfully establishing a session with the server, so something must have been sent through the proxy - TLS handshake and PUSH_REPLY will only be reached if bidirectional communication works, at least for some time.

3.x says "the proxy closed the connection" (EOF = end of file).

selvanair commented 11 months ago

Looks like you are losing the route to the proxy after the tunnel is up. Is there a bypass route to the proxy setup?

As per the logs: remote is 127.0.0.1:8001 proxy is: x.x.x.x:1111

And we set a bypass route like 2023-11-16 01:06:02 /sbin/route add -net 127.0.0.1 192.168.204.124 255.255.255.255 add net 127.0.0.1: gateway 192.168.204.124

Is this 127.0.0.1 a stand-in for the real remote (masked by you?). If so why is it not to the proxy address?

vader-sama commented 11 months ago

Looks like you are losing the route to the proxy after the tunnel is up. Is there a bypass route to the proxy setup?

As per the logs: remote is 127.0.0.1:8001 proxy is: x.x.x.x:1111

And we set a bypass route like 2023-11-16 01:06:02 /sbin/route add -net 127.0.0.1 192.168.204.124 255.255.255.255 add net 127.0.0.1: gateway 192.168.204.124

Is this 127.0.0.1 a stand-in for the real remote (masked by you?). If so why is it not to the proxy address?

I was thinking about this too, I guess it just closes any open connections when the tunnel is up even the proxy connection (In order to re-open connections to go through the tunnel). But if this is the case, The whole http-proxy thing can won't work at all.

Yeah I've changed the actual proxy server to x.x.x.x:1111 for security concerns. I'm using a V2Ray client and expose it using HTTP proxy. The proxy works fine when being set on the system or used for example in FireFox.

selvanair commented 11 months ago

My question was is there a bypass route to the proxy server among those "route add" lines --- I can't figure that from the logs as you have masked the proxy address. If proxy address is uniformly masked as x.x.x.x in the logs, I think no route to it is getting setup -- do not know why.

Can you ping the proxy when the tunnel is up? Also check the routing table.

vader-sama commented 11 months ago

I found the solution, I had to add the proxy route manually using below command:

route add -net PROXY_IP 192.168.1.1 255.255.255.255
ordex commented 11 months ago

I think what @selvanair was alluding to is that this route should be added automatically. So, if this is not happening, something is wrong.

selvanair commented 11 months ago

I think what @selvanair was alluding to is that this route should be added automatically. So, if this is not happening, something is wrong.

Yes, I would think a bypass route to the proxy should automatically get added unless redirect-gateway local is in use. Also adding a bypass route to the actual remote (127.0.0.1 relative to the proxy in this case) looks odd.

ordex commented 11 months ago

I think what @selvanair was alluding to is that this route should be added automatically. So, if this is not happening, something is wrong.

Yes, I would think a bypass route to the proxy should automatically get added unless redirect-gateway local is in use. Also adding a bypass route to the actual remote (127.0.0.1 relative to the proxy in this case) looks odd.

indeed. do we truly need a bypass route for the loopback address? I don't think so..no?

selvanair commented 11 months ago

indeed. do we truly need a bypass route for the loopback address? I don't think so..no?

It's not the local loopback address but the remote address relative to the proxy -- so proxy server's loopback. But, yes, in no case we need such a bypass. Not sure why it appears though.

cron2 commented 11 months ago

Sounds like a bug of the "redirect-gateway " code in combination with HTTP proxy. Should be easy enough to test.

Maybe it's related to having two "redirect-gateway" statements in the PUSH_REPLY, maybe something special-cases 127.0.0.1 in the code.

I do wonder why it supposedly works on Windows with this config - and why it supposedly does not work with Connect either, which is a fully different code base.

selvanair commented 11 months ago

I cannot reproduce this on linux. On using redirect-gateway def1, bypass route is correctly getting set to the proxy (not actual remote). Combination of redirect-gateway and redirect-private did not trigger any issues as well, though I did not try hard.

Could this be something specific to OS-X?

@vader-sama Could you please post the config as well?

Using redirect-gateway local skips the bypass route and then I get "recursive routing.. " in the logs as expected.

The only odd thing I noticed is that if the proxy is on localhost, we do set a bypass route to it. I thought we have code to filter out such addresses from the route list (test_local_addr()).

vader-sama commented 11 months ago

I'm exposing V2Ray connection using a local HTTP proxy server (127.0.0.1:8001) and the V2Ray server ip for example is 185.1.1.1.

OpenVPN adds a route bypass for 127.0.0.1 I guess, But it doesn't bypass the V2Ray ip (185.1.1.1). So in order to make the whole reverse proxy work, I have to manually bypass the V2Ray ip. But somehow it is not required to do anything in a Windows machine. A friend of mine did get this error on a Linux machine (Ubuntu 20) as well, And he also had to bypass the V2Ray IP just like OS-X with command below: 

sudo ip r a 185.1.1.1/32 via 192.168.1.1 dev wlo1

@selvanair I've already posted my .ovpn

It makes sense, OpenVPN can't get the V2ray ip, because I'm giving the local proxy ip to it. I guess the weird part is how its working on Windows?

selvanair commented 11 months ago

@selvanair I've already posted my https://github.com/OpenVPN/openvpn/issues/456#issue-1995567383

I missed that and misread the logs. As per the ovpn

http-proxy 127.0.0.1 8001

So the proxy is indeed on 127.0.0.1 and then bypass route is being set to it (though odd and not needed). Then I'm totally confused: if proxy is on 127.0.0.1, why do you say you fixed it using

route add -net PROXY_IP 192.168.1.1 255.255.255.255

What is the PROXY_IP here? Also why 192.168.1.1 -- your gateway is 192.168.204.124, isn't it? Is the proxy reachable via a different gateway?

vader-sama commented 11 months ago

I've explained in my previous comment, let me explain it better. I've connected to a v2ray server and its ip is 185.1.1.1 for example. I'm able to connect to that by creating an http proxy which is listening on port 8001 on my localhost. So setting the localhost:8001 as my proxy, will tunnel my traffic through 185.1.1.1. Since localhost:8001 is my http proxy, I'm giving it to the openvpn with http-proxy 127.0.0.1 8001. Openvpn creates a tunnel to my localhost proxy, which it self has tunneled to the 185.1.1.1. But when vpn connects, the whole thing breaks. I have to bypass 185.1.1.1 manually to make it work.

# Mac
sudo route add -net 185.1.1.1 192.168.1.1 255.255.255.255

# Linux
sudo ip r a 185.1.1.1/32 via 192.168.1.1 dev wlo1

On a Windows system the above step is not required, and works just fine but on MacOS and Linux, I have to manually bypass the v2ray ip (185.1.1.1). Like I said it makes sense since there is no way for openvpn to detect the v2ray ip (185.1.1.1) and bypass it, But my question is how this is working on Windows?

selvanair commented 11 months ago

Okay, got it finally! You do need a bypass route to the actual server as the proxy is running locally on the client. OpenVPN will not set this up automatically. You can add a route statement to the ovpn for this or set up the route in advance as you have done.

And, yes, the same should be required on Windows too -- what does route print show on Windows?

cron2 commented 11 months ago

So, my interpretation of this is:

I guess it could work on Windows if the proxy service does magic route policy things (windows networking is actually fairly powerful, though badly documented), so it's not affected by routes installed by OpenVPN. We had some discussions on making OpenVPN use different route tables / route policy on Linux, which might make this work "automatically", but MacOS does not seem to have mechanisms for it.

flichtenheld commented 8 months ago

So it seems to me the conclusion here was: Everything works as expected. Should we close it again?

Nicoloman commented 5 months ago

Hey! Im looking for help, having kinda the same problem. I connect to the VPN with globalProtect client. Then i set up my proxies but nothing happens. I bypassed this behavior by setting up proxies on Firefox browser, and being 'Offline' on all my other desktop apps.

Using the same proxy config's on MacOS settings does no effect. Also works on windows flawlessly. Im lost and i could use some help.

Mackbook Pro M2 Sonoma 14.5