search

adrienverge / openfortivpn

Client for PPP+TLS VPN tunnel services
GNU General Public License v3.0
2.7k stars 320 forks source link

Unable to set correct route for OSX high sierra 10.13.6 #378

Closed demonlj closed 5 years ago

demonlj commented 6 years ago

mac osx: high sierra 10.13.6 openfortivpn: 1.7.1 via homebrew installation It seems like delete my default static route, cause the network shutdown, then app terminated config

port = 443
username = user
password = pass
set-routes = 1
set-dns = 0
trusted-cert = c57ed276aab8b6fe7b8bbbe1e9c5f677eaf6916485bd7db9dfd66fa276137d0d

debug

INFO:   Setting ppp interface down.
INFO:   Restoring routes...
DEBUG:  route -n delete 221.224.34.184 -netmask 255.255.255.255
delete net 221.224.34.184
DEBUG:  route -n delete 0.0.0.0 -netmask 0.0.0.0
delete net 0.0.0.0
DEBUG:  route -n add -net 0.0.0.0 -netmask 0.0.0.0 -interface ppp0
add net 0.0.0.0: gateway ppp0
DEBUG:  Waiting for pppd to exit...
DEBUG:  waitpid: pppd exit status code 16
INFO:   pppd: The link was terminated by the modem hanging up.
INFO:   Terminated pppd.
INFO:   Closed connection to gateway.
DEBUG:  server_addr: 221.224.34.184
DEBUG:  server_port: 443
DEBUG:  gateway_addr: 221.224.34.184
DEBUG:  gateway_port: 443
ERROR:  connect: Network is unreachable
INFO:   Could not log out.

Correct route via original forticlient

Routing tables

Internet:
Destination        Gateway            Flags        Refs      Use   Netif Expire
default            link#15            UCS            68        0    ppp0
default            192.168.1.1        UGScI           8        0     en1
1.1.1.1            42.244.60.70       UH              0        0    ppp0
10.9.0.110         link#15            UHWIi           3       18    ppp0
17.248.161.8       link#15            UHWIi           1       11    ppp0
17.248.161.46      link#15            UHWIi           2       29    ppp0
17.252.156.35      link#15            UHWIi           1       28    ppp0
23.198.140.135     link#15            UHW3I           0       34    ppp0      7
74.125.203.188     link#15            UHWIi           2       13    ppp0
117.18.237.29      link#15            UHW3I           0        6    ppp0      6
127                127.0.0.1          UCS             0        0     lo0
127.0.0.1          127.0.0.1          UH              3  1672785     lo0
169.254            link#10            UCS             0        0     en1
192.168.1          link#10            UCS             1        0     en1
192.168.1.1/32     link#10            UCS             1        0     en1
192.168.1.1        ec:88:8f:4f:5f:ba  UHLWIir        10     1288     en1    887
192.168.1.100/32   link#10            UCS             0        0     en1
192.168.1.106      e8:2a:ea:24:c0:b6  UHLWIi          1      103     en1   1104
192.168.56         link#16            UC              1        0 vboxnet
202.114.91.39      link#15            UHWIi           1       11    ppp0
202.195.128.20     link#15            UHW3I           0        3    ppp0      3
221.224.34.184/32  192.168.1.1        UGSc            0        0     en1
224.0.0/4          link#15            UmCS            1        0    ppp0
224.0.0/4          link#10            UmCSI           1        0     en1
239.255.255.250    1:0:5e:7f:ff:fa    UHmLWI          0        5     en1
239.255.255.250    link#15            UHmW3I          0        5    ppp0      4
255.255.255.255/32 link#15            UCS             0        0    ppp0
255.255.255.255/32 link#10            UCSI            0        0     en1

Wrong route 


Internet:
Destination        Gateway            Flags        Refs      Use   Netif Expire
default            ppp0               USc             0        0    ppp0
default            link#15            UCSI           53        0    ppp0
10.9.0.110         link#15            UHWIi          14       25    ppp0
127                127.0.0.1          UCS             0        0     lo0
127.0.0.1          127.0.0.1          UH              3  1672825     lo0
169.254            link#10            UCS             0        0     en1
192.0.2.1          42.244.59.180      UH              0        0    ppp0
192.168.1          link#10            UCS             2        0     en1
192.168.1.1/32     link#10            UCS             1        0     en1
192.168.1.1        ec:88:8f:4f:5f:ba  UHLWIir         1     1307     en1    850
192.168.1.100/32   link#10            UCS             0        0     en1
192.168.1.106      e8:2a:ea:24:c0:b6  UHLWIi          1      103     en1   1067
192.168.1.255      ff:ff:ff:ff:ff:ff  UHLWbI          0        1     en1
192.168.56         link#16            UC              2        0 vboxnet
192.168.56.255     ff:ff:ff:ff:ff:ff  UHLWbI          0        1 vboxnet
221.224.34.184/32  ppp0               USc             0        0    ppp0
224.0.0/4          link#10            UmCS            1        0     en1
224.0.0/4          link#15            UmCSI           1        0    ppp0
239.255.255.250    1:0:5e:7f:ff:fa    UHmLWI          0        7     en1
239.255.255.250    link#15            UHmW3I          0        3    ppp0      9
255.255.255.255/32 link#10            UCS             0        0     en1
255.255.255.255/32 link#15            UCSI            0        0    ppp0
mrbaseman commented 6 years ago

indeed, something seems to go wrong here. the 'link#15' as gateway isn't handled properly. I'll have a look at the parser code for OSX.

demonlj commented 6 years ago

indeed, something seems to go wrong here. the 'link#15' as gateway isn't handled properly. I'll have a look at the parser code for OSX.

Thanks a lot. Just waiting for good news

mrbaseman commented 5 years ago

@demonlj have you tried the --half-internet-routes=1 option? It avoids deleting the default route, which may help in this particular situation as a quick workaround. Anyhow, I still have to look deeper into the code which handles the routing table manipulations on OS X.

demonlj commented 5 years ago

with --half-internet-routes=1 option, the route will be like this. But I cannot connect websites which require the private vpn

$ netstat -nrt
Routing tables

Internet:
Destination        Gateway            Flags        Refs      Use   Netif Expire
default            192.168.9.1        UGSc           90        2     en1
default            link#15            UCSI            1        0    ppp0
10.9.0.110         link#15            UHWIi           5       30    ppp0
127                127.0.0.1          UCS             0        0     lo0
127.0.0.1          127.0.0.1          UH              8  2285837     lo0
169.254            link#10            UCS             0        0     en1
192.0.2.1          42.244.61.4        UH              0        0    ppp0
192.168.9          link#10            UCS             6        0     en1
192.168.9.1/32     link#10            UCS             2        0     en1
192.168.9.1        3a:b7:de:42:bc:cf  UHLWIir        20       32     en1   1178
192.168.9.5        0:11:32:58:62:d4   UHLWIi          1        2     en1   1187
192.168.9.6        36:63:31:62:61:34  UHLWI           0        1     en1   1169
192.168.9.154      8c:be:be:27:e1:ec  UHLWI           0        1     en1   1160
192.168.9.175      0:22:6d:2c:22:d2   UHLWI           0        1     en1   1138
192.168.9.185      e8:4e:6:24:b3:c0   UHLWI           0        1     en1   1142
192.168.9.234/32   link#10            UCS             0        0     en1
192.168.9.255      ff:ff:ff:ff:ff:ff  UHLWbI          0        3     en1
192.168.56         link#16            UC              2        0 vboxnet
192.168.56.255     ff:ff:ff:ff:ff:ff  UHLWbI          0        5 vboxnet
224.0.0/4          link#10            UmCS            2        0     en1
224.0.0/4          link#15            UmCSI           1        0    ppp0
224.0.0.251        1:0:5e:0:0:fb      UHmLWI          0        0     en1
239.255.255.250    1:0:5e:7f:ff:fa    UHmLWI          0        8     en1
239.255.255.250    link#15            UHmW3I          0        4    ppp0
255.255.255.255/32 link#10            UCS             0        0     en1
255.255.255.255/32 link#15            UCSI            0        0    ppp0
mrbaseman commented 5 years ago

Sorry, I believe that I still don't understand your exact use case. My current understanding is that rather the various default routes cause the problem rather than the link#... entries.

The routing tables above are the status after openfortivpn has terminated? and ppp0 is another ppp link (a modem or whatever?), and openfortivpn adds another one (probably ppp1) when it's running? And your problem is that after shutdown of openfortivpn the routing table is not correctly restored, whereas while openfortivpn is running, everything is fine, right?

If this is correct, how does the routing table look before openfortivpn is running an how does it look while it is running? And is the ssl vpn connection established over en1 or over ppp0?

demonlj commented 5 years ago

It's really complicated. The orig network was en1 (wifi), when openfortivpn works, it will established vpn connection over en1. This is the orig routing table.

Routing tables

Internet:
Destination        Gateway            Flags        Refs      Use   Netif Expire
default            192.168.1.1        UGSc           70        0     en1
127                127.0.0.1          UCS             0        0     lo0
127.0.0.1          127.0.0.1          UH              3   153949     lo0
169.254            link#8             UCS             0        0     en1
192.168.1          link#8             UCS             1        0     en1
192.168.1.1/32     link#8             UCS             1        0     en1
192.168.1.1        ec:88:8f:4f:5f:ba  UHLWIir        18       89     en1    986
192.168.1.102/32   link#8             UCS             0        0     en1
192.168.1.255      ff:ff:ff:ff:ff:ff  UHLWbI          0       10     en1
224.0.0/4          link#8             UmCS            2        0     en1
224.0.0.251        1:0:5e:0:0:fb      UHmLWI          0        0     en1
239.255.255.250    1:0:5e:7f:ff:fa    UHmLWI          0       28     en1
255.255.255.255/32 link#8             UCS             0        0     en1

This is the wrong routing table while openfortivpn shutdown the default routes of en1

Routing tables

Internet:
Destination        Gateway            Flags        Refs      Use   Netif Expire
default            ppp0               USc             0        0    ppp0
default            link#15            UCSI           61        0    ppp0
8.8.8.8            link#15            UHWIi           1        1    ppp0
10.9.0.110         link#15            UHWIi           3       21    ppp0
45.32.127.201      link#15            UHWIi           1        7    ppp0
114.114.114.114    link#15            UHWIi           6       14    ppp0
127                127.0.0.1          UCS             0        0     lo0
127.0.0.1          127.0.0.1          UH              4   154361     lo0
169.254            link#8             UCS             0        0     en1
192.0.2.1          42.244.58.100      UH              0        0    ppp0
192.168.1          link#8             UCS             1        0     en1
192.168.1.1/32     link#8             UCS             1        0     en1
192.168.1.1        ec:88:8f:4f:5f:ba  UHLWIir         1      114     en1    772
192.168.1.102/32   link#8             UCS             1        0     en1
192.168.1.255      ff:ff:ff:ff:ff:ff  UHLWbI          0        7     en1
202.195.128.20     link#15            UHW3I           0       14    ppp0     40
220.181.13.229     link#15            UHWIi           1        7    ppp0
221.224.34.184/32  ppp0               USc             0        0    ppp0
224.0.0/4          link#8             UmCS            2        0     en1
224.0.0/4          link#15            UmCSI           1        0    ppp0
224.0.0.251        1:0:5e:0:0:fb      UHmLWI          0        0     en1
239.255.255.250    1:0:5e:7f:ff:fa    UHmLWI          0       40     en1
239.255.255.250    link#15            UHmW3I          0        4    ppp0     35
255.255.255.255/32 link#8             UCS             0        0     en1
255.255.255.255/32 link#15            UCSI            0        0    ppp0
mrbaseman commented 5 years ago

Here a deeper analysis of what's going on:

Before running openfortivpn you have only one default route. As soon as you run it, another default route over ppp0 is added. This is something which is not intended, but I can reproduce it with the current master, with my bsd_support branch and also with 1.6.0, just to take one of the older versions to see if it is a regression which has been introduced recently. At least with 1.6.0 the behavior has also been like this, so maybe it has just not been noticed so far. With my branch that integrates FreeBSD support, there is even more debugging output and it turns out that the additional default route is already there, before openfortivpn begins to set up the routes. Maybe pppd adds it in spite of the option nodefaultroute, or a helper script around pppd. As a consequence, when openfortivpn searches for the appropriate route to the vpn gateway, it picks the wrong one and adds a route to the gateway over the ppp0 interface. For some reason, the tunnel is working nevertheless. That's somehow perplexing, because we have introduced this specific route which should go over the en1 interface in your case for the linux version in order to keep the ssl encoded tunnel traffic to the vpn gateway up, even when the vpn gateway pushes a route which contains the vpn interface itself. On OSX it looks as if this route has exactly the opposite effect due to the wrong interface being assigned, but strikingly it works

Independent of these findings, Fortinet's ssl vpn supports two different modes of operation: split mode and tunnel mode. In split mode, only traffic to specific networks is routed through the tunnel, whereas all other traffic (e.g. to the internet) still goes its regular path. It seems you are expecting tunnel mode, in which all traffic shall be routed through the tunnel. I believe that in this case the Fortigate just pushes a new default route (and perhaps other routes to specific private networks). And it is expected behavior that the existing default route is removed by the client (openfortivpn in this case). It should, however, restore the route correctly when it is terminated.

The operation mode --half-internet-routes=1 avoids removing and adding the default route and as we can see the default route over en1 is still there in your case. It should add two additional routes to 0.0.0.0/1 and 128.0.0.0/1. However, this is supposed to work with tunnel mode, but even with Forticlient you don't recieve an additional default route, so I assume your VPN access is configured as split mode.

Anyhow, when looking at the routing code for OSX I have spotted an issue when a route is deleted. The man page for route recommends to explicitly specifiy the interface when the route has no gateway associated. I'm not sure if this solves your problem, but if you want you can try this branch.

About the wrong default route and the hence wrong interface assigned to the gateway route I have to investigate further.

demonlj commented 5 years ago

I tried to install bsd_support and 1.6.0 branch, got this error

$ make
  CCLD     openfortivpn
Undefined symbols for architecture x86_64:
  "_CRYPTO_free", referenced from:
      _io_loop in openfortivpn-io.o
  "_CRYPTO_malloc", referenced from:
      _io_loop in openfortivpn-io.o
  "_CRYPTO_num_locks", referenced from:
      _io_loop in openfortivpn-io.o
  "_CRYPTO_set_id_callback", referenced from:
      _io_loop in openfortivpn-io.o
  "_CRYPTO_set_locking_callback", referenced from:
      _io_loop in openfortivpn-io.o
  "_ERR_error_string", referenced from:
      _http_send in openfortivpn-http.o
      _http_receive in openfortivpn-http.o
      _ssl_read in openfortivpn-io.o
      _ssl_write in openfortivpn-io.o
      _ssl_connect in openfortivpn-tunnel.o
  "_ERR_peek_last_error", referenced from:
      _http_send in openfortivpn-http.o
      _http_receive in openfortivpn-http.o
      _handle_ssl_error in openfortivpn-http.o
      _ssl_read in openfortivpn-io.o
      _ssl_write in openfortivpn-io.o
      _handle_ssl_error in openfortivpn-io.o
      _ssl_connect in openfortivpn-tunnel.o
      ...
  "_EVP_sha256", referenced from:
      _ssl_connect in openfortivpn-tunnel.o
  "_SSL_CTX_check_private_key", referenced from:
      _ssl_connect in openfortivpn-tunnel.o
  "_SSL_CTX_ctrl", referenced from:
      _ssl_connect in openfortivpn-tunnel.o
  "_SSL_CTX_free", referenced from:
      _ssl_connect in openfortivpn-tunnel.o
  "_SSL_CTX_load_verify_locations", referenced from:
      _ssl_connect in openfortivpn-tunnel.o
  "_SSL_CTX_new", referenced from:
      _ssl_connect in openfortivpn-tunnel.o
  "_SSL_CTX_set_default_verify_paths", referenced from:
      _ssl_connect in openfortivpn-tunnel.o
  "_SSL_CTX_use_PrivateKey_file", referenced from:
      _ssl_connect in openfortivpn-tunnel.o
  "_SSL_CTX_use_certificate_file", referenced from:
      _ssl_connect in openfortivpn-tunnel.o
  "_SSL_connect", referenced from:
      _ssl_connect in openfortivpn-tunnel.o
  "_SSL_ctrl", referenced from:
      _ssl_connect in openfortivpn-tunnel.o
  "_SSL_free", referenced from:
      _ssl_connect in openfortivpn-tunnel.o
  "_SSL_get_error", referenced from:
      _handle_ssl_error in openfortivpn-http.o
      _handle_ssl_error in openfortivpn-io.o
  "_SSL_get_peer_certificate", referenced from:
      _ssl_connect in openfortivpn-tunnel.o
  "_SSL_get_shutdown", referenced from:
      _handle_ssl_error in openfortivpn-http.o
      _handle_ssl_error in openfortivpn-io.o
  "_SSL_get_verify_result", referenced from:
      _ssl_connect in openfortivpn-tunnel.o
  "_SSL_library_init", referenced from:
      _ssl_connect in openfortivpn-tunnel.o
  "_SSL_load_error_strings", referenced from:
      _ssl_connect in openfortivpn-tunnel.o
  "_SSL_new", referenced from:
      _ssl_connect in openfortivpn-tunnel.o
  "_SSL_read", referenced from:
      _http_receive in openfortivpn-http.o
      _ssl_read in openfortivpn-io.o
  "_SSL_set_cipher_list", referenced from:
      _ssl_connect in openfortivpn-tunnel.o
  "_SSL_set_fd", referenced from:
      _ssl_connect in openfortivpn-tunnel.o
  "_SSL_set_verify", referenced from:
      _ssl_connect in openfortivpn-tunnel.o
  "_SSL_shutdown", referenced from:
      _ssl_connect in openfortivpn-tunnel.o
  "_SSL_write", referenced from:
      _http_send in openfortivpn-http.o
      _ssl_write in openfortivpn-io.o
  "_SSLv23_client_method", referenced from:
      _ssl_connect in openfortivpn-tunnel.o
  "_X509_NAME_get_text_by_NID", referenced from:
      _ssl_connect in openfortivpn-tunnel.o
  "_X509_NAME_oneline", referenced from:
      _ssl_connect in openfortivpn-tunnel.o
  "_X509_digest", referenced from:
      _ssl_connect in openfortivpn-tunnel.o
  "_X509_free", referenced from:
      _ssl_connect in openfortivpn-tunnel.o
  "_X509_get_issuer_name", referenced from:
      _ssl_connect in openfortivpn-tunnel.o
  "_X509_get_subject_name", referenced from:
      _ssl_connect in openfortivpn-tunnel.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [openfortivpn] Error 1
mrbaseman commented 5 years ago

this looks as if the Openssl libraries are not properly linked. Normally, the configure script should have checked this before. Did you run ./autogen.sh and ./configure before make?

In order to use the openssl provided by Homebrew I had to add 

export PATH="/usr/local/opt/openssl/bin:$PATH"
export LDFLAGS="-L/usr/local/opt/openssl/lib $LDFLAGS"
export CPPFLAGS="-I/usr/local/opt/openssl/include $CPPFLAGS"

to my $HOME/.bash_profile I'm not sure if this is all still needed. Parts of it might already be handled by pkg-config within the configure script.

We have merged the bsd_support branch into the current master now, so you could use that for testing as well.

The mac_routing branch that I have suggested currently has more problems when deleting the routes than the current master. Probably it shuts down the route to the gateway too early, but I have to double check that.

mrbaseman commented 5 years ago

with my latest commit e21ab8d171cd4b9c7be6b6f4f32e4f0e1dd378ee the mac_routing branch works again. Maybe this solves the current issue?

demonlj commented 5 years ago

Thanks a lot! I found it's libssl & openssl problem to cause the failure of compile. ./configure fail to check libssl & libcrypto

checking for libssl >= 0.9.8 libcrypto >= 0.9.8... no
configure: error: Cannot find OpenSSL 0.9.8 or higher.
$ pkg-config --exists --print-errors "libssl >= 0.9.8 libcrypto >= 0.9.8"
Package libssl was not found in the pkg-config search path.
Perhaps you should add the directory containing `libssl.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libssl' found
Package libcrypto was not found in the pkg-config search path.
Perhaps you should add the directory containing `libcrypto.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libcrypto' found

however, brew fail to install libssl

$ brew install libssl libcrpto
Updating Homebrew...
Error: No available formula with the name "libssl"
==> Searching for a previously deleted formula (in the last month)...
Warning: homebrew/core is shallow clone. To get complete history run:
  git -C "$(brew --repo homebrew/core)" fetch --unshallow

Error: No previously deleted formula found.
==> Searching for similarly named formulae...
Error: No similarly named formulae found.
==> Searching taps...
==> Searching taps on GitHub...
Error: No formulae found in taps.

if I export parameter like this, It will pass the configure but fail the compile.

export OPENSSL_CFLAGS="-I/usr/local/opt/openssl/include"
    export OPENSSL_LIBS="-L/usr/local/opt/openssl/lib"

If I export parameter like this, It will not pass the configure

export PATH="/usr/local/opt/openssl/bin:$PATH"
export LDFLAGS="-L/usr/local/opt/openssl/lib $LDFLAGS"
export CPPFLAGS="-I/usr/local/opt/openssl/include $CPPFLAGS"
demonlj commented 5 years ago

with this parameter, it can pass the configure and compile.

export PKG_CONFIG_PATH='/usr/local/opt/openssl/lib/pkgconfig/'

But the route problem still there

INFO:   Remote gateway has allocated a VPN.
DEBUG:  server_addr: 221.224.34.184
DEBUG:  server_port: 443
DEBUG:  gateway_addr: 221.224.34.184
DEBUG:  gateway_port: 443
DEBUG:  Gateway certificate validation failed.
DEBUG:  Gateway certificate digest found in white list.
DEBUG:  Retrieving configuration
DEBUG:  Establishing the tunnel
DEBUG:  ppp_path: /usr/sbin/pppd
DEBUG:  Switch to tunneling mode
DEBUG:  Starting IO through the tunnel
DEBUG:  pppd_read thread
DEBUG:  ssl_read thread
DEBUG:  ssl_write thread
DEBUG:  if_config thread
DEBUG:  gateway ---> pppd (12 bytes)
DEBUG:  pppd ---> gateway (16 bytes)
DEBUG:  pppd_write thread
DEBUG:  pppd ---> gateway (12 bytes)
DEBUG:  gateway ---> pppd (16 bytes)
DEBUG:  gateway ---> pppd (12 bytes)
DEBUG:  pppd ---> gateway (18 bytes)
DEBUG:  pppd ---> gateway (18 bytes)
DEBUG:  pppd ---> gateway (12 bytes)
DEBUG:  gateway ---> pppd (12 bytes)
DEBUG:  gateway ---> pppd (26 bytes)
DEBUG:  gateway ---> pppd (12 bytes)
DEBUG:  pppd ---> gateway (12 bytes)
DEBUG:  pppd ---> gateway (12 bytes)
DEBUG:  gateway ---> pppd (12 bytes)
DEBUG:  gateway ---> pppd (12 bytes)
DEBUG:  pppd ---> gateway (12 bytes)
DEBUG:  pppd ---> gateway (12 bytes)
DEBUG:  gateway ---> pppd (12 bytes)
INFO:   Got addresses: [42.244.60.17], ns [0.0.0.0, 0.0.0.0]
INFO:   negotiation complete
DEBUG:  Got Address: 42.244.60.17
DEBUG:  gateway ---> pppd (12 bytes)
DEBUG:  pppd ---> gateway (12 bytes)
DEBUG:  if_config: not ready yet...
DEBUG:  gateway ---> pppd (12 bytes)
DEBUG:  pppd ---> gateway (12 bytes)
DEBUG:  gateway ---> pppd (12 bytes)
DEBUG:  pppd ---> gateway (12 bytes)
DEBUG:  gateway ---> pppd (16 bytes)
DEBUG:  pppd ---> gateway (16 bytes)
DEBUG:  gateway ---> pppd (6 bytes)
INFO:   negotiation complete
DEBUG:  pppd ---> gateway (6 bytes)
DEBUG:  Got Address: 42.244.60.17
DEBUG:  Interface Name: ppp0
DEBUG:  Interface Addr: 42.244.60.17
INFO:   Interface ppp0 is UP.
INFO:   Setting new routes...
DEBUG:  ip route show to 0.0.0.0/0.0.0.0
DEBUG:  netstat_path: /usr/sbin/netstat
DEBUG:  line: default            192.168.9.1        UGSc           92        0     en1
DEBUG:  - Destination: default
DEBUG:  - Destination IP Hex: 0
DEBUG:  - Destination Mask Hex: 0
DEBUG:  - Gateway Mask Hex: 109a8c0
DEBUG:  - Interface: en1
DEBUG:
DEBUG:  line: default            link#15            UCSI            0        0    ppp0
DEBUG:  - Destination: default
DEBUG:  - Destination IP Hex: 0
DEBUG:  - Destination Mask Hex: 0
DEBUG:  - Interface: ppp0
DEBUG:
DEBUG:  line: 127                127.0.0.1          UCS             0        0     lo0
DEBUG:  - Destination: 127
DEBUG:  - Destination IP Hex: 7f
DEBUG:  - Destination Mask Hex: ff000000
DEBUG:  - Gateway Mask Hex: 100007f
DEBUG:  - Interface: lo0
DEBUG:
DEBUG:  line: 127.0.0.1          127.0.0.1          UH              4    80583     lo0
DEBUG:  - Destination: 127.0.0.1
DEBUG:  - Destination IP Hex: 100007f
DEBUG:  - Destination Mask Hex: ffffffff
DEBUG:  - Gateway Mask Hex: 100007f
DEBUG:  - Interface: lo0
DEBUG:
DEBUG:  line: 169.254            link#8             UCS             0        0     en1
DEBUG:  - Destination: 169.254
DEBUG:  - Destination IP Hex: fea9
DEBUG:  - Destination Mask Hex: ffff0000
DEBUG:  - Interface: en1
DEBUG:
DEBUG:  line: 192.0.2.1          42.244.60.17       UH              0        0    ppp0
DEBUG:  - Destination: 192.0.2.1
DEBUG:  - Destination IP Hex: 10200c0
DEBUG:  - Destination Mask Hex: ffffffff
DEBUG:  - Gateway Mask Hex: 113cf42a
DEBUG:  - Interface: ppp0
DEBUG:
DEBUG:  line: 192.168.9          link#8             UCS             5        0     en1
DEBUG:  - Destination: 192.168.9
DEBUG:  - Destination IP Hex: 9a8c0
DEBUG:  - Destination Mask Hex: ffffff00
DEBUG:  - Interface: en1
DEBUG:
DEBUG:  line: 192.168.9.1/32     link#8             UCS             2        0     en1
DEBUG:  - Destination: 192.168.9.1/32
DEBUG:  - Destination IP Hex: 109a8c0
DEBUG:  - Destination Mask Hex: ffffffff
DEBUG:  - Interface: en1
DEBUG:
DEBUG:  line: 192.168.9.1        3a:b7:de:42:bc:cf  UHLWIir        28       19     en1   1182
DEBUG:  - Destination: 192.168.9.1
DEBUG:  - Destination IP Hex: 109a8c0
DEBUG:  - Destination Mask Hex: ffffffff
DEBUG:  - Interface: en1
DEBUG:
DEBUG:  line: 192.168.9.5        0:11:32:58:62:d4   UHLWIi          1        3     en1   1161
DEBUG:  - Destination: 192.168.9.5
DEBUG:  - Destination IP Hex: 509a8c0
DEBUG:  - Destination Mask Hex: ffffffff
DEBUG:  - Interface: en1
DEBUG:
DEBUG:  line: 192.168.9.6        36:63:31:62:61:34  UHLWI           0        1     en1   1161
DEBUG:  - Destination: 192.168.9.6
DEBUG:  - Destination IP Hex: 609a8c0
DEBUG:  - Destination Mask Hex: ffffffff
DEBUG:  - Interface: en1
DEBUG:
DEBUG:  line: 192.168.9.154      8c:be:be:27:e1:ec  UHLWI           0        2     en1   1200
DEBUG:  - Destination: 192.168.9.154
DEBUG:  - Destination IP Hex: 9a09a8c0
DEBUG:  - Destination Mask Hex: ffffffff
DEBUG:  - Interface: en1
DEBUG:
DEBUG:  line: 192.168.9.175      0:22:6d:2c:22:d2   UHLWIi          1        2     en1   1197
DEBUG:  - Destination: 192.168.9.175
DEBUG:  - Destination IP Hex: af09a8c0
DEBUG:  - Destination Mask Hex: ffffffff
DEBUG:  - Interface: en1
DEBUG:
DEBUG:  line: 192.168.9.234/32   link#8             UCS             0        0     en1
DEBUG:  - Destination: 192.168.9.234/32
DEBUG:  - Destination IP Hex: ea09a8c0
DEBUG:  - Destination Mask Hex: ffffffff
DEBUG:  - Interface: en1
DEBUG:
DEBUG:  line: 192.168.9.255      ff:ff:ff:ff:ff:ff  UHLWbI          0        6     en1
DEBUG:  - Destination: 192.168.9.255
DEBUG:  - Destination IP Hex: ff09a8c0
DEBUG:  - Destination Mask Hex: ffffffff
DEBUG:  - Interface: en1
DEBUG:
DEBUG:  line: 224.0.0/4          link#8             UmCS            2        0     en1
DEBUG:  - Destination: 224.0.0/4
DEBUG:  - Destination IP Hex: e0
DEBUG:  - Destination Mask Hex: f0000000
DEBUG:  - Interface: en1
DEBUG:
DEBUG:  line: 224.0.0/4          link#15            UmCSI           0        0    ppp0
DEBUG:  - Destination: 224.0.0/4
DEBUG:  - Destination IP Hex: e0
DEBUG:  - Destination Mask Hex: f0000000
DEBUG:  - Interface: ppp0
DEBUG:
DEBUG:  line: 224.0.0.251        1:0:5e:0:0:fb      UHmLWI          0        0     en1
DEBUG:  - Destination: 224.0.0.251
DEBUG:  - Destination IP Hex: fb0000e0
DEBUG:  - Destination Mask Hex: ffffffff
DEBUG:  - Interface: en1
DEBUG:
DEBUG:  line: 239.255.255.250    1:0:5e:7f:ff:fa    UHmLWI          0        4     en1
DEBUG:  - Destination: 239.255.255.250
DEBUG:  - Destination IP Hex: faffffef
DEBUG:  - Destination Mask Hex: ffffffff
DEBUG:  - Interface: en1
DEBUG:
DEBUG:  line: 255.255.255.255/32 link#8             UCS             0        0     en1
DEBUG:  - Destination: 255.255.255.255/32
DEBUG:  - Destination IP Hex: ffffffff
DEBUG:  - Destination Mask Hex: ffffffff
DEBUG:  - Interface: en1
DEBUG:
DEBUG:  line: 255.255.255.255/32 link#15            UCSI            0        0    ppp0
DEBUG:  - Destination: 255.255.255.255/32
DEBUG:  - Destination IP Hex: ffffffff
DEBUG:  - Destination Mask Hex: ffffffff
DEBUG:  - Interface: ppp0
DEBUG:
DEBUG:  ip route show to 221.224.34.184/255.255.255.255
DEBUG:  netstat_path: /usr/sbin/netstat
DEBUG:  line: default            192.168.9.1        UGSc           92        0     en1
DEBUG:  - Destination: default
DEBUG:  - Destination IP Hex: 0
DEBUG:  - Destination Mask Hex: 0
DEBUG:  - Gateway Mask Hex: 109a8c0
DEBUG:  - Interface: en1
DEBUG:
DEBUG:  line: default            link#15            UCSI            0        0    ppp0
DEBUG:  - Destination: default
DEBUG:  - Destination IP Hex: 0
DEBUG:  - Destination Mask Hex: 0
DEBUG:  - Interface: ppp0
DEBUG:
DEBUG:  line: 127                127.0.0.1          UCS             0        0     lo0
DEBUG:  - Destination: 127
DEBUG:  - Destination IP Hex: 7f
DEBUG:  - Destination Mask Hex: ff000000
DEBUG:  - Gateway Mask Hex: 100007f
DEBUG:  - Interface: lo0
DEBUG:
DEBUG:  line: 127.0.0.1          127.0.0.1          UH              4    80583     lo0
DEBUG:  - Destination: 127.0.0.1
DEBUG:  - Destination IP Hex: 100007f
DEBUG:  - Destination Mask Hex: ffffffff
DEBUG:  - Gateway Mask Hex: 100007f
DEBUG:  - Interface: lo0
DEBUG:
DEBUG:  line: 169.254            link#8             UCS             0        0     en1
DEBUG:  - Destination: 169.254
DEBUG:  - Destination IP Hex: fea9
DEBUG:  - Destination Mask Hex: ffff0000
DEBUG:  - Interface: en1
DEBUG:
DEBUG:  line: 192.0.2.1          42.244.60.17       UH              0        0    ppp0
DEBUG:  - Destination: 192.0.2.1
DEBUG:  - Destination IP Hex: 10200c0
DEBUG:  - Destination Mask Hex: ffffffff
DEBUG:  - Gateway Mask Hex: 113cf42a
DEBUG:  - Interface: ppp0
DEBUG:
DEBUG:  line: 192.168.9          link#8             UCS             5        0     en1
DEBUG:  - Destination: 192.168.9
DEBUG:  - Destination IP Hex: 9a8c0
DEBUG:  - Destination Mask Hex: ffffff00
DEBUG:  - Interface: en1
DEBUG:
DEBUG:  line: 192.168.9.1/32     link#8             UCS             2        0     en1
DEBUG:  - Destination: 192.168.9.1/32
DEBUG:  - Destination IP Hex: 109a8c0
DEBUG:  - Destination Mask Hex: ffffffff
DEBUG:  - Interface: en1
DEBUG:
DEBUG:  line: 192.168.9.1        3a:b7:de:42:bc:cf  UHLWIir        28       19     en1   1182
DEBUG:  - Destination: 192.168.9.1
DEBUG:  - Destination IP Hex: 109a8c0
DEBUG:  - Destination Mask Hex: ffffffff
DEBUG:  - Interface: en1
DEBUG:
DEBUG:  line: 192.168.9.5        0:11:32:58:62:d4   UHLWIi          1        3     en1   1161
DEBUG:  - Destination: 192.168.9.5
DEBUG:  - Destination IP Hex: 509a8c0
DEBUG:  - Destination Mask Hex: ffffffff
DEBUG:  - Interface: en1
DEBUG:
DEBUG:  line: 192.168.9.6        36:63:31:62:61:34  UHLWI           0        1     en1   1161
DEBUG:  - Destination: 192.168.9.6
DEBUG:  - Destination IP Hex: 609a8c0
DEBUG:  - Destination Mask Hex: ffffffff
DEBUG:  - Interface: en1
DEBUG:
DEBUG:  line: 192.168.9.154      8c:be:be:27:e1:ec  UHLWI           0        2     en1   1200
DEBUG:  - Destination: 192.168.9.154
DEBUG:  - Destination IP Hex: 9a09a8c0
DEBUG:  - Destination Mask Hex: ffffffff
DEBUG:  - Interface: en1
DEBUG:
DEBUG:  line: 192.168.9.175      0:22:6d:2c:22:d2   UHLWIi          1        2     en1   1197
DEBUG:  - Destination: 192.168.9.175
DEBUG:  - Destination IP Hex: af09a8c0
DEBUG:  - Destination Mask Hex: ffffffff
DEBUG:  - Interface: en1
DEBUG:
DEBUG:  line: 192.168.9.234/32   link#8             UCS             0        0     en1
DEBUG:  - Destination: 192.168.9.234/32
DEBUG:  - Destination IP Hex: ea09a8c0
DEBUG:  - Destination Mask Hex: ffffffff
DEBUG:  - Interface: en1
DEBUG:
DEBUG:  line: 192.168.9.255      ff:ff:ff:ff:ff:ff  UHLWbI          0        6     en1
DEBUG:  - Destination: 192.168.9.255
DEBUG:  - Destination IP Hex: ff09a8c0
DEBUG:  - Destination Mask Hex: ffffffff
DEBUG:  - Interface: en1
DEBUG:
DEBUG:  line: 224.0.0/4          link#8             UmCS            2        0     en1
DEBUG:  - Destination: 224.0.0/4
DEBUG:  - Destination IP Hex: e0
DEBUG:  - Destination Mask Hex: f0000000
DEBUG:  - Interface: en1
DEBUG:
DEBUG:  line: 224.0.0/4          link#15            UmCSI           0        0    ppp0
DEBUG:  - Destination: 224.0.0/4
DEBUG:  - Destination IP Hex: e0
DEBUG:  - Destination Mask Hex: f0000000
DEBUG:  - Interface: ppp0
DEBUG:
DEBUG:  line: 224.0.0.251        1:0:5e:0:0:fb      UHmLWI          0        0     en1
DEBUG:  - Destination: 224.0.0.251
DEBUG:  - Destination IP Hex: fb0000e0
DEBUG:  - Destination Mask Hex: ffffffff
DEBUG:  - Interface: en1
DEBUG:
DEBUG:  line: 239.255.255.250    1:0:5e:7f:ff:fa    UHmLWI          0        4     en1
DEBUG:  - Destination: 239.255.255.250
DEBUG:  - Destination IP Hex: faffffef
DEBUG:  - Destination Mask Hex: ffffffff
DEBUG:  - Interface: en1
DEBUG:
DEBUG:  line: 255.255.255.255/32 link#8             UCS             0        0     en1
DEBUG:  - Destination: 255.255.255.255/32
DEBUG:  - Destination IP Hex: ffffffff
DEBUG:  - Destination Mask Hex: ffffffff
DEBUG:  - Interface: en1
DEBUG:
DEBUG:  line: 255.255.255.255/32 link#15            UCSI            0        0    ppp0
DEBUG:  - Destination: 255.255.255.255/32
DEBUG:  - Destination IP Hex: ffffffff
DEBUG:  - Destination Mask Hex: ffffffff
DEBUG:  - Interface: ppp0
DEBUG:
DEBUG:  Setting route to vpn server...
DEBUG:  ip route show to 221.224.34.184/255.255.255.255 dev ppp0
DEBUG:  /sbin/route -n add -net 221.224.34.184 -netmask 255.255.255.255 -interface ppp0
add net 221.224.34.184: gateway ppp0
DEBUG:  Deleting the current default route...
DEBUG:  /sbin/route -n delete 0.0.0.0 -netmask 0.0.0.0
delete net 0.0.0.0
DEBUG:  Setting new default route...
DEBUG:  /sbin/route -n add -net 0.0.0.0 -netmask 0.0.0.0 -interface ppp0
add net 0.0.0.0: gateway ppp0
INFO:   Tunnel is up and running.
DEBUG:  pppd ---> gateway (205 bytes)
DEBUG:  pppd ---> gateway (94 bytes)
DEBUG:  pppd ---> gateway (68 bytes)
DEBUG:  pppd ---> gateway (94 bytes)
DEBUG:  pppd ---> gateway (91 bytes)
DEBUG:  pppd ---> gateway (73 bytes)
DEBUG:  pppd ---> gateway (68 bytes)
DEBUG:  pppd ---> gateway (67 bytes)
DEBUG:  pppd ---> gateway (67 bytes)
DEBUG:  pppd ---> gateway (74 bytes)
DEBUG:  pppd ---> gateway (71 bytes)
DEBUG:  pppd ---> gateway (85 bytes)
DEBUG:  pppd ---> gateway (205 bytes)
DEBUG:  pppd ---> gateway (65 bytes)
DEBUG:  pppd ---> gateway (59 bytes)
DEBUG:  pppd ---> gateway (62 bytes)
DEBUG:  pppd ---> gateway (73 bytes)
DEBUG:  pppd ---> gateway (205 bytes)
DEBUG:  Error reading from SSL connection (Protocol violation with EOF).
INFO:   Cancelling threads...
DEBUG:  disconnecting
INFO:   Setting ppp interface down.
INFO:   Restoring routes...
DEBUG:  /sbin/route -n delete 221.224.34.184 -netmask 255.255.255.255
delete net 221.224.34.184
DEBUG:  /sbin/route -n delete 0.0.0.0 -netmask 0.0.0.0
delete net 0.0.0.0
DEBUG:  /sbin/route -n add -net 0.0.0.0 -netmask 0.0.0.0 -interface ppp0
add net 0.0.0.0: gateway ppp0
DEBUG:  Waiting for pppd to exit...
DEBUG:  waitpid: pppd exit status code 16
INFO:   pppd: The link was terminated by the modem hanging up.
INFO:   Terminated pppd.
INFO:   Closed connection to gateway.
DEBUG:  server_addr: 221.224.34.184
DEBUG:  server_port: 443
DEBUG:  gateway_addr: 221.224.34.184
DEBUG:  gateway_port: 443
ERROR:  connect: Network is unreachable
INFO:   Could not log out.
mrbaseman commented 5 years ago

Ok, "something" is adding additional routes (e.g. the "default" route through ppp0 here). Maybe the kernel itself adds this, or netstat puts additional information into its output (unfortunately we have to rely on this because we don't have the routing table in the /proc file system as on linux). I'll write some code that filters out "unwanted" entries. In this case we don't want routing entries that use the ppp0 interface, when we want to backup the default route. Also when getting the route to the vpn gateway, we don't want to see the additional route through the tunnel.

mrbaseman commented 5 years ago

can you test again with the current mac_routing branch? I have implemented the above described filtering code for the routing table entries (hoping that it has the desired effect).

demonlj commented 5 years ago

fantastic！ It works

$ netstat -nrt
Routing tables

Internet:
Destination        Gateway            Flags        Refs      Use   Netif Expire
default            ppp0               USc             0        0    ppp0
default            link#15            UCSI           81        0    ppp0
8.8.8.8            link#15            UHWIi           1        1    ppp0
17.248.156.8       link#15            UHWIi           1       17    ppp0
17.252.156.60      link#15            UHW3I           0        2    ppp0    225
17.252.204.139     link#15            UHWIi           1       28    ppp0
23.2.139.15        link#15            UHW3I           0       28    ppp0    230
42.255.255.255     link#15            UHWIi           1        3    ppp0
45.32.127.201      link#15            UHW3I           0       56    ppp0    275
52.6.69.141        link#15            UHWIi           2       19    ppp0
58.221.28.158      link#15            UHWIi           1       14    ppp0
61.151.224.41      link#15            UHW3I           0        7    ppp0    281
64.233.189.188     link#15            UHWIi           1       10    ppp0
68.180.240.56      link#15            UHWIi           1       15    ppp0
101.226.211.46     link#15            UHWIi           1        8    ppp0
101.227.169.159    link#15            UHWIi           1        6    ppp0
103.204.172.120    link#15            UHWIi           1       14    ppp0
103.244.234.87     link#15            UHW3I           0       40    ppp0    290
103.244.234.94     link#15            UHWIi           2      533    ppp0
104.31.71.251      link#15            UHWIi           2       29    ppp0
114.114.114.114    link#15            UHW3I           0       38    ppp0    314
117.25.157.119     link#15            UHWIi           1        5    ppp0
118.178.135.232    link#15            UHWIi           1       17    ppp0
122.115.55.6       link#15            UHWIi          19     3300    ppp0
127                127.0.0.1          UCS             0        0     lo0
127.0.0.1          127.0.0.1          UH              4    83161     lo0
128.199.107.105    link#15            UHW3I           0       21    ppp0    293
130.14.29.110      link#15            UHWIi           1       21    ppp0
169.254            link#8             UCS             0        0     en1
180.97.33.72       link#15            UHW3I           0       44    ppp0    293
192.0.2.1          42.244.59.36       UH              0        0    ppp0
192.168.1          link#8             UCS             1        0     en1
192.168.1.1/32     link#8             UCS             1        0     en1
192.168.1.1        ec:88:8f:4f:5f:ba  UHLWIir         3       56     en1    864
192.168.1.102/32   link#8             UCS             0        0     en1
192.168.1.255      ff:ff:ff:ff:ff:ff  UHLWbI          0        9     en1
192.168.9.6        link#15            UHW3I           0        0    ppp0    248
203.208.41.34      link#15            UHWIi           2       34    ppp0
211.88.112.61      link#15            UHWIi           1       13    ppp0
220.181.13.217     link#15            UHWIi           3       43    ppp0
220.181.13.232     link#15            UHWIi           1       21    ppp0
220.181.57.216     link#15            UHW3I           0        4    ppp0    253
220.181.72.45      link#15            UHWIi           1       11    ppp0
221.224.34.184/32  192.168.1.1        UGSc            1        0     en1
222.28.170.251     link#15            UHW3I           0        7    ppp0    242
224.0.0/4          link#8             UmCS            2        0     en1
224.0.0/4          link#15            UmCSI           1        0    ppp0
224.0.0.251        1:0:5e:0:0:fb      UHmLWI          0        0     en1
239.255.255.250    1:0:5e:7f:ff:fa    UHmLWI          0       17     en1
239.255.255.250    link#15            UHmW3I          0        5    ppp0    228
255.255.255.255/32 link#8             UCS             0        0     en1
255.255.255.255/32 link#15            UCSI            0        0    ppp0
mrbaseman commented 5 years ago

wow, this is an impressive list of host routes. I have created a pull request so that we get this into the next release.

mrbaseman commented 5 years ago

I assume we can close this. The pull request has been landed on master and the fix will go into the next release. In case you encounter further issues, please open a new ticket.

janmasarik commented 5 years ago

Thanks for looking into this! Is there any rough date estimate for next release @mrbaseman? I believe this may be dealbreaker for some OS X users which won't check currently closed issues.

rkuska commented 5 years ago

FYI: I am running a current master on MacOS (2d317e60f1579717f365b1d5d007985f0c5c9a98) and everything works smooth so far - I had the same problem as author of this issue.

mrbaseman commented 5 years ago

thanks for the feedback. We don't have a date for the next release, yet, but I have opened #391 for this

mrbaseman commented 5 years ago

Version 1.8.0 has been released now. It contains the fix for this issue.