search

dlenski / openconnect

OpenConnect client extended to support Palo Alto Networks' GlobalProtect VPN
679 stars 130 forks source link

Update tunnel address after reconnecting #16

Closed hatsuyuki15 closed 7 years ago

hatsuyuki15 commented 7 years ago

Problem description

I am currently using openconnect to connect to my workplace. However, the internet connection is quite unstable so openconnect occasionally has to reconnect.

The problem is that everytime it reconnects, the server sends back a different ip address while the address of tunnel still stays the same as before. If I manually reassign the tunnel address with "ip address del" and "ip address add" then everything starts working again.

Operating system and openconnect-gp version

openconnect-gp version: OpenConnect version v7.08-1ubuntu5 Using GnuTLS. Features present: PKCS#11, RSA software token, HOTP software token, TOTP software token, Yubikey OATH, System keys, DTLS operating system: Ubuntu 16.04

dlenski commented 7 years ago

Hi @hatsuyuki15 …

The problem is that everytime it reconnects, the server sends back a different ip address while the address of tunnel still stays the same as before.

That is puzzling. I also had this issue, and specifically addressed it in b73d4cf.

When reconnecting, openconnect asks the server to reassign the same address as it had before and then aborts if the server doesn't reassign it the same address as requested.

So openconnect-gp should never allow the IP address to change after reconnecting!

Next step

Take a look at the detailed log produced with openconnect --dump -vvvv. When openconnect reconnects you should see something like the following: openconnect is telling the server to give it the same address as was previously assigned (&preferred-ip=10.19.241.164).

No work to do; sleeping for 10000 ms...
Caller paused the connection
User requested reconnect
POST https://vpn.vpn.com/ssl-vpn/getconfig.esp
SSL negotiation with vpn.vpn.com
Connected to HTTPS on vpn.vpn.com
> POST /ssl-vpn/getconfig.esp HTTP/1.1
> Host: vpn.vpn.com
> User-Agent: PAN GlobalProtect
> X-Pad: 0000000000000000000000000000000000000000000000000
> Content-Type: application/x-www-form-urlencoded
> Content-Length: 271
> 
> client-type=1&protocol-version=p1&app-version=3.0.1-10&os-version=linux-64&clientos=linux-64&hmac-algo=sha1%2cmd5&enc-algo=aes-128-cbc%2caes-256-cbc&preferred-ip=10.19.241.164&authcookie=deadbeefdeadbeefdeadbeef&portal=Portal-X&user=daniel.lenski&domain=company.com
Got HTTP response: HTTP/1.1 200 OK
Server: 
Date: Wed, 05 Apr 2017 15:54:16 GMT

Does your log show this as well? Please include as much of the detailed log as possible, but you should be sure to anonymize the authcookie and any passwords that appear in it.

hatsuyuki15 commented 7 years ago

Hi, there is no &preferred-ip= in the reconnect attempt. Here is the log.

POST https://vpn.company.com/ssl-vpn/login.esp
Attempting to connect to server vpn.company.com:443
Connected to vpn.company.com:443
SSL negotiation with vpn.company.com
Server certificate verify failed: signer not found
Connected to HTTPS on vpn.company.com
> POST /ssl-vpn/login.esp HTTP/1.1
> Host: vpn.company.com
> User-Agent: PAN GlobalProtect
> X-Pad: 00000000000000000000000000000000000000000000000000
> Content-Type: application/x-www-form-urlencoded
> Content-Length: 142
> 
> jnlpReady=jnlpReady&ok=Login&direct=yes&clientVer=4100&prot=https:&server=vpn.company.com&computer=G750JW&user=bobby&passwd=password
Got HTTP response: HTTP/1.1 200 OK
Server: 
Date: Wed, 05 Apr 2017 16:21:58 GMT
Content-Type: application/xml; charset=UTF-8
Content-Length: 600
Connection: keep-alive
ETag: "27cf3-1e54-57e5e2c0"
Pragma: no-cache
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Expires: Thu, 19 Nov 1981 08:52:00 GMT
X-FRAME-OPTIONS: DENY
Set-Cookie: PHPSESSID=180ab0d8f74794d0dfe3521e4931f0af; secure; HttpOnly
Set-Cookie: PHPSESSID=180ab0d8f74794d0dfe3521e4931f0af; secure; HttpOnly
Set-Cookie: PHPSESSID=180ab0d8f74794d0dfe3521e4931f0af; secure; HttpOnly
Set-Cookie: PHPSESSID=180ab0d8f74794d0dfe3521e4931f0af; secure; HttpOnly
Set-Cookie: PHPSESSID=180ab0d8f74794d0dfe3521e4931f0af; secure; HttpOnly
Set-Cookie: PHPSESSID=180ab0d8f74794d0dfe3521e4931f0af; secure; HttpOnly
Set-Cookie: PHPSESSID=180ab0d8f74794d0dfe3521e4931f0af; secure; HttpOnly
Set-Cookie: PHPSESSID=180ab0d8f74794d0dfe3521e4931f0af; secure; HttpOnly
Set-Cookie: PHPSESSID=180ab0d8f74794d0dfe3521e4931f0af; secure; HttpOnly
Set-Cookie: PHPSESSID=180ab0d8f74794d0dfe3521e4931f0af; secure; HttpOnly
Set-Cookie: PHPSESSID=180ab0d8f74794d0dfe3521e4931f0af; secure; HttpOnly
Set-Cookie: PHPSESSID=180ab0d8f74794d0dfe3521e4931f0af; secure; HttpOnly
Set-Cookie: PHPSESSID=180ab0d8f74794d0dfe3521e4931f0af; secure; HttpOnly
Set-Cookie: PHPSESSID=180ab0d8f74794d0dfe3521e4931f0af; secure; HttpOnly
Set-Cookie: PHPSESSID=180ab0d8f74794d0dfe3521e4931f0af; secure; HttpOnly
Set-Cookie: PHPSESSID=180ab0d8f74794d0dfe3521e4931f0af; secure; HttpOnly
Set-Cookie: PHPSESSID=180ab0d8f74794d0dfe3521e4931f0af; secure; HttpOnly
Set-Cookie: PHPSESSID=180ab0d8f74794d0dfe3521e4931f0af; secure; HttpOnly
Set-Cookie: PHPSESSID=180ab0d8f74794d0dfe3521e4931f0af; secure; HttpOnly
Set-Cookie: PHPSESSID=180ab0d8f74794d0dfe3521e4931f0af; secure; HttpOnly
Set-Cookie: PHPSESSID=180ab0d8f74794d0dfe3521e4931f0af; secure; HttpOnly
Set-Cookie: PHPSESSID=180ab0d8f74794d0dfe3521e4931f0af; secure; HttpOnly
HTTP body length:  (600)
< <?xml version="1.0" encoding="utf-8"?><jnlp><application-desc><argument>(null)</argument><argument>some-cookie</argument><argument>25a423eb95eb51da5134c8e816684ba7413d12b5</argument><argument>External-GW-N</argument><argument>bobby</argument><argument>LDAP_Auth</argument><argument>vsys1</argument><argument>vnavn</argument><argument>(null)</argument><argument></argument><argument></argument><argument></argument><argument>tunnel</argument><argument>-1</argument><argument>4100</argument><argument></argument><argument></argument><argument></argument></application-desc></jnlp>
GlobalProtect login returned authentication source=LDAP_Auth
POST https://vpn.company.com/ssl-vpn/getconfig.esp
> POST /ssl-vpn/getconfig.esp HTTP/1.1
> Host: vpn.company.com
> User-Agent: PAN GlobalProtect
> Cookie: PHPSESSID=180ab0d8f74794d0dfe3521e4931f0af
> X-Pad: 000000000000000000
> Content-Type: application/x-www-form-urlencoded
> Content-Length: 238
> 
> client-type=1&protocol-version=p1&app-version=3.0.1-10&os-version=linux-64&clientos=linux-64&hmac-algo=sha1%2cmd5&enc-algo=aes-128-cbc%2caes-256-cbc&authcookie=some-cookie&portal=External-GW-N&user=bobby&domain=vnavn
Got HTTP response: HTTP/1.1 200 OK
Server: 
Date: Wed, 05 Apr 2017 16:21:58 GMT
Content-Type: application/xml; charset=UTF-8
Content-Length: 1817
Connection: keep-alive
ETag: "27ced-1f2-57e5e2c0"
Pragma: no-cache
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Expires: Thu, 19 Nov 1981 08:52:00 GMT
X-FRAME-OPTIONS: DENY
HTTP body length:  (1817)
< 
<   <response status="success">
<       <need-tunnel>yes</need-tunnel>
<       <ssl-tunnel-url>/ssl-tunnel-connect.sslvpn</ssl-tunnel-url>
<       <portal>External-GW-N</portal>
<       <user>bobby</user>
<       <lifetime>2592000</lifetime>
<       <timeout>10800</timeout>
<       <disconnect-on-idle>10800</disconnect-on-idle>
<       <bw-c2s>1000</bw-c2s>
<       <bw-s2c>1000</bw-s2c>
<       <gw-address>vpn.company.com</gw-address>
<       <ip-address>192.168.241.0</ip-address>
<       <netmask>255.255.255.255</netmask>
<       <dns>
<       </dns> 
<       <wins>
<       </wins> 
<       <default-gateway>192.168.241.0</default-gateway>
<       <mtu>0</mtu>
<       <dns-suffix>
<       </dns-suffix> 
<       <no-direct-access-to-local-network>no</no-direct-access-to-local-network>
<       <access-routes>
<           <member>10.1.5.0/24</member>
<           <member>10.1.6.0/24</member>
<           <member>10.1.6.104/32</member>
<           <member>10.1.6.115/32</member>
<           <member>10.1.10.0/24</member>
<           <member>10.1.11.0/24</member>
<           <member>10.1.11.163/32</member>
<           <member>10.1.64.74/32</member>
<           <member>10.1.67.236/32</member>
<           <member>10.1.77.82/32</member>
<           <member>10.1.88.153/32</member>
<           <member>10.1.89.1/32</member>
<           <member>10.1.89.12/32</member>
<           <member>10.1.120.0/24</member>
<           <member>10.96.10.20/32</member>
<           <member>10.96.10.20/32</member>
<           <member>57.6.17.107/32</member>
<           <member>57.233.33.13/32</member>
<           <member>151.193.51.13/32</member>
<           <member>151.193.52.22/32</member>
<           <member>151.193.52.23/32</member>
<           <member>151.193.52.48/32</member>
<           <member>151.193.52.76/32</member>
<           <member>151.193.52.83/32</member>
<           <member>151.193.54.115/32</member>
<           <member>151.193.54.119/32</member>
<           <member>151.193.59.51/32</member>
<           <member>151.193.141.249/32</member>
<           <member>151.193.141.253/32</member>
<           <member>151.193.141.254/32</member>
<           <member>10.1.5.171/32</member>
<       </access-routes> 
<   </response>
TCP_INFO rcv mss 1452, snd mss 1452, adv mss 1460, pmtu 1500
Connecting to HTTPS tunnel endpoint ...
> GET /ssl-tunnel-connect.sslvpn?authcookie=some-cookie&portal=External-GW-N&user=bobby&domain=vnavn HTTP/1.1
> 
Connected as 192.168.241.0, using SSL
Sending data packet of 48 bytes
No work to do; sleeping for 10000 ms...
Sending data packet of 48 bytes
No work to do; sleeping for 6000 ms...
Sending data packet of 48 bytes
No work to do; sleeping for 1000 ms...
Send GPST DPD/keepalive request
No work to do; sleeping for 5000 ms...
Got GPST DPD/keepalive response
No work to do; sleeping for 10000 ms...
Send GPST DPD/keepalive request
No work to do; sleeping for 5000 ms...
Send GPST DPD/keepalive request
No work to do; sleeping for 5000 ms...
Send GPST DPD/keepalive request
No work to do; sleeping for 5000 ms...
POST https://vpn.company.com/ssl-vpn/getconfig.esp
Connecting to HTTPS tunnel endpoint ...
sleep 10s, remaining timeout 300s
POST https://vpn.company.com/ssl-vpn/getconfig.esp
SSL negotiation with vpn.company.com
Server certificate verify failed: signer not found
Connected to HTTPS on vpn.company.com
> POST /ssl-vpn/getconfig.esp HTTP/1.1
> Host: vpn.company.com
> User-Agent: PAN GlobalProtect
> Cookie: PHPSESSID=180ab0d8f74794d0dfe3521e4931f0af
> X-Pad: 000000000000000000
> Content-Type: application/x-www-form-urlencoded
> Content-Length: 238
> 
> client-type=1&protocol-version=p1&app-version=3.0.1-10&os-version=linux-64&clientos=linux-64&hmac-algo=sha1%2cmd5&enc-algo=aes-128-cbc%2caes-256-cbc&authcookie=some-cookie&portal=External-GW-N&user=bobby&domain=vnavn
Got HTTP response: HTTP/1.1 200 OK
Server: 
Date: Wed, 05 Apr 2017 16:22:42 GMT
Content-Type: application/xml; charset=UTF-8
Content-Length: 1817
Connection: keep-alive
ETag: "27ced-1f2-57e5e2c0"
Pragma: no-cache
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Expires: Thu, 19 Nov 1981 08:52:00 GMT
X-FRAME-OPTIONS: DENY
HTTP body length:  (1817)
< 
<   <response status="success">
<       <need-tunnel>yes</need-tunnel>
<       <ssl-tunnel-url>/ssl-tunnel-connect.sslvpn</ssl-tunnel-url>
<       <portal>External-GW-N</portal>
<       <user>bobby</user>
<       <lifetime>2591956</lifetime>
<       <timeout>10800</timeout>
<       <disconnect-on-idle>10800</disconnect-on-idle>
<       <bw-c2s>1000</bw-c2s>
<       <bw-s2c>1000</bw-s2c>
<       <gw-address>vpn.company.com</gw-address>
<       <ip-address>192.168.241.1</ip-address>
<       <netmask>255.255.255.255</netmask>
<       <dns>
<       </dns> 
<       <wins>
<       </wins> 
<       <default-gateway>192.168.241.1</default-gateway>
<       <mtu>0</mtu>
<       <dns-suffix>
<       </dns-suffix> 
<       <no-direct-access-to-local-network>no</no-direct-access-to-local-network>
<       <access-routes>
<           <member>10.1.5.0/24</member>
<           <member>10.1.6.0/24</member>
<           <member>10.1.6.104/32</member>
<           <member>10.1.6.115/32</member>
<           <member>10.1.10.0/24</member>
<           <member>10.1.11.0/24</member>
<           <member>10.1.11.163/32</member>
<           <member>10.1.64.74/32</member>
<           <member>10.1.67.236/32</member>
<           <member>10.1.77.82/32</member>
<           <member>10.1.88.153/32</member>
<           <member>10.1.89.1/32</member>
<           <member>10.1.89.12/32</member>
<           <member>10.1.120.0/24</member>
<           <member>10.96.10.20/32</member>
<           <member>10.96.10.20/32</member>
<           <member>57.6.17.107/32</member>
<           <member>57.233.33.13/32</member>
<           <member>151.193.51.13/32</member>
<           <member>151.193.52.22/32</member>
<           <member>151.193.52.23/32</member>
<           <member>151.193.52.48/32</member>
<           <member>151.193.52.76/32</member>
<           <member>151.193.52.83/32</member>
<           <member>151.193.54.115/32</member>
<           <member>151.193.54.119/32</member>
<           <member>151.193.59.51/32</member>
<           <member>151.193.141.249/32</member>
<           <member>151.193.141.253/32</member>
<           <member>151.193.141.254/32</member>
<           <member>10.1.5.171/32</member>
<       </access-routes> 
<   </response>
TCP_INFO rcv mss 1452, snd mss 1452, adv mss 1460, pmtu 1500
Connecting to HTTPS tunnel endpoint ...
> GET /ssl-tunnel-connect.sslvpn?authcookie=some-cookie&portal=External-GW-N&user=bobby&domain=vnavn HTTP/1.1
> 
No work to do; sleeping for 10000 ms...
Send GPST DPD/keepalive request
No work to do; sleeping for 5000 ms...
Got GPST DPD/keepalive response
No work to do; sleeping for 10000 ms...
Send GPST DPD/keepalive request
No work to do; sleeping for 5000 ms...
Got GPST DPD/keepalive response
No work to do; sleeping for 10000 ms...
Send GPST DPD/keepalive request
No work to do; sleeping for 5000 ms...
Got GPST DPD/keepalive response
No work to do; sleeping for 10000 ms...
Send GPST DPD/keepalive request
No work to do; sleeping for 5000 ms...
Got GPST DPD/keepalive response
No work to do; sleeping for 10000 ms...
dlenski commented 7 years ago

Is this log complete, or did you omit some messages in the middle about the dropped/timed-out connection?

… Send GPST DPD/keepalive request No work to do; sleeping for 5000 ms... … ??? where does the connection get dropped ??? … POST https://vpn.company.com/ssl-vpn/getconfig.esp Connecting to HTTPS tunnel endpoint ... sleep 10s, remaining timeout 300s POST https://vpn.company.com/ssl-vpn/getconfig.esp SSL negotiation with vpn.company.com Server certificate verify failed: signer not found Connected to HTTPS on vpn.company.com …

This is important to include, because maybe there's a code path for reconnection which I haven't considered.

hatsuyuki15 commented 7 years ago

Oops, I used tee to redirect the output to file. Seems like it omitted some lines. Here is the full log.

POST https://123.123.123.123/ssl-vpn/login.esp
Attempting to connect to server 123.123.123.123:443
Connected to 123.123.123.123:443
SSL negotiation with 123.123.123.123
Server certificate verify failed: signer not found
Connected to HTTPS on 123.123.123.123
> POST /ssl-vpn/login.esp HTTP/1.1
> Host: 123.123.123.123
> User-Agent: PAN GlobalProtect
> X-Pad: 00000000000000000000000000000000000000000000000000
> Content-Type: application/x-www-form-urlencoded
> Content-Length: 142
> 
> jnlpReady=jnlpReady&ok=Login&direct=yes&clientVer=4100&prot=https:&server=123.123.123.123&computer=G750JW&user=bobby&passwd=password
Got HTTP response: HTTP/1.1 200 OK
Server: 
Date: Wed, 05 Apr 2017 17:28:39 GMT
Content-Type: application/xml; charset=UTF-8
Content-Length: 600
Connection: keep-alive
ETag: "27cf3-1e54-57e5e2c0"
Pragma: no-cache
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Expires: Thu, 19 Nov 1981 08:52:00 GMT
X-FRAME-OPTIONS: DENY
Set-Cookie: PHPSESSID=b1663cab56702ccb980dc5ad3b770b38; secure; HttpOnly
Set-Cookie: PHPSESSID=b1663cab56702ccb980dc5ad3b770b38; secure; HttpOnly
Set-Cookie: PHPSESSID=b1663cab56702ccb980dc5ad3b770b38; secure; HttpOnly
Set-Cookie: PHPSESSID=b1663cab56702ccb980dc5ad3b770b38; secure; HttpOnly
Set-Cookie: PHPSESSID=b1663cab56702ccb980dc5ad3b770b38; secure; HttpOnly
Set-Cookie: PHPSESSID=b1663cab56702ccb980dc5ad3b770b38; secure; HttpOnly
Set-Cookie: PHPSESSID=b1663cab56702ccb980dc5ad3b770b38; secure; HttpOnly
Set-Cookie: PHPSESSID=b1663cab56702ccb980dc5ad3b770b38; secure; HttpOnly
Set-Cookie: PHPSESSID=b1663cab56702ccb980dc5ad3b770b38; secure; HttpOnly
Set-Cookie: PHPSESSID=b1663cab56702ccb980dc5ad3b770b38; secure; HttpOnly
Set-Cookie: PHPSESSID=b1663cab56702ccb980dc5ad3b770b38; secure; HttpOnly
Set-Cookie: PHPSESSID=b1663cab56702ccb980dc5ad3b770b38; secure; HttpOnly
Set-Cookie: PHPSESSID=b1663cab56702ccb980dc5ad3b770b38; secure; HttpOnly
Set-Cookie: PHPSESSID=b1663cab56702ccb980dc5ad3b770b38; secure; HttpOnly
Set-Cookie: PHPSESSID=b1663cab56702ccb980dc5ad3b770b38; secure; HttpOnly
Set-Cookie: PHPSESSID=b1663cab56702ccb980dc5ad3b770b38; secure; HttpOnly
Set-Cookie: PHPSESSID=b1663cab56702ccb980dc5ad3b770b38; secure; HttpOnly
Set-Cookie: PHPSESSID=b1663cab56702ccb980dc5ad3b770b38; secure; HttpOnly
Set-Cookie: PHPSESSID=b1663cab56702ccb980dc5ad3b770b38; secure; HttpOnly
Set-Cookie: PHPSESSID=b1663cab56702ccb980dc5ad3b770b38; secure; HttpOnly
Set-Cookie: PHPSESSID=b1663cab56702ccb980dc5ad3b770b38; secure; HttpOnly
Set-Cookie: PHPSESSID=b1663cab56702ccb980dc5ad3b770b38; secure; HttpOnly
HTTP body length:  (600)
< <?xml version="1.0" encoding="utf-8"?><jnlp><application-desc><argument>(null)</argument><argument>some-cookies</argument><argument>25a423eb95eb51da5134c8e816684ba7413d12b5</argument><argument>External-GW-N</argument><argument>bobby</argument><argument>LDAP_Auth</argument><argument>vsys1</argument><argument>vnavn</argument><argument>(null)</argument><argument></argument><argument></argument><argument></argument><argument>tunnel</argument><argument>-1</argument><argument>4100</argument><argument></argument><argument></argument><argument></argument></application-desc></jnlp>
GlobalProtect login returned authentication source=LDAP_Auth
POST https://123.123.123.123/ssl-vpn/getconfig.esp
> POST /ssl-vpn/getconfig.esp HTTP/1.1
> Host: 123.123.123.123
> User-Agent: PAN GlobalProtect
> Cookie: PHPSESSID=b1663cab56702ccb980dc5ad3b770b38
> X-Pad: 000000000000000000
> Content-Type: application/x-www-form-urlencoded
> Content-Length: 238
> 
> client-type=1&protocol-version=p1&app-version=3.0.1-10&os-version=linux-64&clientos=linux-64&hmac-algo=sha1%2cmd5&enc-algo=aes-128-cbc%2caes-256-cbc&authcookie=some-cookies&portal=External-GW-N&user=bobby&domain=vnavn
Got HTTP response: HTTP/1.1 200 OK
Server: 
Date: Wed, 05 Apr 2017 17:28:39 GMT
Content-Type: application/xml; charset=UTF-8
Content-Length: 1817
Connection: keep-alive
ETag: "27ced-1f2-57e5e2c0"
Pragma: no-cache
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Expires: Thu, 19 Nov 1981 08:52:00 GMT
X-FRAME-OPTIONS: DENY
HTTP body length:  (1817)
< 
<   <response status="success">
<       <need-tunnel>yes</need-tunnel>
<       <ssl-tunnel-url>/ssl-tunnel-connect.sslvpn</ssl-tunnel-url>
<       <portal>External-GW-N</portal>
<       <user>bobby</user>
<       <lifetime>2592000</lifetime>
<       <timeout>10800</timeout>
<       <disconnect-on-idle>10800</disconnect-on-idle>
<       <bw-c2s>1000</bw-c2s>
<       <bw-s2c>1000</bw-s2c>
<       <gw-address>123.123.123.123</gw-address>
<       <ip-address>192.168.241.4</ip-address>
<       <netmask>255.255.255.255</netmask>
<       <dns>
<       </dns> 
<       <wins>
<       </wins> 
<       <default-gateway>192.168.241.4</default-gateway>
<       <mtu>0</mtu>
<       <dns-suffix>
<       </dns-suffix> 
<       <no-direct-access-to-local-network>no</no-direct-access-to-local-network>
<       <access-routes>
<           <member>10.1.5.0/24</member>
<           <member>10.1.6.0/24</member>
<           <member>10.1.6.104/32</member>
<           <member>10.1.6.115/32</member>
<           <member>10.1.10.0/24</member>
<           <member>10.1.11.0/24</member>
<           <member>10.1.11.163/32</member>
<           <member>10.1.64.74/32</member>
<           <member>10.1.67.236/32</member>
<           <member>10.1.77.82/32</member>
<           <member>10.1.88.153/32</member>
<           <member>10.1.89.1/32</member>
<           <member>10.1.89.12/32</member>
<           <member>10.1.120.0/24</member>
<           <member>10.96.10.20/32</member>
<           <member>10.96.10.20/32</member>
<           <member>57.6.17.107/32</member>
<           <member>57.233.33.13/32</member>
<           <member>151.193.51.13/32</member>
<           <member>151.193.52.22/32</member>
<           <member>151.193.52.23/32</member>
<           <member>151.193.52.48/32</member>
<           <member>151.193.52.76/32</member>
<           <member>151.193.52.83/32</member>
<           <member>151.193.54.115/32</member>
<           <member>151.193.54.119/32</member>
<           <member>151.193.59.51/32</member>
<           <member>151.193.141.249/32</member>
<           <member>151.193.141.253/32</member>
<           <member>151.193.141.254/32</member>
<           <member>10.1.5.171/32</member>
<       </access-routes> 
<   </response>
TCP_INFO rcv mss 1452, snd mss 1452, adv mss 1460, pmtu 1500
No MTU received. Calculated 1410
Connecting to HTTPS tunnel endpoint ...
> GET /ssl-tunnel-connect.sslvpn?authcookie=some-cookies&portal=External-GW-N&user=bobby&domain=vnavn HTTP/1.1
> 
Set up DTLS failed; using SSL instead
Connected as 192.168.241.4, using SSL
Sending data packet of 48 bytes
No work to do; sleeping for 10000 ms...
Sending data packet of 48 bytes
No work to do; sleeping for 6000 ms...
Sending data packet of 48 bytes
No work to do; sleeping for 1000 ms...
Send GPST DPD/keepalive request
No work to do; sleeping for 5000 ms...
Send GPST DPD/keepalive request
No work to do; sleeping for 5000 ms...
Send GPST DPD/keepalive request
No work to do; sleeping for 5000 ms...
GPST Dead Peer Detection detected dead peer!
POST https://123.123.123.123/ssl-vpn/getconfig.esp
Failed to reconnect to host 123.123.123.123: Network is unreachable
Failed to open HTTPS connection to 123.123.123.123
Connecting to HTTPS tunnel endpoint ...
Failed to reconnect to host 123.123.123.123: Network is unreachable
sleep 10s, remaining timeout 300s
POST https://123.123.123.123/ssl-vpn/getconfig.esp
Failed to reconnect to host 123.123.123.123: Network is unreachable
Failed to open HTTPS connection to 123.123.123.123
Failed to reconnect to host 123.123.123.123: Network is unreachable
Connecting to HTTPS tunnel endpoint ...
sleep 20s, remaining timeout 290s
POST https://123.123.123.123/ssl-vpn/getconfig.esp
SSL negotiation with 123.123.123.123
Server certificate verify failed: signer not found
Connected to HTTPS on 123.123.123.123
> POST /ssl-vpn/getconfig.esp HTTP/1.1
> Host: 123.123.123.123
> User-Agent: PAN GlobalProtect
> Cookie: PHPSESSID=b1663cab56702ccb980dc5ad3b770b38
> X-Pad: 000000000000000000
> Content-Type: application/x-www-form-urlencoded
> Content-Length: 238
> 
> client-type=1&protocol-version=p1&app-version=3.0.1-10&os-version=linux-64&clientos=linux-64&hmac-algo=sha1%2cmd5&enc-algo=aes-128-cbc%2caes-256-cbc&authcookie=some-cookies&portal=External-GW-N&user=bobby&domain=vnavn
Got HTTP response: HTTP/1.1 200 OK
Server: 
Date: Wed, 05 Apr 2017 17:29:33 GMT
Content-Type: application/xml; charset=UTF-8
Content-Length: 1817
Connection: keep-alive
ETag: "27ced-1f2-57e5e2c0"
Pragma: no-cache
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Expires: Thu, 19 Nov 1981 08:52:00 GMT
X-FRAME-OPTIONS: DENY
HTTP body length:  (1817)
< 
<   <response status="success">
<       <need-tunnel>yes</need-tunnel>
<       <ssl-tunnel-url>/ssl-tunnel-connect.sslvpn</ssl-tunnel-url>
<       <portal>External-GW-N</portal>
<       <user>bobby</user>
<       <lifetime>2591946</lifetime>
<       <timeout>10800</timeout>
<       <disconnect-on-idle>10800</disconnect-on-idle>
<       <bw-c2s>1000</bw-c2s>
<       <bw-s2c>1000</bw-s2c>
<       <gw-address>123.123.123.123</gw-address>
<       <ip-address>192.168.241.5</ip-address>
<       <netmask>255.255.255.255</netmask>
<       <dns>
<       </dns> 
<       <wins>
<       </wins> 
<       <default-gateway>192.168.241.5</default-gateway>
<       <mtu>0</mtu>
<       <dns-suffix>
<       </dns-suffix> 
<       <no-direct-access-to-local-network>no</no-direct-access-to-local-network>
<       <access-routes>
<           <member>10.1.5.0/24</member>
<           <member>10.1.6.0/24</member>
<           <member>10.1.6.104/32</member>
<           <member>10.1.6.115/32</member>
<           <member>10.1.10.0/24</member>
<           <member>10.1.11.0/24</member>
<           <member>10.1.11.163/32</member>
<           <member>10.1.64.74/32</member>
<           <member>10.1.67.236/32</member>
<           <member>10.1.77.82/32</member>
<           <member>10.1.88.153/32</member>
<           <member>10.1.89.1/32</member>
<           <member>10.1.89.12/32</member>
<           <member>10.1.120.0/24</member>
<           <member>10.96.10.20/32</member>
<           <member>10.96.10.20/32</member>
<           <member>57.6.17.107/32</member>
<           <member>57.233.33.13/32</member>
<           <member>151.193.51.13/32</member>
<           <member>151.193.52.22/32</member>
<           <member>151.193.52.23/32</member>
<           <member>151.193.52.48/32</member>
<           <member>151.193.52.76/32</member>
<           <member>151.193.52.83/32</member>
<           <member>151.193.54.115/32</member>
<           <member>151.193.54.119/32</member>
<           <member>151.193.59.51/32</member>
<           <member>151.193.141.249/32</member>
<           <member>151.193.141.253/32</member>
<           <member>151.193.141.254/32</member>
<           <member>10.1.5.171/32</member>
<       </access-routes> 
<   </response>
No MTU received. Calculated 1410
TCP_INFO rcv mss 1452, snd mss 1452, adv mss 1460, pmtu 1500
Connecting to HTTPS tunnel endpoint ...
> GET /ssl-tunnel-connect.sslvpn?authcookie=some-cookies&portal=External-GW-N&user=bobby&domain=vnavn HTTP/1.1
> 
No work to do; sleeping for 10000 ms...
dlenski commented 7 years ago

Huh… unfortunately, I'm having trouble reproducing this behavior.

…
GPST Dead Peer Detection detected dead peer!
POST https://123.123.123.123/ssl-vpn/getconfig.esp
Failed to reconnect to host 123.123.123.123: Network is unreachable
Failed to open HTTPS connection to 123.123.123.123
Connecting to HTTPS tunnel endpoint ...
Failed to reconnect to host 123.123.123.123: Network is unreachable
sleep 10s, remaining timeout 300s
POST https://123.123.123.123/ssl-vpn/getconfig.esp
Failed to reconnect to host 123.123.123.123: Network is unreachable
Failed to open HTTPS connection to 123.123.123.123
Failed to reconnect to host 123.123.123.123: Network is unreachable
Connecting to HTTPS tunnel endpoint ...
sleep 20s, remaining timeout 290s
POST https://123.123.123.123/ssl-vpn/getconfig.esp
SSL negotiation with 123.123.123.123
Server certificate verify failed: signer not found
Connected to HTTPS on 123.123.123.123
…

I am trying to simulate it by blocking connections my VPN's host until it times out and openconnect tries to reconnect. It always correctly appends the &preferred-ip=X parameter.

Could you try building the latest version of this repo from source code, rather than running the slightly out-of-date Ubuntu/.deb package? (I think I include the aforementioned patches in the packaged version, but not 100% sure.)

hatsuyuki15 commented 7 years ago

After a bit more fiddling around:

Then the problem occurs, preferred-ip= is not included in the request.

Here are the logs log.iptables-drop.txt log.iptables-reject.txt log.network-manager.txt

I also tried building from the lastest source code but the same problem still happened.

dlenski commented 7 years ago

Very good find on how to reproduce it, @hatsuyuki15.

I just found the bug:

  1. if the config XML has to be re-requested multiple times before succeeding
  2. … openconnect would forget the previous IP on the first request
  3. … and then would stop requesting that same IP on subsequent requests

The solution is not to forget the previous IP until we're sure we've successfully fetched the new config. It should be fixed by c045c0b601acc1.

markyoder commented 7 years ago

yeah, i'm still seeing what i think is the same problem. my status is as follows:

POST https://******.********.com/ssl-vpn/login.esp
Connected to xxx.xxx.xxx.xxx:443
SSL negotiation with ******.*******.com
Connected to HTTPS on ******.*******.com
GlobalProtect login returned authentication source=********-LDAP-Auth-Profile
POST https://******.*******.com/ssl-vpn/getconfig.esp
No MTU received. Calculated 1410
Set up DTLS failed; using SSL instead
Connected as 192.168.40.42, using SSL
GPST Dead Peer Detection detected dead peer!
POST https://******.*******.com/ssl-vpn/getconfig.esp

if i hotspot to my phone and shut off the phone's wifi, the connection is stable.

dlenski commented 7 years ago

@markyoder, what happens after it reconnects? Does the network then continue working?

POST https://******.********.com/ssl-vpn/login.esp
Connected to xxx.xxx.xxx.xxx:443
SSL negotiation with ******.*******.com
Connected to HTTPS on ******.*******.com
GlobalProtect login returned authentication source=********-LDAP-Auth-Profile
POST https://******.*******.com/ssl-vpn/getconfig.esp
No MTU received. Calculated 1410
Set up DTLS failed; using SSL instead
Connected as 192.168.40.42, using SSL
GPST Dead Peer Detection detected dead peer!
POST https://******.*******.com/ssl-vpn/getconfig.esp

Are you running openconnect --no-dtls, or is ESP just not working? Can you post a log with openconnect --dump -vvvv to show the details of the config?

markyoder commented 7 years ago

I confess that my network and vpn-foo is a bit weak, so I would be most excited to learn that I can pass an option that will fix this. I am not running with --no-dtls; i assumed that ESP is just not working. i expect that i could also improve some aspects of performance by modifying my routing table(s). in any case, here's a more detailed log, with a bit of masking over potentially sensitive areas:

dlenski commented 7 years ago

@markyoder, please don't use email responses for logs like these. (Email responses don't support Markdown so the pre-formatted sections are pretty much unreadable.) R Moving your log here…

>>>>
m*****@Blixen ~/openconnect2 $ sudo ./openconnect --protocol gp --dump -vvv
*****vpn.*****.com
[sudo] password for m*****:
Please enter your username and password
Username: m*****
Password:
POST https://*****vpn.*****.com/ssl-vpn/login.esp
Attempting to connect to server 207.223.39.245:443
Connected to 207.223.39.245:443
SSL negotiation with *****vpn.*****.com
Connected to HTTPS on *****vpn.*****.com
> POST /ssl-vpn/login.esp HTTP/1.1
> Host: *****vpn.*****.com
> User-Agent: PAN GlobalProtect
> X-Pad: 00000000000000000000000000000000000000000000000
> Content-Type: application/x-www-form-urlencoded
> Content-Length: 145
>
>
jnlpReady=jnlpReady&ok=Login&direct=yes&clientVer=4100&prot=https:&server=*****vpn.*****.com&computer=Blixen&user=m*****&passwd=************
Got HTTP response: HTTP/1.1 200 OK
Server:
Date: Fri, 21 Apr 2017 17:19:56 GMT
Content-Type: application/xml; charset=UTF-8
Content-Length: 628
Connection: keep-alive
ETag: "43480-1e54-58a37b58"
Pragma: no-cache
Cache-Control: no-store, no-cache, must-revalidate, post-check=0,
pre-check=0
Expires: Thu, 19 Nov 1981 08:52:00 GMT
X-FRAME-OPTIONS: DENY
Set-Cookie: PHPSESSID=******************************; secure; HttpOnly
HTTP body length:  (628)
< <?xml version="1.0"
encoding="utf-8"?><jnlp><application-desc><argument>(null)</argument><argument>******************************</argument><argument>******************************</argument><argument>*****-GP-Gateway-N</argument><argument>m*****</argument><argument>*****-LDAP-Auth-Profile</argument><argument>vsys1</argument><argument>*****</argument><argument>(null)</argument><argument></argument><argument></argument><argument></argument><argument>tunnel</argument><argument>-1</argument><argument>4100</argument><argument></argument><argument></argument><argument></argument></application-desc></jnlp>
GlobalProtect login returned authentication source=*****-LDAP-Auth-Profile
POST https://*****vpn.*****.com/ssl-vpn/getconfig.esp
> POST /ssl-vpn/getconfig.esp HTTP/1.1
> Host: *****vpn.*****.com
> User-Agent: PAN GlobalProtect
> Cookie: PHPSESSID=******************************
> X-Pad: 0000000
> Content-Type: application/x-www-form-urlencoded
> Content-Length: 249
>
>
client-type=1&protocol-version=p1&app-version=3.0.1-10&os-version=linux-64&clientos=linux-64&hmac-algo=sha1%2cmd5&enc-algo=aes-128-cbc%2caes-256-cbc&authcookie=******************************&portal=*****-GP-Gateway-N&user=m*****&domain=*****
Got HTTP response: HTTP/1.1 200 OK
Server:
Date: Fri, 21 Apr 2017 17:19:56 GMT
Content-Type: application/xml; charset=UTF-8
Content-Length: 2551
Connection: keep-alive
ETag: "4347a-1f2-58a37b58"
Pragma: no-cache
Cache-Control: no-store, no-cache, must-revalidate, post-check=0,
pre-check=0
Expires: Thu, 19 Nov 1981 08:52:00 GMT
X-FRAME-OPTIONS: DENY
HTTP body length:  (2551)
<
< <response status="success">
< <need-tunnel>yes</need-tunnel>
< <ssl-tunnel-url>/ssl-tunnel-connect.sslvpn</ssl-tunnel-url>
< <portal>*****-GP-Gateway-N</portal>
< <user>m*****</user>
< <lifetime>2592000</lifetime>
< <timeout>10800</timeout>
< <disconnect-on-idle>10800</disconnect-on-idle>
< <bw-c2s>1000</bw-c2s>
< <bw-s2c>1000</bw-s2c>
< <gw-address>xxx.xxx.xxx.245</gw-address>
< <ip-address>192.168.40.78</ip-address>
< <netmask>255.255.255.255</netmask>
< <dns>
< <member>192.168.32.108</member>
< <member>10.50.120.221</member>
< </dns>
< <wins>
< </wins>
< <default-gateway>192.168.40.78</default-gateway>
< <mtu>0</mtu>
< <dns-suffix>
< <member>int.*****inc.com</member>
< <member>*****inc.com</member>
< <member>*****.com</member>
< <member>*****mp.com</member>
< </dns-suffix>
< <no-direct-access-to-local-network>no</no-direct-access-to-local-network>
< <access-routes>
< <member>10.0.0.0/24</member>
< <member>10.50.0.0/16</member>
< <member>10.51.0.0/16</member>
< <member>10.60.225.0/24</member>
< <member>10.64.24.0/24</member>
< <member>10.70.10.0/24</member>
< <member>10.70.70.0/23</member>
< <member>10.70.72.0/24</member>
< <member>10.75.13.70/32</member>
< <member>10.75.73.207/32</member>
< <member>10.75.115.88/32</member>
< <member>10.75.176.198/32</member>
< <member>10.75.244.186/32</member>
< <member>10.90.10.0/24</member>
< <member>10.98.2.106/32</member>
< <member>10.186.2.162/32</member>
< <member>10.186.24.199/32</member>
< <member>10.186.32.206/32</member>
< <member>10.186.41.19/32</member>
< <member>10.186.83.44/32</member>
< <member>10.186.146.91/32</member>
< <member>10.186.209.141/32</member>
< <member>10.186.218.83/32</member>
< <member>10.186.231.253/32</member>
< <member>10.196.200.0/24</member>
< <member>10.212.0.0/24</member>
< <member>170.146.0.0/16</member>
< <member>172.16.0.0/16</member>
< <member>192.168.0.0/24</member>
< <member>192.168.10.0/24</member>
< <member>192.168.11.0/24</member>
< <member>192.168.20.0/24</member>
< <member>192.168.32.0/19</member>
< <member>192.168.64.0/19</member>
< <member>192.168.96.0/19</member>
< <member>192.168.128.0/19</member>
< <member>192.168.192.0/21</member>
< <member>192.168.212.0/22</member>
< <member>192.168.224.0/24</member>
< <member>192.168.251.192/26</member>
< <member>205.142.188.0/24</member>
< <member>216.20.235.0/25</member>
< <member>216.20.237.0/25</member>
< <member>192.168.32.108/32</member>
< <member>10.50.120.221/32</member>
< </access-routes>
< </response>
TCP_INFO rcv mss 1460, snd mss 1460, adv mss 1460, pmtu 1500
No MTU received. Calculated 1410
Connecting to HTTPS tunnel endpoint ...
> GET
/ssl-tunnel-connect.sslvpn?authcookie=******************************&portal=*****-GP-Gateway-N&user=m*****&domain=*****
HTTP/1.1
>
Set up DTLS failed; using SSL instead
Connected as 192.168.40.78, using SSL
Sending data packet of 51 bytes
Sending data packet of 74 bytes
Sending data packet of 74 bytes
Sending data packet of 74 bytes
Sending data packet of 74 bytes
Sending data packet of 61 bytes
Sending data packet of 61 bytes
Sending data packet of 76 bytes
Sending data packet of 76 bytes
Sending data packet of 78 bytes
No work to do; sleeping for 8000 ms...
....
....
Sending data packet of 67 bytes
Sending data packet of 67 bytes
No work to do; sleeping for 1000 ms...
Sending data packet of 67 bytes
No work to do; sleeping for 1000 ms...
Send GPST DPD/keepalive request
No work to do; sleeping for 5000 ms...
GPST Dead Peer Detection detected dead peer!
POST https://*****vpn.*****.com/ssl-vpn/getconfig.esp
dlenski commented 7 years ago

@markyoder, your XML config has no <ipsec>…</ipsec> section… so it simply doesn't support ESP. Good to know.

As for your original problem…

if i have my ISP anywhere in the network chain… the VPN becomes unstable, and within a few minutes of use (especially if there is any dead-time), i get something like:

POST https://******.********.com/ssl-vpn/login.esp
Connected to xxx.xxx.xxx.xxx:443
SSL negotiation with ******.*******.com
Connected to HTTPS on ******.*******.com
GlobalProtect login returned authentication source=********-LDAP-Auth-Profile
POST https://******.*******.com/ssl-vpn/getconfig.esp
No MTU received. Calculated 1410
Set up DTLS failed; using SSL instead
Connected as 192.168.40.42, using SSL
GPST Dead Peer Detection detected dead peer!
POST https://******.*******.com/ssl-vpn/getconfig.esp

… there is nothing that OpenConnect can really do about this.

Your underlying network is lossy and we just found out that your VPN only supports the TCP/TLS tunnel, not the UDP/ESP tunnel. Tunneling over TCP does not work when the underlying network is lossy or slow ("Why TCP over TCP is a bad idea") which is why all modern VPNs should support a UDP-based tunnel.

You should complain to your IT department and tell them to configure the GlobalProtect servers for the ESP-based tunnel, not the TCP-based tunnel!!!

You can run with openconnect --force-dpd=5 --reconnect-timeout=30, for example, but all this will do is make openconnect check for a lost connection more frequently, and give up trying to reconnect sooner. ¯\_(ツ)_/¯

markyoder commented 7 years ago

awesome. i'll try that, and take this up with my VPN guys as well as my ISP guys as well.

markyoder commented 7 years ago

one more question: when the VPN fails on the "dead peer" exception, it seems to take all routing with it. again, my vpn-foo is weak, but i believe the connection imports routing instructions from the VPN, so i would think that non-vpn (non-corporate) routing should be unaffected. is it possible that the disconnect corrupts the routing table?

dlenski commented 7 years ago

when the VPN fails on the "dead peer" exception, it seems to take all routing with it.

What do you mean by "take all routing with it"? Can you be more specific?

According to the config you posted above, your VPN does not route 0.0.0.0… so traffic that is not to one of the destinations in the <access-routes> will not be routed through the VPN. (This assumes that you are using the default vpnc-script, which simply follows the server-provided routing configuration.)

<access-routes>
<member>10.0.0.0/24</member>
...

Assuming you're running under Linux: you can confirm this by running ip route while the VPN is connected. This should show you that the default route for IPv4 traffic is via your "normal" network interface (e.g. wlan0 or eth0) rather than through the VPN tunnel device (e.g. tun0).