ERROR: SSL_connect: error:0A000126:SSL routines::unexpected eof while reading, Error happen randomly

leompxp commented 2 weeks ago

Description

A few months ago, I installed OpenFortiVPN to set up an SSL VPN tunnel to my Fortigate 100E. However, I have been encountering an error that appears "randomly":

ERROR:  SSL_connect: error:0A000126:SSL routines::unexpected eof while reading
You might want to try --insecure-ssl or specify a different --cipher-list

The behavior of this error is peculiar as it seems to appear randomly and varies depending on the operating system.

On Ubuntu 24.04 LTS (Noble Numbat) I cannot connect to VPN, the error appear everytime I try to connect

On Ubuntu 22.04.4 LTS (Jammy Jellyfish) I can connect to the VPN most of the time, but occasionally the error prevents me or my colleagues from connecting. Usually, it only appears once, and I can connect on the next attempt, but sometimes it shows up two or three times in a row.

It is really frustrating because this error appears seemingly at random moments.

My configuration

On ubuntu 22.04 : (where it works most of the time, but not always) openfortivpn version : 1.17.1 openfortivpn configuration : pppd version : 2.4.9 login method : certificate on pkcs11 device + password

# openfortivpn configuration file
 host = X.X.X.X
 port = XXXX
 username = username
 user-cert = pkcs11:id=%55
 trusted-cert = e54xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx1

command used to run the vpn : openfortivpn -c /etc/openfortivpn/configuration.conf --pppd-no-peerdns --no-dns

On ubuntu 24.04 : (where it never works) openfortivpn version : 1.21.0 openfortivpn configuration, login method, configuration file, command used to run the vpn : same as above

Debug

Downgrading and compiling version 1.17.1 on Ubuntu 24.04 made it work, but it still generates errors from time to time.

I haven't been able to capture any debug information when it happens randomly (which is quite frequent, approximately once every 5 connection attempts).

I was able to capture debug information from Ubuntu 24.04 and openfortivpn 1.21.0 because it never works.

with -v -v option :

DEBUG:  openfortivpn 1.21.0
DEBUG:  revision unavailable
DEBUG:  Loaded configuration file "/etc/openfortivpn/prod.conf".
VPN account password: 
DEBUG:  Configuration host = "X.X.X.X"
DEBUG:  Configuration realm = ""
DEBUG:  Configuration port = "X.X.X.X"
DEBUG:  Configuration username = "user@enterprise.com"
DEBUG:  Resolving gateway host ip
DEBUG:  Establishing ssl connection
DEBUG:  SO_KEEPALIVE: OFF
DEBUG:  TCP_KEEPIDLE: 7200
DEBUG:  TCP_KEEPINTVL: 75
DEBUG:  TCP_KEEPCNT: 9
DEBUG:  SO_SNDBUF: 16384
DEBUG:  SO_RCVBUF: 131072
DEBUG:  server_addr: X.X.X.X
DEBUG:  server_port: XXXX
DEBUG:  gateway_ip: X.X.X.X
DEBUG:  gateway_port: XXXX
DEBUG:  Setting cipher list to: HIGH:!aNULL:!kRSA:!PSK:!SRP:!MD5:!RC4
DEBUG:  Setting minimum protocol version to: 0x303.
Enter PKCS#11 token PIN for SmartCard-HSM (UserPIN):
ERROR:  SSL_connect: error:0A000126:SSL routines::unexpected eof while reading
You might want to try --insecure-ssl or specify a different --cipher-list
INFO:   Closed connection to gateway.
DEBUG:  SO_KEEPALIVE: OFF
DEBUG:  TCP_KEEPIDLE: 7200
DEBUG:  TCP_KEEPINTVL: 75
DEBUG:  TCP_KEEPCNT: 9
DEBUG:  SO_SNDBUF: 16384
DEBUG:  SO_RCVBUF: 131072
DEBUG:  server_addr: X.X.X.X
DEBUG:  server_port: XXXX
DEBUG:  gateway_ip: X.X.X.X
DEBUG:  gateway_port: XXXX
DEBUG:  Setting cipher list to: HIGH:!aNULL:!kRSA:!PSK:!SRP:!MD5:!RC4
DEBUG:  Setting minimum protocol version to: 0x303.
DEBUG:  Gateway certificate validation failed.
DEBUG:  Gateway certificate digest found in white list.
DEBUG:  http_send:
GET /remote/logout HTTP/1.1
Host: X.X.X.X:XXXX
User-Agent: Mozilla/5.0 SV1
Accept: */*
Accept-Encoding: gzip, deflate, br
Pragma: no-cache
Cache-Control: no-store, no-cache, must-revalidate
If-Modified-Since: Sat, 1 Jan 2000 00:00:00 GMT
Content-Type: application/x-www-form-urlencoded
Cookie: 
Content-Length: 0

DEBUG:  http_receive:
HTTP/1.1 200 OK
Date: Tue, 11 Jun 2024 09:33:49 GMT
Server: xxxxxxxx-xxxxx
Set-Cookie:  SVPNCOOKIE=; path=/; expires=Sun, 11 Mar 1984 12:00:00 GMT; secure; httponly; SameSite=Strict;
Set-Cookie: SVPNNETWORKCOOKIE=; path=/remote/network; expires=Sun, 11 Mar 1984 12:00:00 GMT; secure; httponly; SameSite=Strict
Content-Length: 560
Content-Type: text/html; charset=utf-8
X-Frame-Options: SAMEORIGIN
Content-Security-Policy: frame-ancestors 'self'; object-src 'self'; script-src 'self' https:   'unsafe-eval' 'unsafe-inline' blob:;
X-XSS-Protection: 1; mode=block
X-Content-Type-Options: nosniff
Strict-Transport-Security: max-age=31536000

<!DOCTYPE html>
<html><head><script>function fgt_sslvpn_logout(sid) {var cookies = document.cookie.split(';');for (var c = 0; c < cookies.length; ++c) {var one_c = cookies[0];var cookie_key = one_c.split('=')[0];cookie_key.trim();if (cookie_key.search('_4189773768') == null) {var base_name = cookie_key + '=; expires=Thu, 01-Jan-1970 00:00:01 GMT; path=';document.cookie = base_name + '/';document.cookie = base_name + '/proxy/' + sid;}}window.location.href ='/remote/login';}</script></head><body><script>fgt_sslvpn_logout("00000000");</script></body></html>em
INFO:   Logged out.

I was able to capture some FortiGate logs, and from what I can tell, when it works, after a successful SSL handshake, OpenFortiVPN calls the FortiGate API at /remote/checklogin, likely in src/http.c.

However, when it doesn't work and an error occurs, it first completes a successful SSL handshake (using a strong cipher suite like AES in GCM), but then immediately calls /remote/logout.

From fortigate log :

SSL state:TLSv1.3 early data (x.x.x.x)
SSL state:SSLv3/TLS read client certificate (x.x.x.x)
SSL state:SSLv3/TLS read certificate verify (x.x.x.x)
SSL state:SSLv3/TLS read finished (x.x.x.x)
SSL state:SSLv3/TLS write session ticket (x.x.x.x)
SSL state:SSLv3/TLS write session ticket (x.x.x.x)
SSL established: TLSv1.3 TLS_AES_256_GCM_SHA384
req: /remote/logout

Do someone have any idea of what is going on ?

DimitriPapadopoulos commented 2 weeks ago

Please check with a newer version of openfortivpn. Version 1.17.1 was released back in 2021.

leompxp commented 2 weeks ago

Hi @DimitriPapadopoulos, Thank you for your answer

The version 1.17.1-1build1 is the latest version on ubuntu 22.04 in the universe repository and here it does works sometimes, In my thread I am mostly talking about the version 1.21.0-2build2 on ubuntu 24.04LTS in the noble universe repository where it does not works

DimitriPapadopoulos commented 2 weeks ago

Perhaps the ciphers supported by the VPN gateway are obsolete and have been disabled in Ubuntu 24.04. What is the version of FortiOS on your Fortigate 100E? Start by trying --insecure-ssl --seclevel-1.

Try a direct OpenSSL connection to the server, the result might give some insight:

$ echo -n | openssl s_client -connect X.X.X.X:443

As for clients, to find supported ciphers:

$ openssl ciphers

leompxp commented 1 week ago

I don't think it is a cipher problem,

Using --insecure-ssl --seclevel-1 give the same inconsistent output, sometimes it works, sometimes it don't with the EOL SSL error.

In my initial message we can see that the cipher negociated during the TLS handshake is a strong and supported cipher by both the client and the fortigate : SSL established: TLSv1.3 TLS_AES_256_GCM_SHA384. So anyway, even if weak cipher were supported, the cipher negociate is not an obsolete one

On the fortigate side we enable only strong-crypto which enable strong cipher for TLS 1.2 and TLS 1.3

Here the results of the command :

$ echo -n | openssl s_client -connect X.X.X.X:443

[....]
Requested Signature Algorithms: ECDSA+SHA256:ECDSA+SHA384:ECDSA+SHA512:Ed25519:Ed448:RSA-PSS+SHA256:RSA-PSS+SHA384:RSA-PSS+SHA512:RSA-PSS+SHA256:RSA-PSS+SHA384:RSA-PSS+SHA512:RSA+SHA256:RSA+SHA384:RSA+SHA512:ECDSA+SHA224:ECDSA+SHA1:RSA+SHA224:RSA+SHA1
Shared Requested Signature Algorithms: ECDSA+SHA256:ECDSA+SHA384:ECDSA+SHA512:Ed25519:Ed448:RSA-PSS+SHA256:RSA-PSS+SHA384:RSA-PSS+SHA512:RSA-PSS+SHA256:RSA-PSS+SHA384:RSA-PSS+SHA512:RSA+SHA256:RSA+SHA384:RSA+SHA512
Peer signing digest: SHA256
Peer signature type: RSA-PSS
Server Temp Key: ECDH, secp384r1, 384 bits
---
SSL handshake has read 4110 bytes and written 761 bytes
Verification: OK
---
New, TLSv1.3, Cipher is TLS_AES_256_GCM_SHA384

$ openssl ciphers
TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256:TLS_AES_128_GCM_SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:DHE-RSA-AES256-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES256-SHA:ECDHE-RSA-AES256-SHA:DHE-RSA-AES256-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES128-SHA:DHE-RSA-AES128-SHA:RSA-PSK-AES256-GCM-SHA384:DHE-PSK-AES256-GCM-SHA384:RSA-PSK-CHACHA20-POLY1305:DHE-PSK-CHACHA20-POLY1305:ECDHE-PSK-CHACHA20-POLY1305:AES256-GCM-SHA384:PSK-AES256-GCM-SHA384:PSK-CHACHA20-POLY1305:RSA-PSK-AES128-GCM-SHA256:DHE-PSK-AES128-GCM-SHA256:AES128-GCM-SHA256:PSK-AES128-GCM-SHA256:AES256-SHA256:AES128-SHA256:ECDHE-PSK-AES256-CBC-SHA384:ECDHE-PSK-AES256-CBC-SHA:SRP-RSA-AES-256-CBC-SHA:SRP-AES-256-CBC-SHA:RSA-PSK-AES256-CBC-SHA384:DHE-PSK-AES256-CBC-SHA384:RSA-PSK-AES256-CBC-SHA:DHE-PSK-AES256-CBC-SHA:AES256-SHA:PSK-AES256-CBC-SHA384:PSK-AES256-CBC-SHA:ECDHE-PSK-AES128-CBC-SHA256:ECDHE-PSK-AES128-CBC-SHA:SRP-RSA-AES-128-CBC-SHA:SRP-AES-128-CBC-SHA:RSA-PSK-AES128-CBC-SHA256:DHE-PSK-AES128-CBC-SHA256:RSA-PSK-AES128-CBC-SHA:DHE-PSK-AES128-CBC-SHA:AES128-SHA:PSK-AES128-CBC-SHA256:PSK-AES128-CBC-SHA

leompxp commented 1 week ago

To support my point, this morning a colleague ran the command five times with errors, rebooted his machine, then had the error again after the reboot, and the second time it worked.

Also I don't know if this could be a problem with the pkcs11 devices support but when it fail, sometimes, it takes longer (about 15 to 45 seconds) to open the token.

and the error pop almost after reading the token.

But it works most of the time, so I'm not convinced that the code that reads the token could be inconsistent, as it is very straightforward.

DimitriPapadopoulos commented 1 week ago

I fully agree after rereading the initial post and the new information. I initially thought this is an internal OpenSSL issue. Actually the remote server abruptly shuts down the connection, because it doesn't like something. We need to identify what the server doesn't like.

leompxp commented 1 week ago

I don't entirely read the source code but I feel like you their is something that causing abruptly shuts down the connection.

I feel like this is on the client side because it is openfortivpn that send the GET /remote/logout, right after sending the crentials instead of the GET /remote/logincheck.

But maybe, the server send to the client that it will not continue anyway and so the openfortivpn try to gracefully shutdown the connection by sending /remote/logout.

But I could not capture anything between theSSL established: TLSv1.3 TLS_AES_256_GCM_SHA384 and the /remote/logout. These two log follow each other.

adrienverge / openfortivpn