search

matteocorti / check_ssl_cert

A shell script (that can be used as a Nagios/Icinga plugin) to check an SSL/TLS connection.
GNU General Public License v3.0
368 stars 132 forks source link

SSL Error : SSL routines:tls_process_key_exchange:internal error:ssl/statem/statem_clnt.c #436

Closed K4S1 closed 1 year ago

K4S1 commented 1 year ago

Describe the bug I see following Error output when polling a host of ours: CRITICAL error: SSL error: 48EB1D6C967F0000:error:0A0C0103:SSL routines:tls_process_key_exchange:internal error:ssl/statem/statem_clnt.c:2254:

To Reproduce

Not completly sure myself only have this error on a single host. But Can't deliver information about the host. I am able to do test towards the host, without expose it.

Expected behavior

No internal Error output

System (please complete the following information):

Additional context/output

Add any other context or output (e.g., from check_ssl_cert -d -v) about the problem here.

Hi have tried to do my best herebut have not been able to find the root cause of the error. But Not sure if this can be delt with here or I need to do changes to my docker image to fix.

Thanks for a great Script !

server:~ -> docker exec -it librenms /usr/lib/monitoring-plugins/check_ssl_cert -H [protected.Domain.local] --ignore-tls-renegotiation --ignore-ocsp --ignore-host-cn --ignore-sct --ignore-incomplete-chain --ignore-ssl-labs-cache -vvv -d

[DBG] Shell:
/usr/lib/monitoring-plugins/check_ssl_cert: line 4020: : Permission denied
[DBG]
[DBG] grep: grep
grep: unrecognized option: version
BusyBox v1.35.0 (2022-11-19 10:13:10 UTC) multi-call binary.

Usage: grep [-HhnlLoqvsrRiwFE] [-m N] [-A|B|C N] { PATTERN | -e PATTERN... | -f FILE... } [FILE]...

Search for PATTERN in FILEs (or stdin)

    -H  Add 'filename:' prefix
    -h  Do not add 'filename:' prefix
    -n  Add 'line_no:' prefix
    -l  Show only names of files that match
    -L  Show only names of files that don't match
    -c  Show only count of matching lines
    -o  Show only the matching part of line
    -q  Quiet. Return 0 if PATTERN is found, 1 otherwise
    -v  Select non-matching lines
    -s  Suppress open and read errors
    -r  Recurse
    -R  Recurse and dereference symlinks
    -i  Ignore case
    -w  Match whole words only
    -x  Match whole lines only
    -F  PATTERN is a literal (not regexp)
    -E  PATTERN is an extended regexp
    -m N    Match up to N times per file
    -A N    Print N lines of trailing context
    -B N    Print N lines of leading context
    -C N    Same as '-A N -B N'
    -e PTRN Pattern to match
    -f FILE Read pattern from file
[DBG]
[DBG] hostname: /bin/hostname
[DBG] $PATH: /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
[DBG] Command line arguments: -v
[DBG]   TMPDIR = /tmp
[DBG] Required HTTP headers:
[DBG] Unrequired HTTP headers:
[DBG] Adding the domain if missing
[DBG] HOST = [protected.Domain.local]
[DBG] SNI                 =
[DBG] HOST_NAME           = [protected.Domain.local]
[DBG] HOST_ADDR           = [protected.Domain.local]
[DBG] NAMES_TO_BE_CHECKED =
[DBG] Checking if [protected.Domain.local] is an IP address
[DBG] [protected.Domain.local] is not an IP address
[DBG] HOST_IS_IP.         = 0
[DBG] NAMES_TO_BE_CHECKED =
[DBG] curl binary not needed. SSL Labs = , OCSP =
[DBG] -c specified: 15
[DBG] -w specified: 20
[DBG] Executing comparison '1728000 <= 1296000'
[DBG]   bc result = 0
[DBG]   returning 1
[DBG] ROOT_CA =
[DBG] mktemp available: /bin/mktemp
[DBG] file version: file-5.43
[DBG] magic file from /usr/share/misc/magic
[DBG] nmap binary not specified
[DBG] nmap available: /usr/bin/nmap
[DBG] Checking IPs: host [protected.Domain.local]
expect not available
timeout available (/usr/bin/timeout)
[DBG] perl available: /usr/bin/perl
[DBG] date available: /bin/date
[DBG] checking date version
[DBG] date computation type: GNU
Found GNU date with timestamp support: enabling date computations
[DBG] check_ssl_cert version: 2.57.0
[DBG] OpenSSL binary: /usr/bin/openssl
[DBG] OpenSSL info:
[DBG] OpenSSL 3.0.7 1 Nov 2022 (Library: OpenSSL 3.0.7 1 Nov 2022)
[DBG] built on: Tue Nov  1 15:49:38 2022 UTC
[DBG] platform: linux-x86_64
[DBG] options:  bn(64,64)
[DBG] compiler: gcc -fPIC -pthread -m64 -Wa,--noexecstack -Os -fomit-frame-pointer -g -Os -fomit-frame-pointer -Os -fomit-frame-pointer -g -Wa,--noexecstack -DOPENSSL_USE_NODELETE -DL_ENDIAN -DOPENSSL_PIC -DOPENSSL_BUILDING_OPENSSL -DNDEBUG -Os -fomit-frame-pointer
[DBG] OPENSSLDIR: "/etc/ssl"
[DBG] ENGINESDIR: "/usr/lib/engines-3"
[DBG] MODULESDIR: "/usr/lib/ossl-modules"
[DBG] Seeding source: os-specific
[DBG] CPUINFO: OPENSSL_ia32cap=0xfffa32034fabffff:0x1c27eb
[DBG] OpenSSL configuration directory: /etc/ssl
[DBG] 140 root certificates installed by default
[DBG]  System info: Linux librenms 4.19.0-8-amd64 #1 SMP Debian 4.19.98-1+deb10u1 (2020-04-27) x86_64 GNU/Linux
[DBG] Date computation: GNU
[DBG] '/usr/bin/openssl s_client' supports '-servername': using -servername [protected.Domain.local]
[DBG] Proxy settings (before):
[DBG]   http_proxy  =
[DBG]   https_proxy =
[DBG]   HTTP_PROXY  =
[DBG]   HTTPS_PROXY =
[DBG] Proxy settings (after):
[DBG]   http_proxy  =
[DBG]   https_proxy =
[DBG]   HTTP_PROXY  =
[DBG]   HTTPS_PROXY =
[DBG]   s_client    =
[DBG]   curl        =
[DBG] '/usr/bin/openssl s_client' supports '-name': using librenms
[DBG] '/usr/bin/openssl s_client' supports '-xmpphost': using -xmpphost [protected.Domain.local]
[DBG] HOST_HEADER = [protected.Domain.local]
[DBG] --ignore-tls-renegotiation specified: checking OpenSSL version and -legacy_renegotiation support
[DBG] OpenSSL s_client supports the -legacy_renegotiation option
[DBG] Testing connection with [protected.Domain.local]:443
[DBG] Executing: '/usr/bin/nmap  --unprivileged -Pn -p 443 [protected.Domain.local]'
[DBG] Sanity checks: OK
[DBG] temporary file /tmp/WuLRIQ created
[DBG] temporary file /tmp/CsndOV created
[DBG] temporary file /tmp/5IVul6 created
[DBG] temporary file /tmp/tWBg8k created
[DBG] temporary file /tmp/bpmjfs created
[DBG] Temporary files created
Downloading certificate to /tmp
[DBG] [protected.Domain.local] is not an IP address
[DBG] fetch_certificate: PROTOCOL =
[DBG] exec_with_timeout printf 'HEAD / HTTP/1.1\nHost: [protected.Domain.local]\nUser-Agent: check_ssl_cert/2.57.0\nConnection: close\n\n' | /usr/bin/openssl s_client    -crlf  -connect [protected.Domain.local]:443 -servername [protected.Domain.local]   -showcerts -verify 6       -legacy_renegotiation 2> /tmp/CsndOV 1> /tmp/WuLRIQ
[DBG]   TIMEOUT_REASON = fetching certificate
[DBG] executing with timeout (120s): printf 'HEAD / HTTP/1.1\nHost: [protected.Domain.local]\nUser-Agent: check_ssl_cert/2.57.0\nConnection: close\n\n' | /usr/bin/openssl s_client    -crlf  -connect [protected.Domain.local]:443 -servername [protected.Domain.local]   -showcerts -verify 6       -legacy_renegotiation 2> /tmp/CsndOV 1> /tmp/WuLRIQ
[DBG]   start time = 1673880094
[DBG] /usr/bin/timeout 120 /bin/sh -c "printf 'HEAD / HTTP/1.1\nHost: [protected.Domain.local]\nUser-Agent: check_ssl_cert/2.57.0\nConnection: close\n\n' | /usr/bin/openssl s_client    -crlf  -connect [protected.Domain.local]:443 -servername [protected.Domain.local]   -showcerts -verify 6       -legacy_renegotiation 2> /tmp/CsndOV 1> /tmp/WuLRIQ"
[DBG]   end time = 1673880095
[DBG]   new timeout = 119
[DBG] Return value of the command = 1
[DBG] SSL error: verify depth is 6
[DBG] SSL error: depth=2 OU = GlobalSign Root CA - R3, O = GlobalSign, CN = GlobalSign
[DBG] SSL error: verify return:1
[DBG] SSL error: depth=1 C = BE, O = GlobalSign nv-sa, CN = GlobalSign GCC R3 DV TLS CA 2020
[DBG] SSL error: verify return:1
[DBG] SSL error: depth=0 CN = [Common_protected.Domain.local]
[DBG] SSL error: verify return:1
[DBG] SSL error: 48CB981D3A7F0000:error:0A0C0103:SSL routines:tls_process_key_exchange:internal error:ssl/statem/statem_clnt.c:2254:
[DBG] Unknown error: 48CB981D3A7F0000:error:0A0C0103:SSL routines:tls_process_key_exchange:internal error:ssl/statem/statem_clnt.c:2254:
CRITICAL error: SSL error: 48CB981D3A7F0000:error:0A0C0103:SSL routines:tls_process_key_exchange:internal error:ssl/statem/statem_clnt.c:2254:
[DBG] CRITICAL ----------------------------------------
[DBG] prepend_critical_message: new message    = SSL error: 48CB981D3A7F0000:error:0A0C0103:SSL routines:tls_process_key_exchange:internal error:ssl/statem/statem_clnt.c:2254:
[DBG] prepend_critical_message: CRITICAL_MSG   =
[DBG] prepend_critical_message: ALL_MSG 1      =
[DBG] prepend_critical_message: MSG 2          = SSL_CERT CRITICAL [protected.Domain.local]: SSL error: 48CB981D3A7F0000:error:0A0C0103:SSL routines:tls_process_key_exchange:internal error:ssl/statem/statem_clnt.c:2254:
[DBG] prepend_critical_message: CRITICAL_MSG 2 = SSL_CERT CRITICAL [protected.Domain.local]: SSL error: 48CB981D3A7F0000:error:0A0C0103:SSL routines:tls_process_key_exchange:internal error:ssl/statem/statem_clnt.c:2254:
[DBG] prepend_critical_message: ALL_MSG 2      = \n    SSL_CERT CRITICAL [protected.Domain.local]: SSL error: 48CB981D3A7F0000:error:0A0C0103:SSL routines:tls_process_key_exchange:internal error:ssl/statem/statem_clnt.c:2254:
[DBG] CRITICAL ----------------------------------------
[DBG] openssl_version 3.0.0
[DBG] Checking if OpenSSL version is at least 3.0.0 ( '3' '0' '0' ':0' )
[DBG] openssl version: OpenSSL 3.0.7 1 Nov 2022 (Library: OpenSSL 3.0.7 1 Nov 2022)
[DBG] Current version 3.0.7 ( '3' '0' '7' ':0' )
[DBG]   true
[DBG] Skipping TLS renegotiation check as OpenSSL 3.0.0 enforces it by default
Parsing the x509 certificate file
[DBG] extracting cert attribute enddate
[DBG] extracting cert attribute startdate
[DBG] extracting cert attribute cn
[DBG] extracting cert attribute subject
[DBG] SUBJECT = subject=CN = [Common_protected.Domain.local]
[DBG] extracting cert attribute serial
[DBG] SERIAL = XXXXXXXXXXXXXXXXXXX
[DBG] extracting cert attribute version
[DBG] X509_VERSION = 3 (0x2)
[DBG] extracting cert attribute fingerprint
[DBG] FINGERPRINT = sha1 Fingerprint=XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX
[DBG] Checking if x509 supports the -ext option
[DBG] extracting cert attribute keyUsage
[DBG] Certificate purpose is not defined as critical
[DBG] extracting cert attribute oscp_uri_single
[DBG] extracting cert attribute oscp_uri
[DBG] OCSP_URI = http://ocsp.globalsign.com/gsgccr3dvtlsca2020
[DBG] Extracting issuers
[DBG]   Number of certificates in the chain: 2
[DBG] Checking certificate chain
[DBG]     extracting issuer for element 1
[DBG] extracting cert attribute issuer
[DBG] ELEMENT_ISSUER=GlobalSign nv-sa
[DBG] ELEMENT_ISSUER=GlobalSign GCC R3 DV TLS CA 2020
[DBG] ISSUERS=GlobalSign nv-sa
[DBG] ISSUERS=GlobalSign GCC R3 DV TLS CA 2020
[DBG]     extracting issuer for element 2
[DBG] extracting cert attribute issuer
[DBG] ELEMENT_ISSUER=GlobalSign
[DBG] ELEMENT_ISSUER=GlobalSign
[DBG] ISSUERS=GlobalSign nv-sa
[DBG] ISSUERS=GlobalSign GCC R3 DV TLS CA 2020
[DBG] ISSUERS=GlobalSign
[DBG] ISSUERS=GlobalSign
[DBG] Certificate chain check finished
[DBG] ISSUERS =
[DBG] GlobalSign nv-sa
[DBG] GlobalSign GCC R3 DV TLS CA 2020
[DBG] GlobalSign
[DBG] GlobalSign
[DBG] extracting cert attribute issuer_uri_single
[DBG] extracting cert attribute issuer_uri
[DBG] extracting cert attribute sig_algo
[DBG] subject=CN = [Common_protected.Domain.local]
[DBG] CN         = [Common_protected.Domain.local]
[DBG] CA         = GlobalSign nv-sa
[DBG] CA         = GlobalSign GCC R3 DV TLS CA 2020
[DBG] CA         = GlobalSign
[DBG] CA         = GlobalSign
[DBG] SERIAL     = XXXXXXXXXXXXXXXXXXX
[DBG] FINGERPRINT= sha1 Fingerprint=XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX
[DBG] OCSP_URI   = http://ocsp.globalsign.com/gsgccr3dvtlsca2020
[DBG] ISSUER_URI = http://secure.globalsign.com/cacert/gsgccr3dvtlsca2020.crt
[DBG] rsaEncryption (2048 bit)
[DBG] extracting cert attribute subjectAlternativeName
[DBG] subjectAlternativeName = [Common_protected.Domain.local] [SAN_protected.Domain.local] [protected.Domain.local] [SAN_protected.Domain.local] [SAN_protected.Domain.local] [SAN_protected.Domain.local] [SAN_protected.Domain.local] [SAN_protected.Domain.local] [SAN_protected.Domain.local] [SAN_protected.Domain.local] [SAN_protected.Domain.local] [SAN_protected.Domain.local] [SAN_protected.Domain.local] [SAN_protected.Domain.local] [SAN_protected.Domain.local] [SAN_protected.Domain.local] [SAN_protected.Domain.local] [SAN_protected.Domain.local] [SAN_protected.Domain.local] [SAN_protected.Domain.local] [SAN_protected.Domain.local] [SAN_protected.Domain.local]
The certificate for this site contains a Subject Alternative Name extension
[DBG] Checking expiration date
[DBG] Number of certificates in CA chain: 2
[DBG] ------------------------------------------------------------------------------
[DBG] -- Checking element 1
[DBG] extracting cert attribute cn
[DBG] Checking expiration date of element 1 ([Common_protected.Domain.local])
[DBG] extracting cert attribute enddate
[DBG] Validity date on cert element 1 ([Common_protected.Domain.local]) is Jun 11 11:02:22 2023 GMT
[DBG] Date computations: GNU
[DBG] Computing number of hours until 'Jun 11 11:02:22 2023 GMT' with GNU
[DBG] Computing '(1686481342-1673880096)/3600' (precision 0)
[DBG] Hours until Jun 11 11:02:22 2023 GMT: 3500
[DBG] Computing '3500/24' (precision 0)
[DBG] Computing '3500 * 3600' (precision 0)
[DBG] Adding line to prometheus days output: cert_days_chain_elem{cn="[Common_protected.Domain.local]", element="1"} 145
[DBG]   valid for 145 days
[DBG] executing: /usr/bin/openssl x509 -noout -checkend 0 on cert element 1 ([Common_protected.Domain.local])
[DBG] executing: /usr/bin/openssl x509 -noout -checkend 1296000 on cert element 1 ([Common_protected.Domain.local])
[DBG] executing: /usr/bin/openssl x509 -noout -checkend 1728000 on cert element 1
Certificate element 1 ([Common_protected.Domain.local]) is valid for 145 days
[DBG] Adding line to prometheus validity output: cert_valid_chain_elem{cn="[Common_protected.Domain.local]", element="1"} 0
[DBG] ------------------------------------------------------------------------------
[DBG] Checking OCSP status of element 1
[DBG] temporary file /tmp/iUVd3b created
[DBG] Storing the chain element in /tmp/iUVd3b
[DBG] ------------------------------------------------------------------------------
[DBG] -- Checking element 2
[DBG] extracting cert attribute cn
[DBG] Checking expiration date of element 2 (GlobalSign GCC R3 DV TLS CA 2020)
[DBG] extracting cert attribute enddate
[DBG] Validity date on cert element 2 (GlobalSign GCC R3 DV TLS CA 2020) is Mar 18 00:00:00 2029 GMT
[DBG] Date computations: GNU
[DBG] Computing number of hours until 'Mar 18 00:00:00 2029 GMT' with GNU
[DBG] Computing '(1868486400-1673880096)/3600' (precision 0)
[DBG] Hours until Mar 18 00:00:00 2029 GMT: 54057
[DBG] Computing '54057/24' (precision 0)
[DBG] Computing '54057 * 3600' (precision 0)
[DBG] Adding line to prometheus days output: cert_days_chain_elem{cn="[Common_protected.Domain.local]", element="2"} 2252
[DBG]   valid for 2252 days
[DBG] Executing comparison '54057 < 3500'
[DBG]   bc result = 0
[DBG]   returning 1
[DBG] Executing comparison '2252 < 145'
[DBG]   bc result = 0
[DBG]   returning 1
[DBG] executing: /usr/bin/openssl x509 -noout -checkend 0 on cert element 2 (GlobalSign GCC R3 DV TLS CA 2020)
[DBG] executing: /usr/bin/openssl x509 -noout -checkend 1296000 on cert element 2 (GlobalSign GCC R3 DV TLS CA 2020)
[DBG] executing: /usr/bin/openssl x509 -noout -checkend 1728000 on cert element 2
Certificate element 2 (GlobalSign GCC R3 DV TLS CA 2020) is valid for 2252 days
[DBG] Adding line to prometheus validity output: cert_valid_chain_elem{cn="[Common_protected.Domain.local]", element="2"} 0
[DBG] ------------------------------------------------------------------------------
[DBG] Checking OCSP status of element 2
[DBG] temporary file /tmp/alZCF8 created
[DBG] Storing the chain element in /tmp/alZCF8
[DBG] ------------------------------------------------------------------------------
[DBG] extracting cert attribute email
[DBG] EMAIL =
The certificate was successfully verified
[DBG] openssl_version 1.1.0
[DBG] Checking if OpenSSL version is at least 1.1.0 ( '1' '1' '0' ':0' )
[DBG] openssl version: 3.0.7
[DBG] Current version 3.0.7 ( '3' '0' '7' ':0' )
[DBG]   true
[DBG] Checking Signed Certificate Timestamps (SCTs)
The certificate contains signed certificate timestamps (SCT)
[DBG] Date computations: GNU
[DBG] Computing number of hours until 'Jun 11 11:02:22 2023 GMT' with GNU
[DBG] Computing '(1686481342-1673880097)/3600' (precision 0)
[DBG] Hours until Jun 11 11:02:22 2023 GMT: 3500
[DBG] Date computations: GNU
[DBG] Computing number of hours until 'May 10 11:47:02 2022 GMT' with GNU
[DBG] Computing '(1652183222-1673880097)/3600' (precision 0)
[DBG] Hours until May 10 11:47:02 2022 GMT: -6026
[DBG] Computing '(3500 - -6026)/24' (precision 0)
The certificate validity (396) is shorter then the maximum (397)
[DBG] cleaning up temporary files
[DBG]  /tmp/WuLRIQ /tmp/C
[DBG] ndOV /tmp/5IVul6 /tmp/tWBg8k /tmp/bpmjf
[DBG]  /tmp/iUVd3b /tmp/alZCF8
[DBG] exiting with CRITICAL
[DBG] ALL_MSG = \n    SSL_CERT CRITICAL [protected.Domain.local]: SSL error: 48CB981D3A7F0000:error:0A0C0103:SSL routines:tls_process_key_exchange:internal error:ssl/statem/statem_clnt.c:2254:
[DBG] number of errors = 1
SSL_CERT CRITICAL [protected.Domain.local]: SSL error: 48CB981D3A7F0000:error:0A0C0103:SSL routines:tls_process_key_exchange:internal error:ssl/statem/statem_clnt.c:2254:|days_chain_elem1=145;20;15;; days_chain_elem2=2252;20;15;;
lukastribus commented 1 year ago

Likely your destination host is only supporting TLSv1.0 and you are using a TLSv1.2 only openssl-3 configuration:

https://github.com/openssl/openssl/issues/19867

Possible workaround is setting the seclevel to 0 in openssl: -cipher DEFAULT@SECLEVEL=0

matteocorti commented 1 year ago

I just updated a new version with a better error message. E.g.

$ ./check_ssl_cert --host imap.cern.ch --port 993
SSL_CERT_CRITICAL imap.cern.ch: Legacy signature algorithm unsupported or disallowed

And an option to set the security level as suggested by @lukastribus 

$ ./check_ssl_cert --host imap.cern.ch --port 993 --security-level 0
SSL_CERT OK - imap.cern.ch:993, https, x509 certificate 'mmm.cern.ch' (imap.cern.ch) from 'Sectigo Limited' valid until Nov 12 23:59:59 2023 GMT (expires in 300 days)|days_chain_elem1=300;20;15;; days_chain_elem2=2906;20;15;; days_chain_elem3=2176;20;15;;
$ ./check_ssl_cert --host imap.cern.ch --port 993 --security-level 1
SSL_CERT_CRITICAL imap.cern.ch: Legacy signature algorithm unsupported or disallowed
$ ./check_ssl_cert --host imap.cern.ch --port 993 --security-level 3
SSL_CERT_CRITICAL imap.cern.ch: Unsupported TLS protocol version 
$ ./check_ssl_cert --host imap.cern.ch --port 993 --security-level 4
SSL_CERT_CRITICAL imap.cern.ch:993: TLS handshake error
matteocorti commented 1 year ago

@K4S1 Dear Kasper, this is as much as I can do without testing.

K4S1 commented 1 year ago

@matteocorti Thanks, this was precis the change I needed for for everything to work in my end. image

Confirmed working. Great fix, more scalability, Easy to understand output ! Love it.

Thanks for your help, and a great script.