rsyslog / rsyslog

a Rocket-fast SYStem for LOG processing
http://www.rsyslog.com
GNU Lesser General Public License v3.0
2.02k stars 649 forks source link

rsyslogd[1371]: unexpected GnuTLS error -24 in nsd_gtls.c:612: Decryption has failed. #5367

Open shivangi29g opened 5 months ago

shivangi29g commented 5 months ago

Expected behavior

Shouldn,t see the error in GnuTLS and session close unexpectedly due to decrypt error.

Actual behavior

Apr 24 13:51:22 scs000201555 rsyslogd[1371]: unexpected GnuTLS error -24 in nsd_gtls.c:612: Decryption has failed. [v8.2102.0-15.el8 try https://www.> Apr 24 13:51:22 scs000201555 rsyslogd[1371]: netstream session 0x7fb61c014bb0 from 10.140.53.12 will be closed due to error [v8.2102.0-15.el8 try http>

Steps to reproduce the behavior

send syslog messages and the session will terminate after sometime.

1) For the following the error comes after like 16 hours NAME="Red Hat Enterprise Linux" VERSION="8.9 (Ootpa)" ID="rhel" ID_LIKE="fedora" VERSION_ID="8.9" PLATFORM_ID="platform:el8" PRETTY_NAME="Red Hat Enterprise Linux 8.9 (Ootpa)" ANSI_COLOR="0;31" CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" HOME_URL="https://www.redhat.com/" DOCUMENTATION_URL="https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8" BUG_REPORT_URL="https://bugzilla.redhat.com/"

REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" REDHAT_BUGZILLA_PRODUCT_VERSION=8.9 REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" REDHAT_SUPPORT_PRODUCT_VERSION="8.9"

rsyslogd -v rsyslogd 8.2102.0-15.el8 (aka 2021.02) compiled with: PLATFORM: x86_64-redhat-linux-gnu PLATFORM (lsb_release -d): FEATURE_REGEXP: Yes GSSAPI Kerberos 5 support: Yes FEATURE_DEBUG (debug build, slow code): No 32bit Atomic operations supported: Yes 64bit Atomic operations supported: Yes memory allocator: system default Runtime Instrumentation (slow code): No uuid support: Yes systemd support: Yes Config file: /etc/rsyslog.conf PID file: /var/run/rsyslogd.pid Number of Bits in RainerScript integers: 64

See https://www.rsyslog.com for more information.

2) cat /etc/os-release NAME="Red Hat Enterprise Linux" VERSION="9.3 (Plow)" ID="rhel" ID_LIKE="fedora" VERSION_ID="9.3" PLATFORM_ID="platform:el9" PRETTY_NAME="Red Hat Enterprise Linux 9.3 (Plow)" ANSI_COLOR="0;31" LOGO="fedora-logo-icon" CPE_NAME="cpe:/o:redhat:enterprise_linux:9::baseos" HOME_URL="https://www.redhat.com/" DOCUMENTATION_URL="https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/9" BUG_REPORT_URL="https://bugzilla.redhat.com/"

REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 9" REDHAT_BUGZILLA_PRODUCT_VERSION=9.3 REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" REDHAT_SUPPORT_PRODUCT_VERSION="9.3"

[root@scs000201553 msg]# rsyslogd -v rsyslogd 8.2102.0-117.el9 (aka 2021.02) compiled with: PLATFORM: x86_64-redhat-linux-gnu PLATFORM (lsb_release -d): FEATURE_REGEXP: Yes GSSAPI Kerberos 5 support: Yes FEATURE_DEBUG (debug build, slow code): No 32bit Atomic operations supported: Yes 64bit Atomic operations supported: Yes memory allocator: system default Runtime Instrumentation (slow code): No uuid support: Yes systemd support: Yes Config file: /etc/rsyslog.conf PID file: /var/run/rsyslogd.pid Number of Bits in RainerScript integers: 64

See https://www.rsyslog.com for more information.

This it takes 2 hours

3) cat /etc/os-release NAME="Red Hat Enterprise Linux" VERSION="8.9 (Ootpa)" ID="rhel" ID_LIKE="fedora" VERSION_ID="8.9" PLATFORM_ID="platform:el8" PRETTY_NAME="Red Hat Enterprise Linux 8.9 (Ootpa)" ANSI_COLOR="0;31" CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" HOME_URL="https://www.redhat.com/" DOCUMENTATION_URL="https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8" BUG_REPORT_URL="https://bugzilla.redhat.com/"

REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" REDHAT_BUGZILLA_PRODUCT_VERSION=8.9 REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" REDHAT_SUPPORT_PRODUCT_VERSION="8.9"

[root@scspr2978654016 rsyslog_certs]# rsyslogd -v rsyslogd 8.2102.0-15.el8 (aka 2021.02) compiled with: PLATFORM: x86_64-redhat-linux-gnu PLATFORM (lsb_release -d): FEATURE_REGEXP: Yes GSSAPI Kerberos 5 support: Yes FEATURE_DEBUG (debug build, slow code): No 32bit Atomic operations supported: Yes 64bit Atomic operations supported: Yes memory allocator: system default Runtime Instrumentation (slow code): No uuid support: Yes systemd support: Yes Config file: /etc/rsyslog.conf PID file: /var/run/rsyslogd.pid Number of Bits in RainerScript integers: 64

See https://www.rsyslog.com for more information.

Takes less than 30 mins

Environment

Have mentioned above

I have tried with ossl but it fails too, Apr 23 07:36:45 scspr2978654016 rsyslogd[10043]: SSL_ERROR_SSL Error in 'osslRecordRecv': 'error:00000001:lib(0):func(0):reason(1)(1)' with ret=-1 [v8.2102.0-15.el8] Apr 23 07:36:45 scspr2978654016 rsyslogd[10043]: nsd_ossl:OpenSSL Error Stack: error:1408F119:SSL routines:ssl3_get_record:decryption failed or bad record mac [v8.2102.0-15.el8] Apr 23 07:36:45 scspr2978654016 rsyslogd[10043]: netstream session 0x7f4f90006710 from 10.234.189.118 will be closed due to error [v8.2102.0-15.el8] Apr 23 07:36:45 scspr2978654016 rsyslogd[10043]: SSL_ERROR_SSL Error in 'osslEndSess': 'error:00000001:lib(0):func(0):reason(1)(1)' with ret=-1 [v8.2102.0-15.el8] Apr 23 07:36:45 scspr2978654016 rsyslogd[10043]: nsd_ossl:OpenSSL Error Stack: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init [v8.2102.0-15.el8] Apr 23 07:36:45 scspr2978654016 rsyslogd[10043]: nsd_ossl:TLS session terminated with remote syslog server. [v8.2102.0-15.el8] Apr 23 07:36:46 scspr2978654016 rsyslogd[10043]: SSL_ERROR_SYSCALL Error in 'osslRecordRecv': 'error:00000005:lib(0):func(0):DH lib(5)' with ret=-1 [v8.2102.0-15.el8] Apr 23 07:36:46 scspr2978654016 rsyslogd[10043]: nsd_ossl:TLS session terminated with remote syslog server. [v8.2102.0-15.el8]

Also if it is due to password protected keys then it should fail at beginning only not after some messages are decrypted successfully.

davidelang commented 5 months ago

This error could be a security scanner tool checking for open ports that is not going to negotiate TLS.

If that is not the case, that is a version released in Feb 2021 with some backports that RedHat has applied from the ~20 versions since. I know that there has been some significant work on the encryption side, including a lot of work to get better error messages.

Can you try to upgrade to a more recent version and see if it gives you better information? Otherwise we will need to point you at RedHat support as we are not familiar with what they have and haven't backported across so many versions.

David Lang

On Wed, 24 Apr 2024, shivangi29g wrote:

Date: Wed, 24 Apr 2024 01:48:49 -0700 From: shivangi29g @.> Reply-To: rsyslog/rsyslog @.> To: rsyslog/rsyslog @.> Cc: Subscribed @.> Subject: [rsyslog/rsyslog] rsyslogd[1371]: unexpected GnuTLS error -24 in nsd_gtls.c:612: Decryption has failed. (Issue #5367)

Expected behavior

Shouldn,t see the error in GnuTLS and session close unexpectedly due to decrypt error.

Actual behavior

Apr 24 13:51:22 scs000201555 rsyslogd[1371]: unexpected GnuTLS error -24 in nsd_gtls.c:612: Decryption has failed. [v8.2102.0-15.el8 try https://www.> Apr 24 13:51:22 scs000201555 rsyslogd[1371]: netstream session 0x7fb61c014bb0 from 10.140.53.12 will be closed due to error [v8.2102.0-15.el8 try http>

Steps to reproduce the behavior

send syslog messages and the session will terminate after sometime.

1) For the following the error comes after like 16 hours NAME="Red Hat Enterprise Linux" VERSION="8.9 (Ootpa)" ID="rhel" ID_LIKE="fedora" VERSION_ID="8.9" PLATFORM_ID="platform:el8" PRETTY_NAME="Red Hat Enterprise Linux 8.9 (Ootpa)" ANSI_COLOR="0;31" CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" HOME_URL="https://www.redhat.com/" DOCUMENTATION_URL="https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8" BUG_REPORT_URL="https://bugzilla.redhat.com/"

REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" REDHAT_BUGZILLA_PRODUCT_VERSION=8.9 REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" REDHAT_SUPPORT_PRODUCT_VERSION="8.9"

rsyslogd -v rsyslogd 8.2102.0-15.el8 (aka 2021.02) compiled with: PLATFORM: x86_64-redhat-linux-gnu PLATFORM (lsb_release -d): FEATURE_REGEXP: Yes GSSAPI Kerberos 5 support: Yes FEATURE_DEBUG (debug build, slow code): No 32bit Atomic operations supported: Yes 64bit Atomic operations supported: Yes memory allocator: system default Runtime Instrumentation (slow code): No uuid support: Yes systemd support: Yes Config file: /etc/rsyslog.conf PID file: /var/run/rsyslogd.pid Number of Bits in RainerScript integers: 64

See https://www.rsyslog.com for more information.

2) cat /etc/os-release NAME="Red Hat Enterprise Linux" VERSION="9.3 (Plow)" ID="rhel" ID_LIKE="fedora" VERSION_ID="9.3" PLATFORM_ID="platform:el9" PRETTY_NAME="Red Hat Enterprise Linux 9.3 (Plow)" ANSI_COLOR="0;31" LOGO="fedora-logo-icon" CPE_NAME="cpe:/o:redhat:enterprise_linux:9::baseos" HOME_URL="https://www.redhat.com/" DOCUMENTATION_URL="https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/9" BUG_REPORT_URL="https://bugzilla.redhat.com/"

REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 9" REDHAT_BUGZILLA_PRODUCT_VERSION=9.3 REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" REDHAT_SUPPORT_PRODUCT_VERSION="9.3"

@.*** msg]# rsyslogd -v rsyslogd 8.2102.0-117.el9 (aka 2021.02) compiled with: PLATFORM: x86_64-redhat-linux-gnu PLATFORM (lsb_release -d): FEATURE_REGEXP: Yes GSSAPI Kerberos 5 support: Yes FEATURE_DEBUG (debug build, slow code): No 32bit Atomic operations supported: Yes 64bit Atomic operations supported: Yes memory allocator: system default Runtime Instrumentation (slow code): No uuid support: Yes systemd support: Yes Config file: /etc/rsyslog.conf PID file: /var/run/rsyslogd.pid Number of Bits in RainerScript integers: 64

See https://www.rsyslog.com for more information.

This it takes 2 hours

3) cat /etc/os-release NAME="Red Hat Enterprise Linux" VERSION="8.9 (Ootpa)" ID="rhel" ID_LIKE="fedora" VERSION_ID="8.9" PLATFORM_ID="platform:el8" PRETTY_NAME="Red Hat Enterprise Linux 8.9 (Ootpa)" ANSI_COLOR="0;31" CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" HOME_URL="https://www.redhat.com/" DOCUMENTATION_URL="https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8" BUG_REPORT_URL="https://bugzilla.redhat.com/"

REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" REDHAT_BUGZILLA_PRODUCT_VERSION=8.9 REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" REDHAT_SUPPORT_PRODUCT_VERSION="8.9"

@.*** rsyslog_certs]# rsyslogd -v rsyslogd 8.2102.0-15.el8 (aka 2021.02) compiled with: PLATFORM: x86_64-redhat-linux-gnu PLATFORM (lsb_release -d): FEATURE_REGEXP: Yes GSSAPI Kerberos 5 support: Yes FEATURE_DEBUG (debug build, slow code): No 32bit Atomic operations supported: Yes 64bit Atomic operations supported: Yes memory allocator: system default Runtime Instrumentation (slow code): No uuid support: Yes systemd support: Yes Config file: /etc/rsyslog.conf PID file: /var/run/rsyslogd.pid Number of Bits in RainerScript integers: 64

See https://www.rsyslog.com for more information.

Takes less than 30 mins

Environment

Have mentioned above

I have tried with ossl but it fails too, Apr 23 07:36:45 scspr2978654016 rsyslogd[10043]: SSL_ERROR_SSL Error in 'osslRecordRecv': 'error:00000001:lib(0):func(0):reason(1)(1)' with ret=-1 [v8.2102.0-15.el8] Apr 23 07:36:45 scspr2978654016 rsyslogd[10043]: nsd_ossl:OpenSSL Error Stack: error:1408F119:SSL routines:ssl3_get_record:decryption failed or bad record mac [v8.2102.0-15.el8] Apr 23 07:36:45 scspr2978654016 rsyslogd[10043]: netstream session 0x7f4f90006710 from 10.234.189.118 will be closed due to error [v8.2102.0-15.el8] Apr 23 07:36:45 scspr2978654016 rsyslogd[10043]: SSL_ERROR_SSL Error in 'osslEndSess': 'error:00000001:lib(0):func(0):reason(1)(1)' with ret=-1 [v8.2102.0-15.el8] Apr 23 07:36:45 scspr2978654016 rsyslogd[10043]: nsd_ossl:OpenSSL Error Stack: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init [v8.2102.0-15.el8] Apr 23 07:36:45 scspr2978654016 rsyslogd[10043]: nsd_ossl:TLS session terminated with remote syslog server. [v8.2102.0-15.el8] Apr 23 07:36:46 scspr2978654016 rsyslogd[10043]: SSL_ERROR_SYSCALL Error in 'osslRecordRecv': 'error:00000005:lib(0):func(0):DH lib(5)' with ret=-1 [v8.2102.0-15.el8] Apr 23 07:36:46 scspr2978654016 rsyslogd[10043]: nsd_ossl:TLS session terminated with remote syslog server. [v8.2102.0-15.el8]

Also if it is due to password protected keys then it should fail at beginning only not after some messages are decrypted successfully.