search

fluent / fluentd

Fluentd: Unified Logging Layer (project under CNCF)
https://www.fluentd.org
Apache License 2.0
12.93k stars 1.34k forks source link

Fluentd is not coming up after installing openssl gem on RHEL 8 #4690

Open deepaksb2 opened 3 weeks ago

deepaksb2 commented 3 weeks ago

Describe the bug

I installed fluentd 5 from below script to see if CVE-2024-5535 is resolved or not.

curl -fsSL https://toolbelt.treasuredata.com/sh/install-redhat-fluent-package5-lts.sh | sh

I can still see the output of /opt/td-agent/lib/ruby/2.7.0/x86_64-linux/strings openssl.so | grep OpenSSL shows OpenSSL 1.1.1k FIPS 25 Mar 2021

I attempted to install openssl gem to see if that points latest openssl and resolve the issue command: 

/opt/fluent/bin/ruby gem install openssl
Fetching openssl-3.2.0.gem
Building native extensions. This could take a while...
Successfully installed openssl-3.2.0
Parsing documentation for openssl-3.2.0
Installing ri documentation for openssl-3.2.0
Done installing documentation for openssl after 2 seconds
1 gem installed

After that fluentd start is failing with below error 

journalctl -xe
Oct 21 15:04:28 clm-pun-vie3yf fluentd[451320]:         from /opt/fluent/lib/ruby/gems/3.2.0/gems/fluentd-1.16.5/lib/fluent/event_router.rb:19:in `<top (required)>'
Oct 21 15:04:28 clm-pun-vie3yf fluentd[451320]:         from <internal:/opt/fluent/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:86:in `require'
Oct 21 15:04:28 clm-pun-vie3yf fluentd[451320]:         from /opt/fluent/lib/ruby/gems/3.2.0/gems/fluentd-1.16.5/lib/fluent/engine.rb:19:in `<top (required)>'
Oct 21 15:04:28 clm-pun-vie3yf fluentd[451320]:         from <internal:/opt/fluent/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:86:in `require'
Oct 21 15:04:28 clm-pun-vie3yf fluentd[451320]:         from /opt/fluent/lib/ruby/gems/3.2.0/gems/fluentd-1.16.5/lib/fluent/supervisor.rb:24:in `<top (required)>'
Oct 21 15:04:28 clm-pun-vie3yf fluentd[451320]:         from <internal:/opt/fluent/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:86:in `require'
Oct 21 15:04:28 clm-pun-vie3yf fluentd[451320]:         from /opt/fluent/lib/ruby/gems/3.2.0/gems/fluentd-1.16.5/lib/fluent/command/fluentd.rb:19:in `<top (required)>'
Oct 21 15:04:28 clm-pun-vie3yf fluentd[451320]:         from <internal:/opt/fluent/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:86:in `require'
Oct 21 15:04:28 clm-pun-vie3yf fluentd[451320]:         from /opt/fluent/lib/ruby/gems/3.2.0/gems/fluentd-1.16.5/bin/fluentd:15:in `<top (required)>'
Oct 21 15:04:28 clm-pun-vie3yf fluentd[451320]:         from /opt/fluent/bin/fluentd:25:in `load'
Oct 21 15:04:28 clm-pun-vie3yf fluentd[451320]:         from /opt/fluent/bin/fluentd:25:in `<main>'

To Reproduce

Install fluentd : curl -fsSL https://toolbelt.treasuredata.com/sh/install-redhat-fluent-package5-lts.sh | sh

install openssl gem

ruby gem install openssl

Fluentd does not start

Expected behavior

Fluend should be start after installing openssl gem

Is the standalone fluentd always use system openssl ? How to mitigate if any openssl vulnerability is reported? CVE-2024-5535 in my case.

Your Environment

Fluentd version: fluent-package-5.1.0-1.el8.x86_64.rpm
- TD Agent version:td-agent-4.5.2-1.el8.x86_64.rpm
- Fluent Package version:
- Docker image (tag):
- Operating system: RHEL 8
- Kernel version:

OS Release details:
NAME="Red Hat Enterprise Linux"
VERSION="8.10 (Ootpa)"
ID="rhel"
ID_LIKE="fedora"
VERSION_ID="8.10"
PLATFORM_ID="platform:el8"
PRETTY_NAME="Red Hat Enterprise Linux 8.10 (Ootpa)"
ANSI_COLOR="0;31"

openssl version -a
OpenSSL 1.1.1k FIPS 25 Mar 2021
built on: Thu Nov 30 13:05:10 2023 UTC
platform: linux-x86_64

Your Configuration

Standard configuration.

Your Error Log

journalctl -xe
Oct 21 15:04:28 clm-pun-vie3yf fluentd[451320]:         from /opt/fluent/lib/ruby/gems/3.2.0/gems/fluentd-1.16.5/lib/fluent/event_router.rb:19:in `<top (required)>'
Oct 21 15:04:28 clm-pun-vie3yf fluentd[451320]:         from <internal:/opt/fluent/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:86:in `require'
Oct 21 15:04:28 clm-pun-vie3yf fluentd[451320]:         from /opt/fluent/lib/ruby/gems/3.2.0/gems/fluentd-1.16.5/lib/fluent/engine.rb:19:in `<top (required)>'
Oct 21 15:04:28 clm-pun-vie3yf fluentd[451320]:         from <internal:/opt/fluent/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:86:in `require'
Oct 21 15:04:28 clm-pun-vie3yf fluentd[451320]:         from /opt/fluent/lib/ruby/gems/3.2.0/gems/fluentd-1.16.5/lib/fluent/supervisor.rb:24:in `<top (required)>'
Oct 21 15:04:28 clm-pun-vie3yf fluentd[451320]:         from <internal:/opt/fluent/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:86:in `require'
Oct 21 15:04:28 clm-pun-vie3yf fluentd[451320]:         from /opt/fluent/lib/ruby/gems/3.2.0/gems/fluentd-1.16.5/lib/fluent/command/fluentd.rb:19:in `<top (required)>'
Oct 21 15:04:28 clm-pun-vie3yf fluentd[451320]:         from <internal:/opt/fluent/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:86:in `require'
Oct 21 15:04:28 clm-pun-vie3yf fluentd[451320]:         from /opt/fluent/lib/ruby/gems/3.2.0/gems/fluentd-1.16.5/bin/fluentd:15:in `<top (required)>'
Oct 21 15:04:28 clm-pun-vie3yf fluentd[451320]:         from /opt/fluent/bin/fluentd:25:in `load'
Oct 21 15:04:28 clm-pun-vie3yf fluentd[451320]:         from /opt/fluent/bin/fluentd:25:in `<main>'

Additional context

The actual problem statement is to how to mitigate CVE-2024-5535 vulnerability.

daipom commented 3 weeks ago

fluent-package and td-agent (Linux version) uses system openssl. So, you need to update system openssl (not Ruby gem).

kenhys commented 3 weeks ago

According to https://access.redhat.com/errata/RHSA-2024:7848, it seems openssl-1.1.1k-14 is available for CVE-2024-5535.

deepaksb2 commented 3 weeks ago

fluent-package and td-agent (Linux version) uses system openssl. So, you need to update system openssl (not Ruby gem).

ok, thanks for the reply, is it required to reinstall fluentd post System openssl upgrade? would strings command to /opt/././x86_64-linux to openssl.so gives the updated result?

daipom commented 3 weeks ago

is it required to reinstall fluentd post System openssl upgrade?

You don't need to reinstall Fluentd. Restarting the service just in case would be enough.

daipom commented 3 weeks ago

would strings command to /opt/././x86_64-linux to openssl.so gives the updated result?

Sorry, I don't know it... Please check it out and let me know if something is wrong.

Watson1978 commented 3 weeks ago

I tried it on AlmaLinux 8 and it is no problem to start Fluentd with the openssl gem new version.

[root@test ~]# cat /etc/os-release
NAME="AlmaLinux"
VERSION="8.10 (Cerulean Leopard)"
ID="almalinux"
ID_LIKE="rhel centos fedora"
VERSION_ID="8.10"
PLATFORM_ID="platform:el8"
PRETTY_NAME="AlmaLinux 8.10 (Cerulean Leopard)"
ANSI_COLOR="0;34"
LOGO="fedora-logo-icon"
CPE_NAME="cpe:/o:almalinux:almalinux:8::baseos"
HOME_URL="https://almalinux.org/"
DOCUMENTATION_URL="https://wiki.almalinux.org/"
BUG_REPORT_URL="https://bugs.almalinux.org/"

ALMALINUX_MANTISBT_PROJECT="AlmaLinux-8"
ALMALINUX_MANTISBT_PROJECT_VERSION="8.10"
REDHAT_SUPPORT_PRODUCT="AlmaLinux"
REDHAT_SUPPORT_PRODUCT_VERSION="8.10"
SUPPORT_END=2029-06-01
[root@test ~]#
[root@test ~]#
[root@test ~]#
[root@test ~]#
[root@test ~]# curl -fsSL https://toolbelt.treasuredata.com/sh/install-redhat-fluent-package5-lts.sh | sh
==============================
 fluent-package Installation Script 
==============================
This script requires superuser access to install rpm packages.
You will be prompted for your password by sudo.
AlmaLinux 8 - BaseOS                                                                                                                                                                                                   5.7 MB/s | 7.4 MB     00:01    
AlmaLinux 8 - AppStream                                                                                                                                                                                                8.3 MB/s |  14 MB     00:01    
AlmaLinux 8 - Extras                                                                                                                                                                                                    14 kB/s |  13 kB     00:00    
Fluentd Project                                                                                                                                                                                                        1.7 MB/s | 314 kB     00:00    
Dependencies resolved.

--- (snip) ---

Installed:
  fluent-package-5.0.4-1.el8.x86_64                                                                                               tar-2:1.30-9.el8.x86_64                                                                                              

Complete!

Installation completed. Happy Logging!

[root@test ~]#
[root@test ~]#
[root@test ~]#
[root@test ~]#
[root@test ~]# dnf install -y openssl-devel gcc make
Last metadata expiration check: 0:00:20 ago on Fri Nov  1 08:19:49 2024.
Dependencies resolved.
=======================================================================================================================================================================================================================================================
 Package                                                          Architecture                                        Version                                                             Repository                                              Size
=======================================================================================================================================================================================================================================================
Installing:
 gcc                                                              x86_64                                              8.5.0-22.el8_10                                                     appstream                                               23 M
 make                                                             x86_64                                              1:4.2.1-11.el8                                                      baseos                                                 497 k
 openssl-devel                                                    x86_64                                              1:1.1.1k-14.el8_6                                                   baseos                                                 2.3 M

--- (snip) ---

Installed:
  binutils-2.30-123.el8.x86_64                  cpp-8.5.0-22.el8_10.x86_64               gcc-8.5.0-22.el8_10.x86_64          glibc-devel-2.28-251.el8_10.5.x86_64           glibc-headers-2.28-251.el8_10.5.x86_64  isl-0.16.1-6.el8.x86_64       
  kernel-headers-4.18.0-553.22.1.el8_10.x86_64  keyutils-libs-devel-1.5.10-9.el8.x86_64  krb5-devel-1.18.2-29.el8_10.x86_64  libcom_err-devel-1.45.6-5.1.el8.alma.1.x86_64  libkadm5-1.18.2-29.el8_10.x86_64        libmpc-1.1.0-9.1.el8.x86_64   
  libpkgconf-1.4.2-1.el8.x86_64                 libselinux-devel-2.9-8.el8.x86_64        libsepol-devel-2.9-3.el8.x86_64     libverto-devel-0.3.2-2.el8.x86_64              libxcrypt-devel-4.1.1-6.el8.x86_64      make-1:4.2.1-11.el8.x86_64    
  openssl-devel-1:1.1.1k-14.el8_6.x86_64        pcre2-devel-10.32-3.el8_6.x86_64         pcre2-utf16-10.32-3.el8_6.x86_64    pcre2-utf32-10.32-3.el8_6.x86_64               pkgconf-1.4.2-1.el8.x86_64              pkgconf-m4-1.4.2-1.el8.noarch 
  pkgconf-pkg-config-1.4.2-1.el8.x86_64         zlib-devel-1.2.11-25.el8.x86_64         

Complete!
[root@test ~]#
[root@test ~]#
[root@test ~]#
[root@test ~]#
[root@test ~]# /opt/fluent/bin/gem install openssl
Fetching openssl-3.2.0.gem
Building native extensions. This could take a while...
Successfully installed openssl-3.2.0
Parsing documentation for openssl-3.2.0
Installing ri documentation for openssl-3.2.0
Done installing documentation for openssl after 0 seconds
1 gem installed

A new release of RubyGems is available: 3.4.19 → 3.5.22!
Run `gem update --system 3.5.22` to update your installation.

[root@test ~]#
[root@test ~]#
[root@test ~]#
[root@test ~]#
[root@test ~]# /opt/fluent/bin/gem list | grep openssl
openssl (3.2.0, default: 3.1.0)
[root@test ~]#
[root@test ~]#
[root@test ~]#
[root@test ~]#
[root@test ~]# /opt/fluent/bin/fluentd -c /etc/fluent/fluentd.conf 
2024-11-01 08:21:17 +0000 [info]: init supervisor logger path=nil rotate_age=nil rotate_size=nil
2024-11-01 08:21:17 +0000 [info]: parsing config file is succeeded path="/etc/fluent/fluentd.conf"
2024-11-01 08:21:17 +0000 [info]: gem 'fluentd' version '1.16.5'
2024-11-01 08:21:17 +0000 [info]: gem 'fluent-plugin-calyptia-monitoring' version '0.1.3'
2024-11-01 08:21:17 +0000 [info]: gem 'fluent-plugin-elasticsearch' version '5.4.0'
2024-11-01 08:21:17 +0000 [info]: gem 'fluent-plugin-flowcounter-simple' version '0.1.0'
2024-11-01 08:21:17 +0000 [info]: gem 'fluent-plugin-kafka' version '0.19.2'
2024-11-01 08:21:17 +0000 [info]: gem 'fluent-plugin-metrics-cmetrics' version '0.1.2'
2024-11-01 08:21:17 +0000 [info]: gem 'fluent-plugin-opensearch' version '1.1.4'
2024-11-01 08:21:17 +0000 [info]: gem 'fluent-plugin-prometheus' version '2.1.0'
2024-11-01 08:21:17 +0000 [info]: gem 'fluent-plugin-prometheus_pushgateway' version '0.1.1'
2024-11-01 08:21:17 +0000 [info]: gem 'fluent-plugin-record-modifier' version '2.1.1'
2024-11-01 08:21:17 +0000 [info]: gem 'fluent-plugin-rewrite-tag-filter' version '2.4.0'
2024-11-01 08:21:17 +0000 [info]: gem 'fluent-plugin-s3' version '1.7.2'
2024-11-01 08:21:17 +0000 [info]: gem 'fluent-plugin-sd-dns' version '0.1.0'
2024-11-01 08:21:17 +0000 [info]: gem 'fluent-plugin-systemd' version '1.0.5'
2024-11-01 08:21:17 +0000 [info]: gem 'fluent-plugin-td' version '1.2.0'
2024-11-01 08:21:17 +0000 [info]: gem 'fluent-plugin-utmpx' version '0.5.0'
2024-11-01 08:21:17 +0000 [info]: gem 'fluent-plugin-webhdfs' version '1.5.0'
2024-11-01 08:21:17 +0000 [info]: using configuration file: <ROOT>
  <match td.*.*>
    @type tdlog
    @id output_td
    apikey xxxxxx
    auto_create_table 
    <buffer>
      @type "file"
      path "/var/log/fluent/buffer/td"
    </buffer>
    <secondary>
      @type "secondary_file"
      directory "/var/log/fluent/failed_records"
    </secondary>
  </match>
  <match debug.**>
    @type stdout
    @id output_stdout
  </match>
  <source>
    @type forward
    @id input_forward
  </source>
  <source>
    @type http
    @id input_http
    port 8888
  </source>
  <source>
    @type debug_agent
    @id input_debug_agent
    bind "127.0.0.1"
    port 24230
  </source>
</ROOT>
2024-11-01 08:21:17 +0000 [info]: starting fluentd-1.16.5 pid=1949 ruby="3.2.4"
2024-11-01 08:21:17 +0000 [info]: spawn command to main:  cmdline=["/opt/fluent/bin/ruby", "-Eascii-8bit:ascii-8bit", "/opt/fluent/bin/fluentd", "-c", "/etc/fluent/fluentd.conf", "--under-supervisor"]
2024-11-01 08:21:18 +0000 [info]: #0 init worker0 logger path=nil rotate_age=nil rotate_size=nil
2024-11-01 08:21:18 +0000 [info]: adding match pattern="td.*.*" type="tdlog"
2024-11-01 08:21:18 +0000 [info]: adding match pattern="debug.**" type="stdout"
2024-11-01 08:21:18 +0000 [info]: adding source type="forward"
2024-11-01 08:21:18 +0000 [info]: adding source type="http"
2024-11-01 08:21:18 +0000 [info]: adding source type="debug_agent"
2024-11-01 08:21:18 +0000 [info]: #0 starting fluentd worker pid=1954 ppid=1949 worker=0
2024-11-01 08:21:18 +0000 [info]: #0 [input_debug_agent] listening dRuby uri="druby://127.0.0.1:24230" object="Fluent::Engine" worker=0
2024-11-01 08:21:18 +0000 [info]: #0 [input_forward] listening port port=24224 bind="0.0.0.0"
2024-11-01 08:21:18 +0000 [info]: #0 fluentd worker is now running worker=0