Can't use multi kafka_group sources? Got 'ruby-kafka' error

[p-eye]

my yaml

image:
  repository: "fluent/fluentd-kubernetes-daemonset"
  pullPolicy: "IfNotPresent"
  tag: "v1.15.3-debian-opensearch-1.1"

plugins: 
  - fluent-plugin-kafka

fileConfigs:
  01_source.conf: |-
    ## logs from kafka
    <source>
      @type kafka_group
      @id 1
      brokers my_broker:9092
      topics my_topic
      consumer_group my_consumer_group
      format json
      time_source kafka
    </source>
  02_source.conf: |-
    ## logs from kafka
    <source>
      @type kafka_group
      @id 2
      brokers my_broker2:9092
      topics my_topic2
      consumer_group my_consumer_group2
      format json
      time_source kafka
    </source>

OR

image:
  repository: "fluent/fluentd-kubernetes-daemonset"
  pullPolicy: "IfNotPresent"
  tag: "v1.15.3-debian-opensearch-1.1"

plugins: 
  - fluent-plugin-kafka

fileConfigs:
  01_source.conf: |-
    ## logs from kafka
    <source>
      @type kafka_group
      @id 1
      brokers my_broker:9092
      topics my_topic
      consumer_group my_consumer_group
      format json
      time_source kafka
    </source>
    <source>
      @type kafka_group
      @id 2
      brokers my_broker2:9092
      topics my_topic2
      consumer_group my_consumer_group2
      format json
      time_source kafka
    </source>

Hi, when I try to use multi kafka_group sources with yaml above, the app is broken with error

#<Thread:0x00007fdfe6c76280 /fluentd/vendor/bundle/ruby/3.1.0/gems/ruby-kafka-1.5.0/lib/kafka/fetcher.rb:57 run> terminated with exception (report_on_exception is true):
fluentd-dev-fluentd-5c6bb8cc76-5s8vc: /fluentd/vendor/bundle/ruby/3.1.0/gems/ruby-kafka-1.5.0/lib/kafka/lz4_codec.rb:16:in `rescue in load': using lz4 compression requires adding a dependency on the `extlz4` gem to your Gemfile. (LoadError)

And now, I cannot find 'extlz4' in https://github.com/fluent/fluentd-kubernetes-daemonset/blob/master/docker-image/v1.15/debian-kafka/Gemfile Adding 'extlz4' to Gemfile is the solution? or any ways for multi kafka_group sources?

thanks

[github-actions[bot]]

This issue has been automatically marked as stale because it has been open 90 days with no activity. Remove stale label or comment or this issue will be closed in 30 days

[github-actions[bot]]

This issue was automatically closed because of stale in 30 days