rdkafka 0.12.0 is incompatible with fluent-plugin-kafka v0.18.0 or former

[bertpassek]

Describe the bug

We noticed a null pointer exception when td-agent is being restartet. But so far everything is working properly, no side effects , just this exxception stracktrace.

To Reproduce

restart td-agent

Expected behavior

no exception

Your Environment

- Fluentd version: 1.15.1
- TD Agent version:
- Operating system: 20.04.4 LTS (Focal Fossa)
- Kernel version: 5.13.0-1031-aws

Your Configuration

<match home.tracking.**>
    @type "rdkafka2"
    @id "out_kafka"
    @log_level "warn"
    brokers "..."
    use_event_time "true"
    topic_key "topic"
    default_topic "unknown"
    required_acks 1
    max_enqueue_retries 60
    enqueue_retry_backoff 5
    compression_codec "snappy"
    rdkafka_buffering_max_ms 10
    rdkafka_buffering_max_messages 10000000
    rdkafka_message_max_num 1000
    max_send_retries 2147483647
    ack_timeout 60
    rdkafka_delivery_handle_poll_timeout 0

    <format >
      @type "json"
    </format>

    <buffer topic>
      @type "file"
      chunk_limit_size "256MB"
      flush_interval "5s"
      flush_at_shutdown "true"
      flush_thread_count 4
      retry_forever "true"
      retry_wait "5s"
      total_limit_size "2G"
      overflow_action "throw_exception"
      compress "gzip"
    </buffer>

  </match>

Your Error Log

2022-08-01 10:56:20 +0000 [warn]: #0 unexpected error while shutting down on output plugin plugin=Fluent::Rdkafka2Output plugin_id="out_kafka" error_class=NoMethodError error="undefined method `join' for nil:NilClass"
  2022-08-01 10:56:20 +0000 [error]: #0 /var/lib/gems/2.7.0/gems/fluent-plugin-kafka-0.18.0/lib/fluent/plugin/out_rdkafka2.rb:330:in `block (3 levels) in shutdown_producers'
  2022-08-01 10:56:20 +0000 [warn]: #0 /var/lib/gems/2.7.0/gems/fluent-plugin-kafka-0.18.0/lib/fluent/plugin/out_rdkafka2.rb:15:in `close'
  2022-08-01 10:56:20 +0000 [warn]: #0 /var/lib/gems/2.7.0/gems/fluent-plugin-kafka-0.18.0/lib/fluent/plugin/out_rdkafka2.rb:342:in `close_producer'
  2022-08-01 10:56:20 +0000 [warn]: #0 /var/lib/gems/2.7.0/gems/fluent-plugin-kafka-0.18.0/lib/fluent/plugin/out_rdkafka2.rb:330:in `block (3 levels) in shutdown_producers'

Additional context

No response

[ashie]

It seems caused at

https://github.com/fluent/fluent-plugin-kafka/blob/eb740212711b1ca1b01089f3feb4e71f2deca381/lib/fluent/plugin/out_rdkafka2.rb#L328-L331 https://github.com/fluent/fluent-plugin-kafka/blob/eb740212711b1ca1b01089f3feb4e71f2deca381/lib/fluent/plugin/out_rdkafka2.rb#L341-L344 rdkafa v0.12.0 seems including incompatible change against above code, Rdkafka::Producer no longer has @polling_thread. https://github.com/fluent/fluent-plugin-kafka/blob/eb740212711b1ca1b01089f3feb4e71f2deca381/lib/fluent/plugin/out_rdkafka2.rb#L14-L19

Are you using rdkafka v0.12.0?

[ashie]

Are you using td-agent 4.4.0? If so, bundle rdkafka is v0.11.1 and the above change isn't included yet, so that the cause of your problem isn't concerned with rdkafka v0.12.0's change.

[bertpassek]

we have installed fluentd gem 1.15.1 and rdkafka is 0.12.0

[ashie]

we have installed fluentd gem 1.15.1 and rdkafka is 0.12.0

Thanks. Please downgrade to 0.11.x or former.

[bertpassek]

Thanks, we will downgrade to 0.11.x

[github-actions[bot]]

This issue has been automatically marked as stale because it has been open 90 days with no activity. Remove stale label or comment or this issue will be closed in 30 days

[github-actions[bot]]

This issue was automatically closed because of stale in 30 days