search

logstash-plugins / logstash-integration-kafka

Kafka Integration for Logstash, providing Input and Output Plugins
Apache License 2.0
32 stars 60 forks source link

"Can't modify frozen String" For Kafka Tombstone #155

Closed jpohl41 closed 8 months ago

jpohl41 commented 8 months ago

Logstash information:

Please include the following information:

  1. Logstash version (e.g. bin/logstash --version) 8.11.3
  2. Logstash installation source (e.g. built from source, with a package manager: DEB/RPM, expanded from tar or zip archive, docker) Dockerized .tar.gz
  3. How is Logstash being run (e.g. as a service/service manager: systemd, upstart, etc. Via command line, docker/kubernetes) supervisord
  4. How was the Logstash Plugin installed Default install

JVM (e.g. java -version): openjdk 17.0.9

OS version (uname -a if on a Unix-like system): Alpine linux docker

Description of the problem including expected versus actual behavior: Actual Behavior: When sending a Kafka "tombstone" record (a.k.a. a record with a null value) I am seeing the following failure:

2023-11-21T00:01:36,761][ERROR][org.logstash.Logstash    ] uncaught exception (in thread kafka-input-worker-logstash-jobs-kafka-topic-0)
org.jruby.exceptions.FrozenError: (FrozenError) can't modify frozen String
        at org.jruby.RubyString.force_encoding(org/jruby/RubyString.java:6659) ~[jruby.jar:?]
        at org.jruby.RubyString.force_encoding(org/jruby/RubyString.java:6655) ~[jruby.jar:?]
        at opt.logstash.logstash_minus_core.lib.logstash.util.charset.convert(/opt/logstash/logstash-core/lib/logstash/util/charset.rb:29) ~[?:?]
        at opt.logstash.vendor.bundle.jruby.$3_dot_1_dot_0.gems.logstash_minus_codec_minus_plain_minus_3_dot_1_dot_0.lib.logstash.codecs.plain.decode(/opt/logstash/vendor/bundle/jruby/3.1.0/gems/logstash-codec-plain-3.1.0/lib/logstash/codecs/plain.rb:50) ~[?:?]
        at opt.logstash.logstash_minus_core.lib.logstash.codecs.delegator.decode(/opt/logstash/logstash-core/lib/logstash/codecs/delegator.rb:62) ~[?:?]
        at org.logstash.instrument.metrics.AbstractSimpleMetricExt.time(org/logstash/instrument/metrics/AbstractSimpleMetricExt.java:74) ~[logstash-core.jar:?]
        at org.logstash.instrument.metrics.AbstractNamespacedMetricExt.time(org/logstash/instrument/metrics/AbstractNamespacedMetricExt.java:68) ~[logstash-core.jar:?]
        at opt.logstash.logstash_minus_core.lib.logstash.codecs.delegator.decode(/opt/logstash/logstash-core/lib/logstash/codecs/delegator.rb:61) ~[?:?]
        at opt.logstash.vendor.bundle.jruby.$3_dot_1_dot_0.gems.logstash_minus_integration_minus_kafka_minus_11_dot_3_dot_2_minus_java.lib.logstash.inputs.kafka.handle_record(/opt/logstash/vendor/bundle/jruby/3.1.0/gems/logstash-integration-kafka-11.3.2-java/lib/logstash/inputs/kafka.rb:359) ~[?:?]
        at opt.logstash.vendor.bundle.jruby.$3_dot_1_dot_0.gems.logstash_minus_integration_minus_kafka_minus_11_dot_3_dot_2_minus_java.lib.logstash.inputs.kafka.thread_runner(/opt/logstash/vendor/bundle/jruby/3.1.0/gems/logstash-integration-kafka-11.3.2-java/lib/logstash/inputs/kafka.rb:329) ~[?:?]
        at opt.logstash.vendor.bundle.jruby.$3_dot_1_dot_0.gems.logstash_minus_integration_minus_kafka_minus_11_dot_3_dot_2_minus_java.lib.logstash.inputs.kafka.thread_runner(/opt/logstash/vendor/bundle/jruby/3.1.0/gems/logstash-integration-kafka-11.3.2-java/lib/logstash/inputs/kafka.rb:329) ~[?:?]

Expected Behavior: I would expect to be able to handle "tombstone" records - this does not seem to have been an issue with Logstash v8.9.2 running plugin version 11.2.1.

Inside my Docker container running Logstash v8.9.2, ruby --version shows:

jruby 9.3.10.0 (2.6.8) 2023-02-01

Inside my Docker container running Logstash v8.11.3, ruby --version shows:

jruby 9.4.5.0 (3.1.4) 2023-11-02

I wonder if a solution similar to https://github.com/logstash-plugins/logstash-filter-mutate/commit/b0b923fec168a2cdf98a0fa1bdb6325bcf6ce51d could be implemented here?

Steps to reproduce:

Please include a minimal but complete recreation of the problem, including (e.g.) pipeline definition(s), settings, locale, etc. The easier you make for us to reproduce it, the more likely that somebody will take the time to look at it.

  1. Send a record with a String key and a null value (e.g. "key1":null)
  2. See logged stacktrace
edmocosta commented 8 months ago

Hi @jpohl41!

Thank you very for reporting! This issue is caused by this Ruby change https://bugs.ruby-lang.org/issues/16150, which made boolean, module name & nil .to_s values frozen by default.

We can indeed apply the same https://github.com/logstash-plugins/logstash-filter-mutate/commit/b0b923fec168a2cdf98a0fa1bdb6325bcf6ce51d solution here. I'll be working on it soon and will let you know once we have it fixed :)

edmocosta commented 8 months ago

Fix released on version 11.3.3. Upgrading the plugin should solve the problem:

bin/logstash-plugin install --version 11.3.3 logstash-integration-kafka