search

logstash-plugins / logstash-integration-kafka

Kafka Integration for Logstash, providing Input and Output Plugins
Apache License 2.0
32 stars 60 forks source link

Fix "Can't modify frozen string" error when record value is `nil` #156

Closed edmocosta closed 10 months ago

edmocosta commented 10 months ago

This PR fixes the issue caused by the Ruby change https://bugs.ruby-lang.org/issues/16150, which made boolean, module name & nil .to_s values frozen by default, making this plugin unable to process Kafka "tombstone" records (a.k.a. a record with a null value).

How to test it locally?

1 - Set up the Kafka test environment:

./kafka_test_setup.sh

2 - Run the following pipeline config:

 - pipeline.id: kafka
   config.string: |
     input {
       kafka {
         topics => "logstash_integration_topic_plain"
       }
     }

    output { stdout { codec => rubydebug { metadata => true } } }

3 - Produce a message with a null content:

echo "NULL" | build/kafka/bin/kafka-console-producer.sh --topic logstash_integration_topic_plain --broker-list localhost:9092 --property null.marker=NULL

Using the main branch code, it should terminate the pipeline and log an error similar to:

Exception in thread "kafka-input-worker-logstash-0" org.jruby.exceptions.FrozenError: (FrozenError) can't modify frozen String
    at org.jruby.RubyString.force_encoding(org/jruby/RubyString.java:6667)
    at org.jruby.RubyString.force_encoding(org/jruby/RubyString.java:6663)

The error should not happen using the fixed version.

Closes: https://github.com/logstash-plugins/logstash-integration-kafka/issues/155