joekiller / logstash-kafka

Kafka plugin for Logstash
Apache License 2.0
206 stars 63 forks source link

NoMethodError: private method `sprintf' called for #<String:0x6c7662ca> #51

Closed klyr closed 9 years ago

klyr commented 9 years ago

Hello,

I'm having an issue since tag v0.7.3 and the output kafka plugin. Logstash package generated with make tarball from v0.7.3.tgz

Minimal configuration:

input {
   stdin {}
}

output {

    stdout {}

    kafka {
      broker_list => "127.0.0.1:9092"
      topic_id => "logstash.catchall"
    }
}

When running I got a NoMethodError exception for the sprintfmethod:

Using milestone 1 output plugin 'kafka'. This plugin should work, but would benefit from use by folks like you. Please let us know if you find bugs or have suggestions on how to improve this plugin.  For more information on plugin milestones, see http://logstash.net/docs/1.4.2/plugin-milestones {:level=>:warn}
This is a test
2015-02-25T14:12:11.032+0000 test-elk-install This is a test
kafka producer threw exception, restarting {:exception=>#<NoMethodError: private method `sprintf' called for #<String:0x6c7662ca>>, :level=>:warn}
joekiller commented 9 years ago

Yeah use 0.7.0 for now. I forgot that sprintf was part of Logstash 1.5+. I'm going to try to clean this back to logstash 1.4 centric as the logstash-output-kafka and logstash-input-kafka libraries are needed for 1.5

AdeMiller commented 9 years ago

I installed 0.7.3 and ran into this issue and tried to revert to 0.7.0 by removing the 0.7.3 logstash-kafka gem and the doing a complete reinstall of logstash and the 0.7.0 gem. If seems like message deserialization is broken. I get the following messages as output.

{"message":"MessageAndMetadata(ateam_appfog,0,Message(magic = 0, attributes = 0, 
crc = 3178577346, key = java.nio.HeapByteBuffer[pos=0 lim=4 cap=412], payload = java.nio.HeapByteBuffer[pos=0 lim=404 cap=404]),614672,
kafka.serializer.DefaultDecoder@169446df,kafka.serializer.DefaultDecoder@60b8e60d)",
"@version":"1","@timestamp":"2015-04-09T08:31:41.431Z","type":"########","location":"###","coords":[####,####]}

My input configuration is unchanged and is as follows:

    kafka {
        zk_connect => "kafka-LLL-01.DDD.dom:2181,kafka-LLL-02.DDD.dom:2181,kafka-LLL-03.DDD.dom:2181"
        topic_id => "analytics_heartbeat"
        group_id => "logstash_LLL_analytics_heartbeat"
        consumer_id => "logstash_LLL_NNN_analytics_heartbeat"
        type => "analytics_heartbeat"
        reset_beginning => false
        consumer_threads => 4
        queue_size => 40
        rebalance_max_retries => 4
        rebalance_backoff_ms => 2000
        consumer_timeout_ms => -1
        consumer_restart_on_error => true
        consumer_restart_sleep_ms => 0
        decorate_events => false
        fetch_message_max_bytes => 1048576
        tags => [ "LLL" ]
    }

(#### are mine)

I'm working on the basis that this is some sort of dependency version issue with something that got installed with 0.7.3 and is now being used by 0.7.0 but some pointers would help. It also suggests your 0.7.0 dependencies are not declared correctly?

joekiller commented 9 years ago

Sorry for the trouble. I have updated the project to work with logstash 1.4.2 again. The sprintf features had to go but the plugin build is working. I'll try to get a new gem pushed tonight or tomorrow.

joekiller commented 9 years ago

Let me know if you still have problems. I'm going to close this for now.

AdeMiller commented 9 years ago

I actually tried to upgrade to 1.5 rather than try the 1.4.2 route again. Closing it is fine. Thanks!