elastic / logstash

Logstash - transport and process your logs, events, or other data
https://www.elastic.co/products/logstash
Other
72 stars 3.5k forks source link

logstash event cannot new with ruby hash(which has the integer value) #10185

Open WoolenWang opened 5 years ago

WoolenWang commented 5 years ago

Please post all product and debugging questions on our forum. Your questions will reach our wider community members there, and if we confirm that there is a bug, then we can open a new issue here.

For security vulnerabilities please only send reports to security@elastic.co. See https://www.elastic.co/community/security for more information.

Logstash Plugins are located in a different organization: https://github.com/logstash-plugins. For bugs on specific Logstash plugins, for example, if Redis Output has a defect, please open it in the respective Redis Output repository.

For all general issues, please provide the following details for fast resolution:

code:: `ruby

event_msg_array.each do |one_msg|
  event = LogStash::Event.new
  one_msg.each do |k, v|
    begin
      event.set(k, v)
    rescue Exception => ex
      error "出现异常:#{k}=>#{v}", ex
    end
  end
  yield event
end

`

the event_msg_array is a hash array each one is a hash object, in my codec plugin , i have try the event set api and also put the hash in LogStash::Event.new(one_msg) get the same java exception flow "Missing Converter handling for full class name=org.jruby.RubyObjectVar3"

is that the logstash event valuefier can not handle the integer value??

ERROR ,Thread::#<Thread:0xf6737c>,18-11-30 09:56:30 /src/logstash/codecs/code_log.rb:50:inblock in decode': 出现异常:thread_id=>24660 Missing Converter handling for full class name=org.jruby.RubyObjectVar3, simple name=RubyObjectVar3 org.logstash.Valuefier.fallbackConvert(Valuefier.java:97) org.logstash.Valuefier.convert(Valuefier.java:75) org.logstash.ext.JrubyEventExtLibrary$RubyEvent.ruby_set_field(JrubyEventExtLibrary.java:99) E3a.git.code_helper.src.logstash.codecs.code_log.RUBY$block$decode$2(E:/git/code_helper/src/logstash/codecs/code_log.rb:48) org.jruby.runtime.CompiledIRBlockBody.yieldDirect(CompiledIRBlockBody.java:156) org.jruby.runtime.BlockBody.yield(BlockBody.java:114) org.jruby.runtime.Block.yield(Block.java:165) org.jruby.RubyHash$12.visit(RubyHash.java:1362) org.jruby.RubyHash$12.visit(RubyHash.java:1359) org.jruby.RubyHash.visitLimited(RubyHash.java:662) org.jruby.RubyHash.visitAll(RubyHash.java:647) org.jruby.RubyHash.iteratorVisitAll(RubyHash.java:1319) org.jruby.RubyHash.each_pairCommon(RubyHash.java:1354) org.jruby.RubyHash.each(RubyHash.java:1343) E3a.git.code_helper.src.logstash.codecs.code_log.RUBY$block$decode$1(E:/git/code_helper/src/logstash/codecs/code_log.rb:46) org.jruby.runtime.CompiledIRBlockBody.yieldDirect(CompiledIRBlockBody.java:156) org.jruby.runtime.BlockBody.yield(BlockBody.java:114) org.jruby.runtime.Block.yield(Block.java:165) org.jruby.RubyArray.each(RubyArray.java:1734) E3a.git.code_helper.src.logstash.codecs.code_log.RUBY$method$decode$0(E:/git/code_helper/src/logstash/codecs/code_log.rb:44) E3a.git.code_helper.$3part.ruby.logstash_minus_6_dot_5_dot_1.vendor.jruby.lib.ruby.gems.shared.gems.logstash_minus_input_minus_udp_minus_3_dot_3_dot_4.lib.logstash.inputs.udp.RUBY$method$inputworker$0(E:/git/code_helper/3part/ruby/logstash-6.5.1/vendor/jruby/lib/ruby/gems/shared/gems/logstash-input-udp-3.3.4/lib/logstash/inputs/udp.rb:151) E3a.git.code_helper.$3part.ruby.logstash_minus_6_dot_5_dot_1.vendor.jruby.lib.ruby.gems.shared.gems.logstash_minus_input_minus_udp_minus_3_dot_3_dot_4.lib.logstash.inputs.udp.RUBY$method$inputworker$0$VARARGS(E:/git/code_helper/3part/ruby/logstash-6.5.1/vendor/jruby/lib/ruby/gems/shared/gems/logstash-input-udp-3.3.4/lib/logstash/inputs/udp.rb) org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:77) org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:93) org.jruby.ir.targets.InvokeSite.invoke(InvokeSite.java:145) E3a.git.code_helper.$3part.ruby.logstash_minus_6_dot_5_dot_1.vendor.jruby.lib.ruby.gems.shared.gems.logstash_minus_input_minus_udp_minus_3_dot_3_dot_4.lib.logstash.inputs.udp.RUBY$block$run$2(E:/git/code_helper/3part/ruby/logstash-6.5.1/vendor/jruby/lib/ruby/gems/shared/gems/logstash-input-udp-3.3.4/lib/logstash/inputs/udp.rb:63) org.jruby.runtime.CompiledIRBlockBody.callDirect(CompiledIRBlockBody.java:145) org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:71) org.jruby.runtime.Block.call(Block.java:124) org.jruby.RubyProc.call(RubyProc.java:289) org.jruby.RubyProc.call(RubyProc.java:246) org.jruby.internal.runtime.RubyRunnable.run(RubyRunnable.java:104) `

WoolenWang commented 5 years ago

Oh, i have solve this problem with deep analysis, because the thread_id is come from the gem bindata record, so it seems that the hash converter is not understand this type of object, just call to_i method before new the logstash event is ok