search

logstash-plugins / logstash-codec-protobuf

Codec plugin for parsing Protobuf messages
Apache License 2.0
26 stars 16 forks source link

Check encoder result empty #55

Closed IngaFeick closed 4 years ago

IngaFeick commented 4 years ago

This PR addresses the issue that if the pb encoding fails the pipeline stops. In order to reproduce just add a field to the event which is not present in the protobuf definition.

[WARN ] 2020-06-10 13:43:38.952 [[rum-akamai2bi]>worker1] protobuf - Protobuf encoding error 1: Argument error (#<ArgumentError: field bot is not found>). Reason: probably mismatching protobuf definition.       Required fields in the protobuf definition are: user_agent, geo, dom, active_ctests, header, timestamp, domain, tracking_id and fields must not begin with @ sign. The event has been discarded.
[ERROR] 2020-06-10 13:43:38.954 [[rum-akamai2bi]>worker1] WorkerLoop - Exception in pipelineworker, the pipeline stopped processing new events, please check your filter configuration and restart Logstash.
org.jruby.exceptions.NoMethodError: (NoMethodError) undefined method `to_java_bytes' for #<LogStash::Logging::Logger:0x57183637>
    at usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_output_minus_kafka_minus_8_dot_0_dot_1.lib.logstash.outputs.kafka.register(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-kafka-8.0.1/lib/logstash/outputs/kafka.rb:195) ~[?:?]
    at usr.share.logstash.vendor.local_gems.$26ab4dac.logstash_minus_codec_minus_protobuf_minus_1_dot_2_dot_2.lib.logstash.codecs.protobuf.encode(/usr/share/logstash/vendor/local_gems/26ab4dac/logstash-codec-protobuf-1.2.2/lib/logstash/codecs/protobuf.rb:228) ~[?:?]
    at usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_output_minus_kafka_minus_8_dot_0_dot_1.lib.logstash.outputs.kafka.multi_receive(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-kafka-8.0.1/lib/logstash/outputs/kafka.rb:217) ~[?:?]
    at org.jruby.RubyArray.each(org/jruby/RubyArray.java:1792) ~[jruby-complete-9.2.7.0.jar:?]
    at usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_output_minus_kafka_minus_8_dot_0_dot_1.lib.logstash.outputs.kafka.multi_receive(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-kafka-8.0.1/lib/logstash/outputs/kafka.rb:215) ~[?:?]
    at org.logstash.config.ir.compiler.OutputStrategyExt$AbstractOutputStrategyExt.multi_receive(org/logstash/config/ir/compiler/OutputStrategyExt.java:118) ~[logstash-core.jar:?]
    at org.logstash.config.ir.compiler.AbstractOutputDelegatorExt.multi_receive(org/logstash/config/ir/compiler/AbstractOutputDelegatorExt.java:101) ~[logstash-core.jar:?]
    at usr.share.logstash.logstash_minus_core.lib.logstash.java_pipeline.start_workers(/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:235) ~[?:?]
[FATAL] 2020-06-10 13:43:39.065 [LogStash::Runner] runner - An unexpected error occurred! {:error=>java.lang.IllegalStateException: org.jruby.exceptions.NoMethodError: (NoMethodError) undefined method `to_java_bytes' for #<LogStash::Logging::Logger:0x57183637>, :backtrace=>["org.logstash.execution.WorkerLoop.run(org/logstash/execution/WorkerLoop.java:85)", "java.lang.reflect.Method.invoke(java/lang/reflect/Method.java:498)", "org.jruby.javasupport.JavaMethod.invokeDirectWithExceptionHandling(org/jruby/javasupport/JavaMethod.java:425)", "org.jruby.javasupport.JavaMethod.invokeDirect(org/jruby/javasupport/JavaMethod.java:292)", "usr.share.logstash.logstash_minus_core.lib.logstash.java_pipeline.start_workers(/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:235)", "org.jruby.RubyProc.call(org/jruby/RubyProc.java:295)", "org.jruby.RubyProc.call(org/jruby/RubyProc.java:274)", "org.jruby.RubyProc.call(org/jruby/RubyProc.java:270)", "java.lang.Thread.run(java/lang/Thread.java:748)"]}