search

logstash-plugins / logstash-codec-protobuf

Codec plugin for parsing Protobuf messages
Apache License 2.0
26 stars 16 forks source link

extend is also a reserved word #71

Open recursethis opened 9 months ago

recursethis commented 9 months ago

I ran into a deserialization issue with a proto that had a field name called "extend" It may be best to skip calling send() for any of ruby's reserved words and not just "class"? https://github.com/logstash-plugins/logstash-codec-protobuf/blob/65454f60a43181540dc2f61c35b19bf4200e71a8/lib/logstash/codecs/protobuf.rb#L303

the field name caused a deserialization error with the following stack trace

org.jruby.exceptions.ArgumentError: (ArgumentError) wrong number of arguments (given 0, expected 1)
at usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_codec_minus_protobuf_minus_1_dot_3_dot_0_minus_java.lib.logstash.codecs.protobuf.pb3_to_hash(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-codec-protobuf-1.3.0-java/lib/logstash/codecs/protobuf.rb:303) ~[?:?]
at org.jruby.RubyArray.each(org/jruby/RubyArray.java:1821) ~[jruby-complete-9.2.20.1.jar:?]
at usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_codec_minus_protobuf_minus_1_dot_3_dot_0_minus_java.lib.logstash.codecs.protobuf.pb3_to_hash(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-codec-protobuf-1.3.0-java/lib/logstash/codecs/protobuf.rb:297) ~[?:?]
at usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_codec_minus_protobuf_minus_1_dot_3_dot_0_minus_java.lib.logstash.codecs.protobuf.pb3_to_hash(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-codec-protobuf-1.3.0-java/lib/logstash/codecs/protobuf.rb:306) ~[?:?]
at org.jruby.RubyArray.each(org/jruby/RubyArray.java:1821) ~[jruby-complete-9.2.20.1.jar:?]
at usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_codec_minus_protobuf_minus_1_dot_3_dot_0_minus_java.lib.logstash.codecs.protobuf.pb3_to_hash(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-codec-protobuf-1.3.0-java/lib/logstash/codecs/protobuf.rb:297) ~[?:?]
at usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_codec_minus_protobuf_minus_1_dot_3_dot_0_minus_java.lib.logstash.codecs.protobuf.pb3_to_hash(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-codec-protobuf-1.3.0-java/lib/logstash/codecs/protobuf.rb:306) ~[?:?]
at org.jruby.RubyArray.each(org/jruby/RubyArray.java:1821) ~[jruby-complete-9.2.20.1.jar:?]
at usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_codec_minus_protobuf_minus_1_dot_3_dot_0_minus_java.lib.logstash.codecs.protobuf.pb3_to_hash(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-codec-protobuf-1.3.0-java/lib/logstash/codecs/protobuf.rb:297) ~[?:?]
at usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_codec_minus_protobuf_minus_1_dot_3_dot_0_minus_java.lib.logstash.codecs.protobuf.pb3_to_hash(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-codec-protobuf-1.3.0-java/lib/logstash/codecs/protobuf.rb:318) ~[?:?]
at org.jruby.RubyArray.each(org/jruby/RubyArray.java:1821) ~[jruby-complete-9.2.20.1.jar:?]
at com.google.protobuf.jruby.RubyRepeatedField.each(com/google/protobuf/jruby/RubyRepeatedField.java:318) ~[protobuf_java.jar:?]
at usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_codec_minus_protobuf_minus_1_dot_3_dot_0_minus_java.lib.logstash.codecs.protobuf.pb3_to_hash(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-codec-protobuf-1.3.0-java/lib/logstash/codecs/protobuf.rb:317) ~[?:?]
at usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_codec_minus_protobuf_minus_1_dot_3_dot_0_minus_java.lib.logstash.codecs.protobuf.pb3_to_hash(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-codec-protobuf-1.3.0-java/lib/logstash/codecs/protobuf.rb:306) ~[?:?]
at org.jruby.RubyArray.each(org/jruby/RubyArray.java:1821) ~[jruby-complete-9.2.20.1.jar:?]
at usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_codec_minus_protobuf_minus_1_dot_3_dot_0_minus_java.lib.logstash.codecs.protobuf.pb3_to_hash(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-codec-protobuf-1.3.0-java/lib/logstash/codecs/protobuf.rb:297) ~[?:?]
at usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_codec_minus_protobuf_minus_1_dot_3_dot_0_minus_java.lib.logstash.codecs.protobuf.decode(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-codec-protobuf-1.3.0-java/lib/logstash/codecs/protobuf.rb:215) ~[?:?]
at usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_integration_minus_kafka_minus_10_dot_9_dot_0_minus_java.lib.logstash.inputs.kafka.handle_record(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-integration-kafka-10.9.0-java/lib/logstash/inputs/kafka.rb:356) ~[?:?]
at usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_integration_minus_kafka_minus_10_dot_9_dot_0_minus_java.lib.logstash.inputs.kafka.thread_runner(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-integration-kafka-10.9.0-java/lib/logstash/inputs/kafka.rb:329) ~[?:?]
at usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_integration_minus_kafka_minus_10_dot_9_dot_0_minus_java.lib.logstash.inputs.kafka.thread_runner(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-integration-kafka-10.9.0-java/lib/logstash/inputs/kafka.rb:329) ~[?:?]
bielko-igt commented 7 months ago

"method" is also a reserved keyword; the plugin tries to run any field with that name during parsing. That's quite a bummer, because Elastic's own recommended schema, ECS, has one such field - http.request.method

svempa434 commented 1 month ago

I managed to trigger this- seem like "test" is also a reserved word.