logstash-plugins / logstash-codec-protobuf

Codec plugin for parsing Protobuf messages
Apache License 2.0
26 stars 16 forks source link

Add support for protobuf 3 #18

Closed IngaFeick closed 6 years ago

IngaFeick commented 6 years ago

This version will allow using protobuf 3 definitions. Under the hood it will use the official google lib for that, which not only brings new functionality but has shown massive speed improvements in my tests aswell (think ~factor 2.5 faster). The new lib can be activated by setting protobuf_version_3 => true. The code is fully backwards compatible for users of protobuf 2, no changes required on their side. Thanks to @jorgelbg for his contribution! We have tested this against logstash 6.2.2 and 5.6.7.

IngaFeick commented 6 years ago

Travis indicates a problem with the location of the rspec helper classes. I am not sure how to tackle that. Works fine for the 5.6 branch but not for 6.x. If anyone can shed some light on what has changed I would be happy to adapt the specs.

IngaFeick commented 6 years ago

@jsvd I think I implemented all the requested changes, except for the one regarding the dependency version because that didn't work for us. If you have an idea on how to make it work I'd be very happy to change it aswell. Thank you very much for your time!

xoliver commented 6 years ago

Hi, sorry to butt in, I wanted to thank you for doing this work, it is super useful! ~I'm curious as to why the plugin is effectively doing the deserialisation itself instead of resorting to the ruby protobuf implementation?~ EDIT: I can't read code well enough it seems

jsvd commented 6 years ago

@IngaFeick I just wanted to say I haven't forgotten about your work. I'm afk today but I'll try to review it this week.

IngaFeick commented 6 years ago

@jsvd no worries and no hurries! I appreciate any time you can find. Thank you lots! :)

IngaFeick commented 6 years ago

@jsvd squashing into one sounds great. Thank you very much 😍

jsvd commented 6 years ago

Published https://rubygems.org/gems/logstash-codec-protobuf/versions/1.1.0

IngaFeick commented 6 years ago

Thank you @jsvd 👍