protobuf release is breaking fluent-plugin-kinesis

[isukapalli]

Google has released 3.8.0 version and that's breaking fluent-plugin-kinesis installation from td-agent. The latest release is dependent on Ruby 2.3 version.

---- Begin output of /usr/sbin/td-agent-gem install fluent-plugin-kinesis -q --no-rdoc --no-ri -v "2.0.0" --source=https://www.rubygems.org ----
       STDOUT:
       STDERR: ERROR:  Error installing fluent-plugin-kinesis:
           google-protobuf requires Ruby version >= 2.3.
       ---- End output of /usr/sbin/td-agent-gem install fluent-plugin-kinesis -q --no-rdoc --no-ri -v "2.0.0" --source=https://www.rubygems.org ----
       Ran /usr/sbin/td-agent-gem install fluent-plugin-kinesis -q --no-rdoc --no-ri -v "2.0.0" --source=https://www.rubygems.org returned 1

We wouldn't expect google will release breaking changes in their minor release but it's good have version pin on the gem dependencies.

https://github.com/awslabs/aws-fluent-plugin-kinesis/blob/master/fluent-plugin-kinesis.gemspec#L50

[simukappu]

Thank you for your feedback. However, the earliest v3 supported td-agent v3.0.0 seems to include Ruby 2.4.0. https://support.treasuredata.com/hc/en-us/articles/360001479187-The-td-agent-ChangeLog https://docs.fluentd.org/quickstart/td-agent-v2-vs-v3

Which version of td-agent are you using? Which versions' installation is broken?

[bozerkins]

Hello. We've faced the same problem. We're running td-agent 2.3.6 and were trying to install kinesis plugin version 2.1.1, which by all the accounts and documentation should support td-agent 2.3.6 (which comes with fluentd 0.12.35 version support). The problem is in this specific line https://github.com/awslabs/aws-fluent-plugin-kinesis/blob/v2.1.1/fluent-plugin-kinesis.gemspec#L43 The dependency on the protobuf library is too broad. When we were installing this plugin on one of the machines, the latest protobuf library version (of 3 major release) was 3.6.1, which worked fine with ruby 2.1. Now the latest is 3.8.0, which requires ruby 2.3, which only comes in 2.4+ td-agent.

The fix is easy, just restrict the dependency for protobuf library on the appropriate package versions of kinesis plugin

[bozerkins]

Also, the problems start with google-protobuf gem v3.7.0, where they dropped support for ruby < 2.3 Here are the release notes https://github.com/protocolbuffers/protobuf/releases/tag/v3.7.0

[ernescz]

Hello. We've faced the same problem. We're running td-agent 2.3.6 and were trying to install kinesis plugin version 2.1.1, which by all the accounts and documentation should support td-agent 2.3.6 (which comes with fluentd 0.12.35 version support). The problem is in this specific line https://github.com/awslabs/aws-fluent-plugin-kinesis/blob/v2.1.1/fluent-plugin-kinesis.gemspec#L43 ...

In addition to @bozerkins comment platform specs: Linux Ubuntu 16.04 (Xenial), td-agent version: 2.3.6-0 (taken from http://packages.treasuredata.com/2/ubuntu/xenial/)

[simukappu]

Thank you for your feedback! I understand that this is a plugin v2 dependency issue, not v3. I have updated this dependency in v2 branch to "google-protobuf", "~> 3", "< 3.7" and will release v2.2.0 as v2 new version.

[riywo]

We've released v2.2.0 to fix this dependency issue. https://rubygems.org/gems/fluent-plugin-kinesis/versions/2.2.0

Please add more comment if it doesn't work for you.