fluent / fluentd-docs

This repository is deprecated. Go to fluentd-docs-gitbook repository.
49 stars 119 forks source link

Add FAQ for elasticsearch plugin dependent client #537

Closed cosmo0920 closed 5 years ago

cosmo0920 commented 6 years ago

elasticsearch Ruby client sometimes causes API version mismatch.

elasticsearch-ruby#compatibility says that users should take care of Elasticsearch and its client library version mismatch but document has no warning or FAQ. Should we add FAQ or note to prevent version mismatch?

fujimotos commented 5 years ago

I generally agree that it's worth adding a note somewhere

I'm kinda curious that how td-agent users are handling this issue. SInce td-agent has all the dependencies packaged, users cannot help but accepting what td-agent provides, right?

What should they do if their elasticsearch does not have the correct version?

cosmo0920 commented 5 years ago

td-agent and Fluentd users sometimes fall a pitfall of Elasticsearch protocol version compatibility. https://github.com/uken/fluent-plugin-elasticsearch/issues/458#issuecomment-411967281

What should they do if their elasticsearch does not have the correct version?

Users should confirm their installed elasticsearch client gems and Elasticsearch have same major version.

e.g.) fluent-plugin-elasticsearch with elasticsearch 1.0.19 cannot send data into Elasticsearch 6.x. But, fluent-plugin-elasticsearch with elasticsearch 6.0.3 can send data into Elasticsearch 6.x.

fujimotos commented 5 years ago

What is the exact step to resolve the issue for td-agent users?

Again, because td-agent bundles all its dependency in its package, I do not believe there is any feasible way for them to solve the issue.

What are they supposed to do if they find their versions do not match? Downgrading their Elasticsearch?

cosmo0920 commented 5 years ago

What are they supposed to do if they find their versions do not match? Downgrading their Elasticsearch?

They should upgrade/Downgrade elasticsearch client gems by hand to match elasticsearch protocol. It is dangerous way to solve this issue. But there is no safe way to solve it.

repeatedly commented 5 years ago

Yeah, the problem is elasticsearch ruby client doesn't support multiple es version in one version. This strategy is not developer friendly...

fujimotos commented 5 years ago

Ok. I've added a brief "Troubleshooting" section to the manual page of the Elasticsearch plugin.

If you guys have feedbacks, please add comments there.

cosmo0920 commented 5 years ago

I'm trying to another approach which introduces debugging purpose configuration parameter. https://github.com/uken/fluent-plugin-elasticsearch/pull/463

fujimotos commented 5 years ago

Patch merged. I close this ticket too.