Deprecating Node Protocol

[andrewvc]

The node protocol is a giant pain for both Logstash users and Logstash developers.

Why node is bad for users: Users think they can use it to gain a speed boost (which it doesn't actually provide). After trying node they then wind up posting bug reports due to broken configurations (setting up a valid node is more complex than HTTP or transport). This makes for disappointed users and disappointed developers.

Why node is bad for developers:

• We waste time debugging node configuration errors
• We currently have to maintain an extra logstash-output-elasticsearch-license plugin JUST for the use case of shield / marvel + logstash. Additionally, the error messages generated in this scenario are plain ridiculous to debug.

Why user like node

• They can see which Logstashes are active via the Elasticsearch node list (the next major of Logstash will cover this with Metrics however)
• perceived speed
• multicast discoverability (though we now discourage this in Elasticsearch)

IMHO we should deprecate node soon, and consider removing it in the next major release. Perhaps a community member would be interested in maintaining that as a separate plugin.

[andrewvc]

Moved to https://github.com/elastic/logstash/issues/4433

[markwalkom]

I think this is a good idea.

But we need to be aware that some users choose node so that they have visibility into LS metrics via Marvel, so we'd need to make sure we have metrics APIs to replace this.

[andrewvc]

Hey @markwalkom , I accidentally linked back to this ticket instead of to https://github.com/elastic/logstash/issues/4433 , could you leave your comment there?