Refactor: Move From Elasticsearch Ruby client to the the Java Rest client

[ph]

Refactor the usage of the elasticsearch client into a common way.

Currently the module feature uses the Elasticsearch Ruby client and the X-Pack config management uses the elasticsearch output as a factory to create a concrete es client.

With the release of the ES Java Rest client we now have a pure java and solid implementation of the API. We should move our custom code to use the official client.

Also Logstash core should provide a small infrastructure to retrieve a correctly configured client with the the right TLS options.

The refactor has the following goals:

• Reduce duplication of logic
• Improve user feedback
• Optimize the threadpools.

[ph]

I think we should do the following:

• wrap the low level client into a higher level builder with our common options
• Create a factory that can directly takes a subset of the settings object and use the builder to return a wrapped client.
• Add client header with a LS signature + version.

[ph]

personnal note: Look to correctly handle PKI/keystore, openssl vs java store.

[ph]

• [x] Import the Low level java client
• [x] Create a builder with our normal options
• [x] Create a simple high level client for our need xpack and LS related.
• [ ] Move to a more Java OOP to query es.
• [ ] Move the Module Ruby client to the java classes.
• [ ] Create the ssl context correctly, I have the code written I just didn't test all the possible scenario/configuration.

[ph]

The builder interface is similar to what the low level rest client offer but will support the types used inside logstash and will take care on instantiating the right object for us.

So we don't have to deal with Sniffer class or SSLContect directly and it also provide a bit more ruby friendly types for the library.

# The name of the methods are still subject to change.
ElasticsearchClient.builder()
.hosts("https://127.0.0.1"...)
.username("elastic")
.password("secret")
.certificate("/tmp/mycert.ca") # might turn on SSL auto, I still prefer the explicit call.
.certificateKey("....")
.ssl(true)
.sniffer(true)
.build()

The #build method return a wrapped client that offer some higher level method for convenience, but you can retrieve the original restclient by calling getRestclient()

The sniffer is correctly setup for the appropriate protocol that we use.

We could also leverage the central point to minimize the number of sniffer or client threads and reuse the same connection when the options are the same.

[mjtrangoni]

@ph, Is there any ETA for this? I recently upgraded from ES5.6 to ES6.3, and I realized that this problem is still open. I would really like to use TLS without having to deploy intermediate certificates everywhere!!