search

pires / kubernetes-elasticsearch-cluster

Elasticsearch cluster on top of Kubernetes made easy.
Apache License 2.0
1.51k stars 687 forks source link

using Java API to access the elasticsearch running in the kubernetes #142

Open shengbiaohong opened 7 years ago

shengbiaohong commented 7 years ago

Hello,everyone.I am now following this project https://github.com/pires/kubernetes-elasticsearch-cluster and successfully running the elasticsearch in kubernetes. And I also get the elasticsearch health status by the command "curl http://Endpoint:9200".It seems that the elasticsearch now running in the kubernetes and I can use the restful api to access the elasticsearch.Now I want to use Java API to access elasticsearch and insert some data to the exit index.How should I do?Do I use the 9300 port? In this project,there are two service named elasticsearch and elasticsearch-discovery.In the file named es-discovery-svc.yaml, I found the 9300 port is exposed.I edit the es-discovery-svc.yaml and let the 9300 port mapped the 30003 port in the host.When I use the host IP and the 30003 port to connect the elasticsearch and add some data,it failed.The error show "NodeDisconnectedException:22.106.103.30:30003 cluster:monitor/nodes/liveness disconnected".How should I do?Thanks a lot if you know how to solve this problem.

rocketraman commented 6 years ago

@shengbiaohong Port 9300 is the "transport" API and is used by the cluster internally. The Java API used to also use the transport API as a client, but Elastic is discouraging this in favor of their new clients that use the REST API via HTTP on port 9200 (see https://www.elastic.co/guide/en/elasticsearch/client/java-api/current/java-api.html and https://www.elastic.co/guide/en/elasticsearch/client/java-rest/6.3/java-rest-high.html).

The discovery service is used by cluster members internally to discover each other. As a headless service (clusterIP: None), it is basically just a DNS entry which is used by the discovery system to find other members.

Using the transport API is problematic with Kubernetes because I believe the way the transport API works is that the client needs to be able to connect to all members of the cluster, not just one of them. I don't believe there is any good way to expose all the nodes of the cluster to an external client in this way. I believe this is why you are getting a NodeDisconnectedException.

Therefore, you'll want to use the elastic search service on port 9200 defined in es-svc.yaml to connect to your cluster, along with the Java REST client on the client side.

HTH!