search

pires / kubernetes-elasticsearch-cluster

Elasticsearch cluster on top of Kubernetes made easy.
Apache License 2.0
1.51k stars 690 forks source link

enable discovery, other nodes to join the cluster #225

Closed bw2 closed 5 years ago

bw2 commented 5 years ago

Thank you for maintaining this code. We've been using it for over a year and haven't had issues.

I'm now trying to switch to a setup where 1) one or more persistent elasticsearch instances are manually deployed on google VMs 2) temporary instances are created for indexing new datasets using the kuberentes configs 3) when indexing is finished, the shards are transferred from the temp instances to the persistent ones using shard allocation settings.

I'm having trouble getting the manual and temp elasticsearch instances to join into a single cluster.

The manual instance is running on a VM called elasticsearch-persistent-node and I can curl it from inside the client pods (and vice-versa - I can reach the kubernetes es instance from the persistent node).

The nodes join up correctly if I add this to the client config:

- name: DISCOVERY_SERVICE
  value: "elasticsearch-persistent-node:9300"

but then when I try to write documents to the cluster - I get:

elasticsearch.exceptions.TransportError: TransportError(503, u'{\n  "name" : "es-client-75d64c7fc4-fx2gf",\n  "cluster_name" : "es-cluster",\n  "cluster_uuid" : "_na_",\n  "version" : {\n    "number" : "6.3.2",\n    "build_flavor" : "default",\n    "build_type" : "tar",\n    "build_hash" : "053779d",\n    "build_date" : "2018-07-20T05:20:23.451332Z",\n    "build_snapshot" : false,\n    "lucene_version" : "7.3.1",\n    "minimum_wire_compatibility_version" : "5.6.0",\n    "minimum_index_compatibility_version" : "5.0.0"\n  },\n  "tagline" : "You Know, for Search"\n}\n')
2018-09-12 09:47:20,371 INFO     ERROR: (gcloud.dataproc.jobs.submit.pyspark) Job [4e8c0b9a622a42c9b7f72108b1a48280] entered state [ERROR] while waiting for [DONE].

If I go back to the default:

- name: DISCOVERY_SERVICE
  value: "elasticsearch-discovery:9300"

then the temp cluster can index documents, but doesn't join up with the persistent nodes.

I also tried setting it to "elasticsearch-persistent-node:9300,elasticsearch-discovery:9300" but this doesn't help.

bw2 commented 5 years ago

switched to an entirely different approach, so closing this issue.