Live migration from helm charts to eck

[wtibbitts]

Proposal

Add documentation on how to live migrate from a cluster created with the elastic helm charts to a cluster created with the eck operator. Use case. Why is this important? If you are deprecating the elastic helm charts and want user to move to the eck operator, you need to explain how they can do that, especially in cases where they can't stop writing and do a snapshot/restore.

Questions in the forums do not get usable answers, nor in the slack channel. It is also unclear how to make something like https://github.com/elastic/helm-charts/blob/main/elasticsearch/examples/migration/README.md work with the eck operator.

Bug Report

What did you do?

What did you expect to see?

What did you see instead? Under which circumstances?

Environment

• ECK version:

  insert version or git commit id here

• Kubernetes information:

  insert any information about your Kubernetes environment that could help us:

  • On premise ?
  • Cloud: GKE / EKS / AKS ?
  • Kubernetes distribution: Openshift / Rancher / PKS ?

  for each of them please give us the version you are using

  $ kubectl version

• Resource definition:

  if relevant insert the resource definition

• Logs:

  insert operator logs or any relevant message to the issue here

[pebrc]

Live migrating from a Helm chart managed cluster is currently not supported. This is due to the way how ECK exclusively manages the cluster/master node quorum/transport x509 certificates.

Your options today are one of the following:

• you can use snapshots to restore the data from your existing Helm chart based cluster into an ECK managed Elasticsearch cluster
• you can use cross cluster replication to move data into a new ECK managed Elasticsearch cluster
• you can use reindex from remote to move data to a new ECK managed Elasticsearch cluster

You can e.g. cut over your ingest workloads to the new cluster and move historic data to the new cluster. Or do a combination of moving historic data, cut over and move the difference that has accrued between the initial move and the cut over.

I do know that all the options I listed are far from perfect and will keep your issue open as a feature request.

[wtibbitts]

@pebrc How do I get ECK to add additional CA certs for remote clusters not managed by ECK. I don't see the right way to mount a secret that has the cert.

[rootkid9233]

@pebrc @wtibbitts Has this CA issue been documented somewhere? You can use the kubectl cp command to just add certs to your nodes but the issue I have is thet Xpack.security.transport.ssl.certificate_authorities: defaults to the file paths with the self signed certs it created