Improve support for infrastructure-as-Code/Kubernetes-based deployments

[remmeier]

we make use of ELK stack for monitoring and the visualization of business data (as everbody else I guess). We deploy to a larger number of environments/instances. For that Kubernetes and infrastructure-as-code principles are in use to align the setup as much as possible. Nowadays this approach is quite common place. But unfortunately it is currently rather hard to get ELK well running in such a setup. Maybe this ticket can serve as a starting point to help improve in the area because overall the ELK stack is fairly great.

• An official Helm package would be nice bringing everything together. There is one in https://github.com/helm/charts, but as of yet rather basic and the various components not well aligned. To consider here is that not always people have cluster privileges for deployment, so custom resource definitions and deamon sets may not always applicable. Things like an ILM setup should come out-of-the-box.
• An empermal Kibana pod spends a lot of time optimizing itself with every startup. Leading to startup times in the minutes range
• Many operational tasks are assumed be handled through the REST API once deployment completed. But this in stark contrast of how things are done in Kubernetes. One would expect to be able to place Kibana dashboards, index templates, index patterns, ilm policies, etc. in some directory and the ELK components will pick them up and apply automatically. Instead one has to write a dedicated component to wait for everything to start and make use of the rest services.
• Components should reregister themselves with Kibana/Elastic if they got reset/cleared (add dashboards again, etc.). Usually pods are independent and if one is not touched it will continue running even tough the others might got cleared/reset.

[tylersmalley]

@remmeier we do actually have Helm Charts for the products in our stack. Though, there are located at elastic/helm-charts, and not in the helm/charts repo. You will probably have more luck directing k8s specific questions there.

[elasticmachine]

Pinging @elastic/kibana-operations

[remmeier]

ah the Helm chart is good. Potentially we could deprecate the ones from the official Helm repository (https://github.com/helm/charts/tree/master/stable/elasticsearch and related). Or move the Elastic ones. Maybe duplicate efforts can be avoided.

All the other things in the ticket still hold I think. In particular having an alternative to the REST API would be nice to see.