search

elastic / helm-charts

You know, for Kubernetes
Apache License 2.0
1.88k stars 1.93k forks source link

Not sure of best approach #426

Closed pbecotte closed 4 years ago

pbecotte commented 4 years ago

So I put in some PRs last month related to getting these helm charts properly installed on a Digital Ocean cluster. Initially, I just basically used host_networking as described in the official docs, but there was rightly some pushback there- its not an ideal solution after all!

So I spent the last week actually figuring out the root causes and solutions. In the end, I think I have everything working as intended on a full ES stack on DO, along with an understanding of why it works. I would love to contribute that knowledge back so that future peeps have an easier time. I am not sure the best way though. The reason is that most of the solution happens with the config.yaml files though.

As an example, the first issue I faced is that on Digital Ocean, the node names aren't routable, so you can't use them to talk to the kubelet. To fix it, I passed in node.ipaddress as an environment variable, and used that instead of HOST_NAME for the kubelet config. Lots of other stuff like that.

But the way these charts are set up, the config is a big unstructured blob of yaml. It works great. But I am not sure the best way to do this. So, looking for feedback- which of these would be the best place to collect information about how to get this stuff working?

  1. Add extensive comments to values.yaml
  2. Add a big section to the README
  3. Add to the source metricbeat docs and put a link from the helm README or values.yaml pointing back to them.
jmlrt commented 4 years ago

Hi, @pbecotte, thank your for opening this ticket.

Our recommended way to add cloud providers specific configuration is to add an an example with specific values.yaml and eventually goss tests in the examples folder of the charts. For instance, we already have Openshift specific examples for Elasticsearch and Kibana charts. Adding a DOKS example for the charts requiring specific configuration would be a great addition.

pbecotte commented 4 years ago

That was my first thought process, but once I started to actually learn about how this stuff worked, I became less sure that is the best solution. The reason is that various cloud providers can have different combinations of settings, and some of them can be overridden, and they can change over time. For example, the "readonly" port that the default uses to scrape the kubelet stats is marked as deprecated in k8s, and is disabled by default. Further, figuring out why it still didn't work after I changed the port (DO hostnames not being routable) took a lot of learning. Having something written down about how this stuff works and how the various settings interact with Kubernetes feels like it would be more useful than a couple pre-packaged solutions, unless we can be sure that those pre-packaged solutions always worked out of the box.

As a further point of reference, check out https://github.com/awslabs/amazon-eks-ami/issues/128 and https://github.com/kubernetes/kubeadm/issues/732 - I am not using EKS, and I believe the automated tests are running on GKE... but this makes me wonder if the default config would work on Amazon either.

jmlrt commented 4 years ago

Hi @pbecotte,

We just merged #471 which is now using kubelet secure port instead of read-only port as default config as a lot of k8s providers are now using secure port.

We will also work on how to address the other issues we have with Metricbeat chart (hostNetwork, ...) and better document it soon.

All your feedbacks in this and others tickets will help us a lot.

Many thanks for all your contributions and for your patience 👍

botelastic[bot] commented 4 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.