Create an Helm chart for this

staticdev commented 5 years ago

Proposal

Create a Helm chart for this operator just as there are for Elasticsearch and Kibana (https://github.com/elastic/helm-charts/tree/master/elasticsearch).

There are other operators that have Helm charts like https://github.com/storageos/charts/tree/master/stable/storageos-operator

This is important to make the usage/management easier.

LaurentGoderre commented 5 years ago

@staticdev I'm going to take a crack at creating it. We need it in my project so might as well do it for everyone!

LaurentGoderre commented 5 years ago

Here is the WIP pull request: https://github.com/elastic/helm-charts/pull/266

krichter722 commented 4 years ago

Reading the reason for closing #1906 this is not going to be supported. Maybe a member of the dev team can take a look, escalate this and hopefully change the minds of the project owners.

I think having a helm chart is the only way to make this project usable as copy-and-pasting bits and pieces from a quickstart guide (the current modus operandi for installing afaik) is unacceptable in the Kubernetes world - and an insane waste of resources give the great standardization of these tools (Kubernetes and helm).

LaurentGoderre commented 4 years ago

@krichter722 you are more than welcome to use the chart I created in the meantime.

olemarkus commented 4 years ago

A helm chart is pretty much a must for Sportradar to adopt the operator.

a-hat commented 4 years ago

A helm chart would really simplify the integration of the ECK operator into existing infrastructures which are already utilising helm. In our setup, the ECK operator is the only component we need to install via a shell script.

Additionally it would provide a standardized way of customizing the installation, there is already a requirement for this in https://github.com/elastic/cloud-on-k8s/issues/2406

@LaurentGoderre I had a look at your PR. It seems to me, that this helm chart packages an instance of a Elasticsearch, not of the operator itself?

LaurentGoderre commented 4 years ago

@a-hat it is a chart for the operator, it creates the k8s resources for the operator. I used it internally to deploy instances.

LaurentGoderre commented 4 years ago

Actually @a-hat I had created two charts the one referred above is out of date. The more up to date one is this one here: #1906

anyasabo commented 4 years ago

We are considering a proof of concept of an operator helm chart. One concern we have is the lack of support in Helm for changing resource apiVersions, which we have done multiple times already and will likely continue to do, See one example issue here (there are a few): https://github.com/helm/helm/issues/6850 And multiple open PRs attempting to resolve it. That said, it should still be possible to have a chart for the operator, even if having one for the custom resources themselves is problematic.

kevinnoel-be commented 4 years ago

We needed a Helm chart to deploy the operator itself, so we created it and made it available here: https://github.com/collibra/elastic-operator

LaurentGoderre commented 4 years ago

This is why I had #1906 to prevent having multiple variation of charts that do the same thing

bradenwright commented 4 years ago

Has this been reconsidered from Elastic recently? I think its pretty obvious its something numerous users want to the point there are now 2 implementations. Consider what you are maintaining its a really small chart and at least IMHO having multiple charts that individuals are maintaining seems worse than what I could make of issues on helm/helm#6850 seems like with 6850 helm would throw an error with apiVersion changing and then the user could either fix it (a) via kubectl apply or (b) via deleting and using helm to recreate it.

@anyasabo maybe you could elaborate a little about Elastic's concern otherwise, there are a number of operators with charts to install so if there is an issue it seems like it would be a common one, that hopefully if community feels is a big issue would ultimately get fixed in Helm. Anyway I'm interested b/c right now we are deploying everything via Helm and plan to use a chart linked here or create our own.

PS: So far the Elastic Operator has been great! Really appreciate the work that's gone into it, sure makes deploying ES a lot easier. Thanks for all the work put in, its much appreciated

anyasabo commented 4 years ago

We recently added a manifest generator tool to reach a similar end state: https://github.com/elastic/cloud-on-k8s/pull/3124

Though there is ongoing work to make it more user-facing

staticdev commented 4 years ago

@anyasabo I am a huge fan of Elastic and @bradenwright has a good point. Why create your own generator if you can use an industry-standard, stable, GA, CNCF graduated project of generator? With all the respect, but you are having much more effort than you should to get a less practical solution. If you were building the Helm chart, you wouldn't need at all to make it more user-facing, since many companies already know how to use it and it is very well documented.

anyasabo commented 4 years ago

@staticdev please see the related issue for that PR which includes more of the reasoning: https://github.com/elastic/cloud-on-k8s/issues/2406

Also note that the generator avoids some of the previously discussed Helm issues for our use case, but uses Helm templating under the hood which may make opening that up an option in the future

johnpemberton commented 4 years ago

Would love to see a supported stable helm chart for this operator. Short of manually editing the all-in-one.yaml, messing around with the manifest generator, or building a helm chart myself - all of which incur maintenance overhead, I'm really finding it difficult to support the decision to use this operator on our platform, as good as the operator is. Considering using opendistro as an alternative, given the reluctance of this project to play nicely with the industry standard tooling that we're all so used to using nowadays.

js-timbirkett commented 4 years ago

It is rather discouraging to see various Kubernetes projects resorting to Helm driven cli tools for deployments rather than sticking with plain old Helm charts that users can consume. Istio has done the same... Whilst, for a few clusters it is okay, if you're managing 10's or 100's of clusters you really don't want to be *ctl'ing everything.

pebrc commented 4 years ago

Why have we not released a Helm chart for ECK?

Because it simply did not work. This is due to the fact that Helm could not handle apiVersion changes until very recently. We want to provide our users with a seamless upgrade path between versions. Completely uninstalling the previous version of ECK before being able to upgrade just to work around the Helm bug did not seem a good trade-off.

Will we ever release a Helm chart for ECK?

Maybe. We are reevaluating the situation now that Helm 3.2.0 has been released which contains a fix for one issue and are checking if a ECK Helm chart is viable now.

What can I do in the meantime?

Our manifest-generator is using Helm as a library internally to generate the manifests but not to install them which sidesteps the issue. You can build your own chart based on that and customize it to your needs, as long as you are aware that it is internal tooling subject to change without notice. The manifest-generator can already cater to different use cases e.g. ECK deployments restricted to a single namespace or just a set of namespaces and so forth.

Romiko commented 4 years ago

Hi, Please can you let us know when it is released, This solution is way better than API calls post install.

kpavan95 commented 4 years ago

Hey @pebrc is there any update on the reevaluation? I would like to give my +1 towards this feature. It would be a huge maintenance offload as I am already managing multiple other operator helm charts with a helmfile but for this alone on every release migration I have to reupdate my custom chart for the operator.

barrelful commented 4 years ago

@pebrc I can't see any limitations on helm that justify what you wrote. Also, we are now on Helm v3.3.1, and that fix you mentioned was shipped. Not having a chart for this is increasing the barriers to having more ECK clients!

ryudice commented 3 years ago

Bump. We deploy everything using helm and we would like to stick to that standard. I hope you change your mind.

mattjw commented 3 years ago

Here's a friendly "yes please" vote from me too. We use the HelmRelease CRD (via Flux) as much as possible for our deployment of third party apps. An official chart for ECK would be a huge help.

sadoMasupilami commented 3 years ago

We would also really appreciate that as we are using helm to install all our software. We do now need to build our own chart with the all in one yamls.

david-kow commented 3 years ago

With todays release of ECK 1.3.0 we've introduced an official Helm chart for ECK.

You can start with:

helm repo add elastic https://helm.elastic.co
helm repo update
helm install elastic-operator elastic/eck-operator -n elastic-system --create-namespace

For more details, see our docs about installing ECK with Helm and migrating from your current deployment method to Helm.

Given the above, I'm closing this issue. For any feedback or in case of problems, please open a new issue. For questions, please use our forum.

elastic / cloud-on-k8s