search

benfiola / minio-operator-ext

A Kubernetes operator that allows for declarative management of MinIO resources
13 stars 0 forks source link

Helm chart for deployment #19

Closed nogweii closed 3 weeks ago

nogweii commented 1 month ago

I might take a stab at this soon, but wanted to ask if you had ideas on this already.

benfiola commented 1 month ago

I haven't - but think it'd be cool. While I've played with (and have been a client of) helm, I've never written a chart intended to be used by others. If you'd like to give it a shot, I'd be more than happy to review a PR. Otherwise, I can give it a try myself (and learn more about vendoring good helm charts in the process) - either works!

steinwelberg commented 1 month ago

Is it an idea to separate the crds from the operator itself? This pattern is used in multiple other operators as well, given that Helm is not able to upgrade or delete crds if you put them in the crds folder. See also: https://helm.sh/docs/chart_best_practices/custom_resource_definitions/#some-caveats-and-explanations

Given that there is no Helm chart yet, it's currently also not a breaking change to use this pattern. The likelihood that changes to the existing crds will be made is quite big given the operator is this young :-).

I'm more than happy to contribute btw.

benfiola commented 1 month ago

Is it an idea to separate the crds from the operator itself? This pattern is used in multiple other operators as well ...

Unfortunately, I purely rely upon helm template - so I inadvertently side-step the limitations of Helm CRD management. Despite this, I've seen Helm chart authors approach this problem from three different angles:

I don't personally have a strong opinion about which approach we take (- though I'd prefer CRD management to be user initiated vs. controlled by the operator). Is there an approach that either of you would prefer?

steinwelberg commented 1 month ago

With separate the CRDs from the operator I mean that we create a separate Helm chart for the CRDs. See for example: https://artifacthub.io/packages/helm/snowplow-devops/cert-manager-crds.

IMO this is the cleanest solution.

That would be a fourth option in your list 😉.

benfiola commented 1 month ago

Whoops - looks like merging the related branch accidentally closed this issue. Anyhow, with a little bit of work (and @nogweii's charts), I was able to set up a helm repository.

The repo is located at https://benfiola.github.io/minio-operator-ext/charts - and both development and release charts should be published as of version 2.2.0.

Give it a shot and let me know if you run into any issues!

nogweii commented 1 month ago

@benfiola I'm trying to deploy the helm chart and I'm getting an error from Kubernetes:

Error: failed to create containerd task: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: exec: "/operator": stat /operator: no such file or directory: unknown

And inspecting the 2.2.0 tag (using dive and docker image inspect) the image only shows 1 layer - the base distroless layer. No operator binary to be found. Was there a hiccup in the docker release process?

EDIT: And switching to the 2.2.0-rc.1 tag makes the deployment work. Seems like something with #21 didn't go as planned.

benfiola commented 1 month ago

:grimacing: - yep, I goofed when I modified the docker image publishing workflow.

Turns out docker buildx build ... --output type=tar,... does not do what I thought it did. Loading the resulting tar and publishing it does indeed produce a bad image. Will work on fixing this ASAP - thanks for the heads up!

benfiola commented 1 month ago

Alright, the new versions are building, but things should be fixed as of version 2.2.1. Let me know if you run into any issues with the updated helm charts!

steinwelberg commented 3 weeks ago

I haven't gotten around to migrating to the helm charts from the repo yet. Once I have done so, I will give a heads up if all is fine :-).

steinwelberg commented 3 weeks ago

The charts work like a charm! They were a drop-in replacement for the ones we had temporarily constructed.

benfiola commented 3 weeks ago

yeah! i migrated my infrastructure-as-code project to use these charts and it was very convenient.

thanks again for the contribution @nogweii - and thanks for verifying that things look good @steinwelberg! 🥳

am going to close this issue out, feel free to open new issues for any problems you run into.