search

fluxcd / helm-controller

The GitOps Toolkit Helm reconciler, for declarative Helming
https://fluxcd.io
Apache License 2.0
408 stars 161 forks source link

How to select which pre-release I want to install #527

Open silveraignacio opened 2 years ago

silveraignacio commented 2 years ago

Hello! We have implemented Flux CD on our project, and now we are having the following issue:

We have all the helm charts on the same repo, but we are tagging it different depending on the environment. We have 3 environments, dev, uat and prod.

We decided that all the helm charts that will be installed on production, will be Release versions like 1.10.1, and charts for uat and dev, will be pre-release versions but differentiated by the postfix, for example for dev will be 1.10.1-dev and for uat will be 1.10.1-uat.

The issue that we are having now, is that we cannot filter by postfix of which version we will install on the environment. We are using semver in flux, so dev and uat are taking the same chart version due of this. In this case version 1.10.1-uat is the last pre-release available and it will be installed in dev and in UAT.

Below you can find one of our config files:

Devel

kind: HelmRelease
metadata:
  name: myhelmrelease
  namespace: default
spec:
  chart:
    spec:
      version: "1.x-devel"
  values:
    ingress:

UAT

kind: HelmRelease
metadata:
  name: myhelmrelease
  namespace: default
spec:
  chart:
    spec:
      version: "1.x-uat"
  values:
    ingress:

So, there is any way to say to Flux use charts with this specifc Postfix or add some kind of filter?

We have deployed Flux on our kubernetes cluster and kustomize-controller, source-controller, helm-controller and notification-controller were deployed.

Thanks in advance!

kingdonb commented 2 years ago

Related:

I think there might be an issue here for Helm Controller, since you can already filter tags in ImageUpdateAutomation, but there is no corresponding filter tags feature in Helm Controller there's no way to set a wildcard that filters prereleases.

Do we want to support this behavior in Helm Controller? I'll leave this issue open until there is consensus, or at least one other Flux maintainer can weigh in. 👍

defenestration commented 1 year ago

This bit me today, luckily didn't impact our prod environment. But i was planning on doing something similar with -beta/-dev/-stage versions of the chart i could test and release to different environments. I imagine this could surprise some people.

version: v0.*-dev

i notice in this helmchart CRD it shows how odd this looks with the different versions.

❯ k get helmchart -n flux-system zoom-zoom
NAME        CHART       VERSION    SOURCE KIND      SOURCE NAME   AGE   READY   STATUS
zoom-zoom   tsc-chart   v0.*-dev   HelmRepository   azure-oci     56d   True    pulled 'tsc-chart' chart with version '0.1.20-beta.1'
kingdonb commented 1 year ago

Thanks for raising this issue again when you found it @defenestration

I think we've established here or elsewhere that the right-hand side of the version in your HelmChart spec doesn't have any effect except to allow pre-releases. Eg. it can be -* or -0 and either will have the same effect, there is no compare on the right-hand side. I went looking for docs that say this, then checked the source code, and really did not get answers.

I actually cannot find the code for this behavior myself, what decides if pre-releases are allowed and if they are filtered:

I located pkg/version which I see defers almost everything to Masterminds semver (https://github.com/Masterminds/semver/blob/master/version.go) where there is a section that explains how pre-releases are compared to each other, in some detail. So we know this will be the sort order and dev comes after beta – (so if there are dev builds I'd expect you to get them instead of beta, but not because the beta are filtered by v0.*-dev instead because they are "newer" according to semver sorting. Unless they didn't match the semver filter pattern at all!)

What are your -dev tags shaped like?

I think there's a chance they just don't fit the semver rigid pattern, which says that right-hand side has to be either dotted alphas.1.num.2 but never alpha2 beta3 – the alphas and numbers have to be separated. Maybe that's what's wrong with your -dev tags?

Take a close look at Spec 11 for pre-releases, and be sure your dev charts match the pattern.

On the other hand I'm not saying there is no issue here that we need to fix. I think we should have regex filters too.

So, to reiterate, there isn't a wildcard regex in helm controller's version syntax. When you specify a version as whatever-* the wildcard part is insignificant. Whatever you put on the right-hand side just means that prereleases are OK, they will not be filter-matched based on what you put on the right-hand side of the hyphen.

We usually just put -0 so nobody gets confused.

And that's technically correct according to Semver, I guess, but super inconvenient for anyone that wanted to manage their phases of pre-releases in a single Helm repository like:

while imposing a >v0.*-rc in the last environment, to filter out the earlier phases of development or beta/alpha builds that are "too early phase for this advanced staging environment"

defenestration commented 1 year ago

Our dev tags follow a pattern like this version: v0.*-dev

example versions: 0.1.20-dev.1 0.1.20-dev.2

The idea is we release our dev prerelease chart to services in our dev cluster currently all at once and use 0.* for our stage/live cluster (eg; 0.1.20, 0.1.21).

I was hoping to introduce a beta tag 0.1.20-beta.1 to be able to target a single service using this beta identifier. But IIRC it overrode all the dev tags too.

Today i was revisiting this issue and noticed filter-tags on the imagePolicy resource, but didn't notice this on HelmReleases. If this was possible to use on HelmReleases, i think i could maybe use it to meet my original goal of using the different prerelease tags for different environments.

However i was instead considering just deploying the chart to a different name for beta use case which is probably less complex overall. Still wonder if filterTags for HelmRelease might be useful for others though.

jeroenlandheer commented 1 year ago

We use GitVersion in our workflow to version our software and Flux to deploy this to Kubernetes. GitVersion uses alpha, beta, etc as pre-release tags and we would love it to be able to use something like a regex or something similar to select which versions go where.

Filtering out specific branches in Git (which are associated with different semver tags) is a must for us. We work around this now by using automation in our release processes (and in a few cases even different container registries,) but this is far from ideal.

I think a regex could be an elegant fix and should provide enough flexibility for most cases.

There's just one thing, let's say you have a version 2.3.0 and 2.3.0-xyz then Flux should choose 2.3.0 as a higher version than the one with any pre-release tag as long as there's no newer version than 2.3.0 present. This might work differently for other developers, who might prefer to only use pre-release tags so I think the regex should not just cover the prerelease part of the semver but the entire version.

stefanprodan commented 7 months ago

This would be addressed for OCI charts when we implement:

joaocc commented 1 month ago

In the meanwhile, with HelmRelease (no OCI), we were able to use beta versions as follows. ">=4.0.0-0 <4.1.0-0" to use the latest 4.0.0 pre-release or release