search

grafana / k6-operator

An operator for running distributed k6 tests.
Apache License 2.0
543 stars 148 forks source link

Suggestions for chart versioning and release management #344

Open illrill opened 7 months ago

illrill commented 7 months ago

Feature Description

Release notes

Would like some type of chart release notes. Bare minimum populate What's Changed in the Github release, in the same way as the grafana and loki-distributed charts do it by just referencing PRs.

In addition to referencing PRs under What's Changed, there could be a custom release note annotation in Chart.yaml. When a contributor creates a new version, they also overwrite the "old" release notes with the release notes applicable for this version. This could get picked up by the helm chart releaser workflow and included in the Github release. Something like this:

version: 3.2.1
[...]
annotations:
  changes: |
    - kind: added
      description: Description of something I added in this version
    - kind: changed
      description: Description of something I modified in this version

Versioning

Maybe it's on its way in #330, but some guidance and enforcement of chart versioning would be good. Again, perhaps we could use argo-helm's guidelines as inspiration.

yorugac commented 7 months ago

Thanks for sharing your observations, @illrill

It seems like there are several points made here at once:

  1. "how to bump versions" for contributors to Helm chart: that's definitely is supposed to be covered by the issue #330. Ideally, I'd like a workflow that doesn't require contributors to bump versions in the first place but it's a more broad problem than just Helm chart.
  2. "adding release notes for Helm charts": I've thought about this one but didn't create an issue. I suggest we consider this issue #344 as keeping track of this problem
  3. "use artifcathub.io for release notes": k6-operator does not have release notes on artifacthub.io now, no, but I assume it can be seen as a consequence of the point 2.

So more on "release notes for Helm charts" problem.

In general, I'm definitely inclined towards re-using the same approach that other Grafana charts utilize, whenever possible. But, not all Grafana charts have the same approach in the first place. E.g. both grafana and loki-distributed are within helm-charts domain, it seems. In comparison, Mimir chart does not follow this approach and it doesn't have release notes on artifacthub.io either. Currently I have an impression that k6-operator chart is more of a Mimir case than Grafana case... I admit that I haven't had time to dig into all details around those though.

That said, if anyone sees a simple and straight-forward way to add release notes to Helm releases in k6-operator (i.e. similar to what loki-distributed has), please feel free to make a PR :slightly_smiling_face: If not, we'll improve on this, but no promises on when exactly.

On a kind of related note, I would hardly call current Helm chart release "stable" (just today's incident), still :disappointed:

illrill commented 7 months ago

Ah yes, I see now the grafana and loki-distributed charts are hosted in the helm-charts repo itself. But could we anyway give it a try setting generate-release-notes: true in this repo's cr.yaml? Looks like that's how they're doing it (e.g. https://github.com/grafana/helm-charts/pull/2469).

I didn't necessarily mean that we should publish the chart or its changes to artifacthub.io, it was just a real-life example of how the Argo project has chosen to handle chart release notes and versioning in general, using the approach I suggested with a annotations.changes field in the chart to get the "input" for the release note. (The "target" being artifacthub.io wasn't the relevant part).

yorugac commented 7 months ago

But could we anyway give it a try setting generate-release-notes: true in this repo's cr.yaml?

Actually, yes, definitely! Thank you for the pointer :slightly_smiling_face:

yorugac commented 3 months ago

For record's sake. generate-release-notes: true was set to true for the latest Helm release (today) but it didn't seem to generate anything :thinking: https://github.com/grafana/helm-charts/releases/tag/helm-k6-operator-3.6.0 https://github.com/grafana/k6-operator/releases/tag/helm-k6-operator-3.6.0

illrill commented 3 months ago

Have you considered to version/release the application and the chart together, instead of them having separate life cycles and release notes? The application release notes in this repository are superb!

yorugac commented 3 months ago

Have you considered to version/release the application and the chart together, instead of them having separate life cycles and release notes?

Right, I see two main problems with it ATM: updates for application don't always map 1-1 to Helm updates. Right now it's possible to release a fix to Helm without touching application release and vice versa. So there's a flexibility in the current setup which we might want to preserve for the future. Secondly, I just haven't had the time to dig into how to push manually written notes onto helm-charts release process :sweat_smile: Is it even possible? and so on. Clearly, the simple cr.yaml didn't work (unless I'm missing something obvious).

None of this is set in stone yet. I'll be pondering this more and likely discussing it internally as well; but it'd be good to improve release process for sure.