Open hairmare opened 3 years ago
hairmare
commented
2 years ago all my unpublished proof of concepting for this has been following the "write some code to do it" approach and i still have nothing to show that resembles a solution that is maintainable.
this post highlighted the idea of using an existing tool rather than having to write our own. The linked updatecli look like something we'd want to investigate closely.
hairmare
commented
2 years ago i spent some time with updatecli and it looks like it will be able to do alot of the work we'd like it to
the main caveat is that it's still missing some features/has some quirks that we'd probably want to wait for:
yaml.Marshal to write changes anything touched by updatecli would get canonicalized (imo we can live with this, the tradeoff wrt to reordering the contents of Charts.yaml is ok and this doesn't affect values.yaml)i also haven't figured out how to configure how updatecli names branches, how to template commit messages, and how i can analyse an updated targetRevision to help deciding if the charts versions needs a major, minor, or patch bump. I'm still reading updatecli's actual code to figure some of these things out and might end up writing some PRs if the stuff we're missing isn't too large.
One thing that is sure is that we would be writing a lot of yaml to cover all the -apps charts, an example that partially updates argo-cd in infra-apps is below:
olblak
commented
1 year ago Hi here,
I accidentally found your discussion. I hope you don't mind me jumping here.
I am the creator of Updatecli
First of all thank you for considering Updatecli.
Your use case is interesting. @dduportal has been pushing in the same direction than you I just want to add that if you use the GitHub flow, then you could have two commits but one pullrequest. That's what we do all the time, you need the following snippet in your manifest.
pullrequests:
helm-charts:
kind: github
scmid: helm-charts
spec:
automerge: false
labels:
- dependenciesAnother thing that appears to be confusing here is the difference between the scm of kind git and github.
the git scm would commit directly to the branch specified in your manifest while the "github" one create a temporary branch and is designed to be used with a pullrequest, so it open a pullrqeuest from that temporary branch to the one specified in the manifest.
So the github scm implements the GitHub flow, I got many feedback that it's not an intuitive behavior. We had lengthy discussion about how this behavior should evolved, and I also have an open pullrequest but we never finished that discussion...
Meanwhile I have been focusing on another area.
To reduce the need to write and maintain manifest, I worked on a feature named "autodiscovery" where Updatecli would detected common behavior. It's still experimental as I am still playing with it to identify the best way to use it. And I would love some feedback
You can have taste with the latest version of Updatecli. The following manifest should be enough to start with.
By the way manifests under the directory named "updatecli.d" are recognized by IDE(s) that support jsonschema store (like vscode), so you get validation, and autocompletion for free.
updatecli.d/autodiscovery.yaml
name: Update app of apps charts
scms:
helm-charts:
kind: "github"
spec:
branch: "main"
user: "adfinisbot"
email: "support@adfinis.com"
owner: "hairmare"
repository: "adfinis-helm-charts"
token: '{{ requiredEnv "GITHUB_TOKEN" }}'
username: '{{ requiredEnv "GITHUB_ACTOR" }}'
pullrequests:
helm-charts:
kind: github
scmid: helm-charts
spec:
automerge: false
labels:
- dependencies
autodiscovery:
# scmid is applied to all crawlers
scmid: helm-charts
crawlers:
helm:
# To ignore specific path
#ignore:
# # - path: <filepath relative to scm repository>
# # - path: chart/*
#only:
# # - path: <filepath relative to scm repository>
# # - path: chart/*Then run updatecli diff --experimental --config updatecli.d/autodiscovery.yaml
To see what would have to change.
I still believe that we need to execute more custom pipeline in parallel, hence the reason to allow to opt out using the ignore/only configuration. But it's not yet clear how much configuration I need from the autodiscovery versus writing the source/condition/target manifest
hairmare
commented
1 year ago Hey @olblak
Thanks very much for you input. I have in fact been keeping an eye out for the autodiscovery feature as i have a bunch of additional repos that will probably benefit from it.
I have been experimenting with kind: github and it seems to do what we need from it. I'll certainly take autodiscovery for a spin and check if it works for our use case.
What make our use case special seems to be that we want to update versions in random values in a helm chart rather than managing dependencies.
One area where i still see potential for improvement is when it comes to generating descriptions for PRs. In the best case i'd like to configure updatecli so it does something roughly similar to how dependabot adds autogenerated changelogs to it's PR descriptions. I'm still groking the updatecli code and plan on looking into maybe implementing something in this direction once i understand it.
olblak
commented
1 year ago What make our use case special seems to be that we want to update versions in random values in a helm chart rather than managing dependencies.
Tell me if I am wrong, but what you want to do is to update a random key such as in the following file where you want to automatically bump the value of the tag based on a value coming from let say a git tag, or a github release.
yourApp:
image: ghcr.io/adfinis/yourApp
tag: 1.0.0If it's the case, then we do that all the time. In this example
We want to update UI docker image version used by the Helm chart as soon as a developer publish a new tag but only if a docker image with the corresponding tag as been published.
The source information is a git tag, the target to update is a helm chart value. The kind "helmchart" provides more settings specific to helm chart update than the kind "yaml"
Something that is maybe not clear enough is that every condition or target spec has a default field set to the source out. So for example this line could be removed. I guess it was there because at some point we had something like "'{{ source "ui-backend-tag" }}-beta"
One area where i still see potential for improvement is when it comes to generating descriptions for PRs. In the best case i'd like to configure updatecli so it does something roughly similar to how dependabot adds autogenerated changelogs to it's PR descriptions. I'm still groking the updatecli code and plan on looking into maybe implementing something in this direction once i understand it.
That's something difficult to solve because depending on the source the changelog is not alway available. So for example github releases usually have a changelog but from a docker registry, it can be difficult to identify... I have been scratching my head around this for a while, and I think we need some kind of standartisation. At the moment I use the pullrequest setting name "description" to provide custom message in my PR body.
Every Updatecli must implement the changelog interface as here but most of the time there is no standard way to retrieve that information.
Is your feature request related to a problem? Please describe. Keeping Chart versions bumped when implementing features and preparing PRs for rollout can be cumbersome and a tad frustrating.
Describe the solution you'd like A GitHub Action that takes care of keeping the versions how they should be for us.
chart-updater-action/<chartname>/<depname>)*-appschartDescribe alternatives you've considered We continue bumping manually and maybe start using tools like helm-local-chart-version as part of our local workflow.
Affected chart none, CI
Additional context This might save some time in those moments where a lot of bumps are ready for a given chart. This could also be the foundation for more automation like update PRs at a later stage.