[meta] Charts Infrastructure

[slack]

This is a meta ticket to build out the chart publishing infrastructure. This issue may move somewhere else, but for now this seems like the best place!

Relates to #349

Workflow releases should be published as signed and versioned charts available at charts.deis.com.

We need to build out the production repo infrastructure and automation, including:

• nail down chart hosting location (GCS, S3)
  • bucket access logs so we can keep an eye on downloads
• CDN-ify charts.deis.com (optional?)
• SSL-ify charts.deis.com
• automate chart publishing (packaging):
  • index regeneration
  • provenance file generation
  • publish .prov and .tgz to hosting location
• document as part of the release process

[slack]

This is a draft so @helgi @kmala feel free to edit/update the issue!

[helgi]

Off hand: throwing CDN in there often means none / reduce / funky logs but something to look at if we want CDN (eventually)

[kmala]

we can publish charts in two ways: 1) Individual component repo:

• Each chart will be its own repo like builder in deis/builder etc., and a merge to master will package the charts and upload to the chart repo with the same semver(ex: 2.0.0-dev) always and a tag creation should package and upload to the chart repo with the semver as the tag.
• Workflow repo will have the main chart with all other components as dependencies and version as the semver which we used to merge to master( 2.0.0-dev) and a tag release should update those versions with the latest tags of the each component and publish the release charts.
• Jenkins e2e job should fetch the chart in the master and do a helm dependency update before an install so that the latest changes are always taken.
• The main disadvantage with this approach is that some of our components are dependent on common resources like builder, database and registry require objectstore secret and where should we place that secret. One way would be to duplicate the secret under each repo which means we now create 3 objectstore secrets instead of one.

2) Current Approach:

• Have all the charts in a single repo like https://github.com/kmala/charts/tree/gh-pages with all the charts of workflow under a single folder but each component has subfolder.
• e2e jobs will work in the same way as currently pull the dev charts and install it or release chart during a release.

[bacongobbler]

I like the first approach.

With each chart in their own repository, we can refer to the meta-package for the packages that depend on each other, while allowing the components that can stand alone as a separate chart some breathing room (i.e. the router).

The main disadvantage with this approach is that some of our components are dependent on common resources like builder, database and registry require objectstore secret and where should we place that secret.

I'd place that secret in the workflow chart, and just document the components that cannot stand on their own (i.e. Workflow-specific components) a big fat warning/message saying "this chart cannot be deployed alone. Please install the workflow chart to get this package up and running".

[mboersma]

I'd place that secret in the workflow chart, and just document the components that cannot stand on their own.

I like that solution as well as the rest of approach 1). Having the chart for each component live in its repo furthers the goal of making components independently usable (even if some of them aren't yet).

[kmala]

@slack @helgi do you want to add anything?

[jchauncey]

we shoudl think of components like builder as a library. Its not useful on its own but can be shipped independently from other things.

[kmala]

@jchauncey did you mean can't be shipped independently?

[bacongobbler]

I think @jchauncey agrees that we should go forward with option 1 :)

[jchauncey]

yes we should do option 1

[helgi]

I want option 1 but we need to think about how we are going to deal with development - How do I install all current components that have not been "shipped" yet?

Until we have a good answer to that then I think we may have to start with option 2 while we are pre-stable with helm v2 charts

[kmala]

How do I install all current components that have not been "shipped" yet?

for that we have to use same semver tag and do a helm dep update before installing. I had mentioned about tackling of the jenkins jobs in my approach. My only concern was with the common secrets stuff.

[helgi]

Can you describe that workflow in more detail? What if I have not committed the code? Just have it locally that is

[kmala]

if anyone is playing with the chart locally then he would install using helm install <path-to-charts> and not use remote.

[mboersma]

One way would be to duplicate the secret under each repo which means we now create 3 objectstore secrets instead of one.

I don't see a good way around this, and it doesn't sound that bad anyway.

[kmala]

approaches for release of builder and controller: 1) while creating builder/controller charts check the latest tags of the slugbuilder/runner and populate them in the builder/controller charts. This would mean we have to create a builder/controller chart after the slugbuilder/runner release is done and we should release a new builder/controller when a new slugbuilder/runner release is done even when there is no need for release of builder/controller.

2) Have slugbuilder/slugrunner have a secret or configmap as their release artifact which mentions the image tag and other things and make the builder/controller use them. This would mean each thing can be released on their own without depending on others and workflow chart will have the latest versions of all the things.

[vdice]

I vote for 2) from a cleaner/less 'special' logic CI perspective