PROPOSAL: Chart tables for Adding Custom (Public or Private) Repos

[technosophos]

In nautical terminology, a Chart Table is a place where charts are stored. This proposal outlines a way to extend Helm's notion of charts to include arbitrary external sources, which we call Chart Tables.

Changes:

• Change charttable to table.
• Added Namespacing section
• Changed Charttables.yaml to Tables.yaml

  TL;DR

Add a method for supplying external "chart tables" (chart repositories), and seemlessly use them within the tool.

$ helm table list
 charts (default)
$ helm table add mycharts https://github.com/myname/myrepo
----> Added new repo named 'myrepo'
$ helm table list
 charts (default)
 mycharts
$ helm update
----> Updated 'charts'
----> Updated 'myrepo'
$ helm search myrepo/*
 mychart1
 mychart2
$ helm fetch myrepo/mychart2
----> Installing 'myrepo/mychart2` into $HELM_HOME/workspace
....

Goals

• Keep a base repository where many prominent charts can go
• Allow users to add ad hoc public Git-based repositories
• Support private repositories, too!

Non-goal: We are not trying to switch to a fully distributed chart-per-repo model.

What Is A Chart Table?

A chart table is an extra source of charts. The default chart table, which is built into Helm, is github.com/deis/charts. This spec defines how additional chart tables are added, stored, and utilized.

To build a chart table, we need three pieces of information:

• the local name of the chart table
• the remote access point
• the protocol(s) used to access the remote

At this point, we suggest that (a) all remote access points are Git repos, and (b) only git protocols are suppoted.

The Structure of the $HELM_HOME Chart Tables

Inside of $HELM_HOME, charts are currently stored inside of cache/charts. The cache directory has no other subdirectories. The reasoning for having it this way is so that there is a place to put extra chart tables. So we can carry out the initial design by adding structure as follows:

$HELM_HOME
    |
    |- cache/
        |
        |- charts/    # Default charts
        |
        |- CHARTTABLE1/
        |
        |- CHARTTABLE2/

In the above, each chart table (charts, CHARTTABLE1, CHARTTABLE2) contains any number of charts.

Namespace Model

The chart tables are organized into namespaces of the form tablename/chartname.

Therefore, to install a chart named nginx in a table named jackfrancis, the proper install command is:

$ helm install jackfrancis/nginx

Exactly one table may be specified as the default chart table. By default, the github.com/deis/charts repository is the default. A default table does not require a prefix. Consider the following command:

$ helm install nginx

This will look for a chart named nginx in the default table. If no such chart is found in the default table, an error is returned. There is no other hierarchy or resolution order for other tables.

Other commands are expected to follow this convention, with one notable exception. Searching may be done across chart tables. (In the future, other operations may be added that follow that rule.)

The Tables.yaml File

Metadata about chart tables are stored inside of $HELM_HOME/cache/Tables.yaml. The general structure of this file is as follows:

default: charts
tables:
  - name: charts
    repo: github.com/deis/charts
  - name: mycharts
    repo: github.com/technosophos/mycharts

This defines two top-level entities: the default keyword, which references one of the tables by name, and the tables keyword, which supplies a list of tables.

Each chart table has a name and a repo, where the name is user-supplied and the repo is the remote destination.

• name is composed of alphanumeric characters plus -, _, and .. It MUST begin with an alphanumeric character (as defined by Unicode)
• repo is a repository URL. TODO: Do we require protocol prefix, or guess it?

OPEN QUESTION: What is the best way to allow users to authenticate to private repos? I am assuming we can use the default Git method for keypairs. Am I missing alternatives?

Additions to the Helm Command

The following new commands are added to Helm:

• helm table add [NAME] [REPO]: Add a new chart table with the name NAME and the repository URL REPO. The user may give her/his own names to the repositories.
• helm table remove [NAME]: Remove a table by name
• helm table or helm table list: List all known tables.

  Modifications to the Helm Command

One flag is deleted: --repo

The following commands are modified:

• helm update: Now updates ALL chart tables.
• helm update NAME: Now updates a specific chart table by name
• helm search: Now searches all tables, optionally takes a chart table prefix (helm search foo/* searches only the foo table)
• helm install and helm update: These accept namespaced charts. Any chart that does not have a table prefix is assumed to be in the default table.

  Modifications to Manifests

Manifests ought to expose labels indicating their origin chart.

labels:
  chart: mytable/chart1

[technosophos]

Addresses (I think) #111 #109

/cc @jstrachan @bacongobbler and @sgoings

[bacongobbler]

LGTM. RE: the protocol question for the repo URL, I think it's completely fine to just go forward with support for git chart tables only. That way we don't have to worry about the URL format as much and focus on https, git and ssh protocol support.

If we're using git, then we only need to handle ssh protocol support, which then the user would just have to add the appropriate keys to their ssh agent to pull down a private chart table. Does that seem fair?

[jackfrancis]

• On authentication, +1 on relying on git for this. (A "private" table would be a "private" repo with appropriate git permissions that match to analogous chart CRUD operations.)
• Question: are the tables in "tables" array in the config file ordered? Is there fall-through a-là switch-case? Follow-up question: if not, how do we handle name collisions?
• I think the terms Charttable and charttable are a mouthful. I would prefer for our official terminology to go all in on table, and to employ/evoke the concept of a "Chart Table" when discussing and describing the project at a high level. What do I mean in practice? It would mean that the sub-heading above would read What is a Chart Table? instead of What is a Charttable?. And it would mean that Charttable.yml becomes Tables.yml (or Table.yml).

This is a terrific proposal!

[technosophos]

@jackfrancis I was figuring that we'd treat the "name" of a chart as the combination of its table name and its chart name. So if you and I each had an nginx chart, they'd be technosophos/nginx and jackfrancis/nginx. Only the chart in the default table can be used without qualification. Thus, if there is not chart/nginx, then helm fetch nginx would fail, perhaps with a message suggesting searching.

Regarding your third bullet, I agree. We should call them Chart Tables, but in code and references, abbreviate to table. I'll modify the proposal above to incorporate your suggestions.

[tristanz]

+1 for making the packaging system more federated.

-1 for chart and table as name. Why not just call this stuff packages? Helm is a great name for a package manager, but I don't see any advantage of the term chart or table. It's confusing. I thought charts was some custom graph dependency framework you guys were inventing until I looked into it more closely and realized it was just a package.

Helm is a package manager for Kubernetes and it allows you to install, remove, and update packages.

My two cents.

[mboersma]

-1 for chart and table as name. Why not just call this stuff packages?

I agree the terminology may be getting intricate, and maybe "table" instead of "package source" is more clever than informative.

(In Helm's defense, homebrew installs packages, but does so through "formulae." Helm installs packages, but does so through "charts.")

[jackfrancis]

That's great feedback, @tristanz. Here are my personal thoughts on your questions:

• A reason for not just calling the primary unit of helm a "package" is because of the meta-nature of what helm is trying to organize (it would be just as correct to say that it is a "package" of "packages"), and to suggest to the user that the primary unit he/she will be using/maintaining is less of a thing and more of a plan.
• Another reason is that by continuing further the maritime theme begun with the kubernetes project, we are able to inject a metaphorical, literary energy into the project that, hopefully, comes across as spirited, and will give the CLI user a sense of "fun" when typing these commands. In this we were also inspired by the Homebrew project, which eschews the more obvious nouns/verbs in certain areas, e.g., brew tap to manage repos and referring to packages as "formulae".

I agree that one of the side-effects to taking this tack could be confusion. We have discussed having more straightforward synonyms for the CLI especially, for example being able to identically express helm table commands as helm repo.

Thanks for your two cents, this makes four!

[mboersma]

to inject a metaphorical, literary energy into the project...

I am all in favor of this, Captain Jack. Arrr!

[tristanz]

:) I understand. As a point of reference, I still don't understand what a homebrew tap or formulae is despite using homebrew to install everything. helm + a whimsical logo perhaps...

[technosophos]

Off the table for the present discussion: "chart" as a name. That's pretty much a decision that's been made, and this present proposal doesn't really have any bearing on that.

On the table for the present discussion: "table" as a name. Anyone have suitable alternatives? Someone suggested "repo" and someone suggested "package source". Git uses "remote". Pros and cons to using a custom term (like table) in lieu of a generic term (like repo)

Short of the name, are there any other comments on the proposed architecture?

[technosophos]

Right now, I'm using helm remote add, following the git convention. Maybe repo would be better since it's shorter.

[rimusz]

+1