Change proposal: `500-CNAB-dependencies` - section [Depend on a named bundle]

[MChorfa]

Background

Since the nested dependencies are not supported yet. So we decided to create multiple dependent bundles, and we want them to execute in a certain orders. But on each run we get a random execution at Runtime. Here what we get:

Declared

MYDEP-core
MYDEP-s3
MYDEP-db
MYDEP-experiments

Executed

MYDEP-experiments
MYDEP-s3
MYDEP-core
MYDEP-db

Investigation

After a back-and-forth with Porter Authors @carolynvs and @vdice, we think it would be more practical to have an array of dependencies and introduce a name field to insure bundle identification and maintain the functionality of templating values, e.g. {{ bundle.dependencies.mysql.output.foo }}.

dependencies:
  - name: mysql
    tag: myregistry/mysql:v0.1.0
  - name: wordpress
    tag: myregistry/wordpress:v0.1.0

Related issue: https://github.com/deislabs/porter/issues/969

Proposal

Change the requires listing to an array, an individual listing now has a name field. Thus, Runtimes are expected to process listings in order of appearance. Thus we would have:

{
  "custom":{
    "dependencies":{
      "requires":[
        {
          "name":"storage",
          "bundle":"somecloud/blob-storage"
        },
        {
          "name":"mysql",
          "bundle":"somecloud/mysql",
          "version":{
            "prereleases":true,
            "ranges":[
              "5.7.x"
            ]
          }
        }
      ]
    },
    "name":"wordpress"
  }
}

[jlegrone]

Disclaimer: this comment is incomplete but I wanted to get something down quickly to help provide for further discussion.

This is very similar to what we've been calling "CNAB workflows" in our (currently closed source) CNAB runtime. CNAB workflows allow users to specify dependencies between multiple installations, expressed as a directed acyclic graph.

Describing workflows as a graph allows the runtime to create a deployment plan that can optimize for parallelism, and also provides a mechanism for installations to share data by mapping the outputs of one bundle to parameters of another.

One other property of CNAB workflows is that components deployed by separate bundles are expected to have unique installation names. This ensures that if a bundle relies on the uniqueness of the installation name it is passed in order to track some external state, multiple invocations of that bundle can be used from the same workflow without introducing collisions. This does also come with the downside that dependencies cannot be self-described by a bundle alone; the workflow becomes the deployable artifact instead.

Here is an example workflow spec encoded as yaml:

metadata:
  description: deploy service foo and its dependencies
spec:
  workflow:
    tasks:
      cache:
        conditions:
        - taskSpec:
            action:
              action: snapshot
              bundle:
                contentDigest: sha256:deadbeef
                name: library/redis
              installation: foo-redis.example.com
        spec:
          installation:
            bundle:
              contentDigest: sha256:deadbeef
              name: library/redis
            name: foo-redis.example.com
      database:
        conditions:
        - taskSpec:
            action:
              action: snapshot
              bundle:
                name: library/postgres:12.2
              installation: foo-postgresql.example.com
        spec:
          installation:
            bundle:
              name: library/postgres:12.2
            name: foo-postgresql.example.com
      service:
        conditions:
        - taskName: cache
        - taskName: database
        spec:
          installation:
            bundle:
              contentDigest: sha256:deadbeef
              name: foo
            name: foo.example.com
            parameters:
              postgres_password: {}

[MChorfa]

So the magic happens with:

      service:
        conditions:
        - taskName: cache
        - taskName: database

If the dependencies object does have a sequence list. We could come around the issue by keeping the uniqueness aspect and ensure the desired order. Where we can have:

{
  "custom": {
    "dependencies": {
      "sequence": ["storage", "mysql"],
      "requires": {
        "storage": {
          "bundle": "somecloud/blob-storage"
        },
        "mysql": {
          "bundle": "somecloud/mysql",
          "version": {
            "prereleases": true,
            "ranges": ["5.7.x"]
          }
        }
    },
  },
  "name": "wordpress"
}

[carolynvs]

I really like the ordering change, and how it is additive. This is something we'd like to try out immediately in Porter as everyone is asking for it as soon as we can ship it. 😀