Investigate dynamic TypeInstances support in umbrella workflows

[pkosiec]

Description

Currently, our PostgreSQL installation Implementation produces static TypeInstance for PostgreSQL Helm release (link). The reason is the current limitation of accessing our dynamic TypeInstances in umbrella workflows.

In this case, the umbrella workflow is the Mattermost installation (link).

If we used the Helm template storage, the postgresql artifact would contain:

backend:
  context:
    goTemplate: |
      host: '{{ template "postgresql.primary.fullname" . }}'
      port: '{{ template "postgresql.port" . }}'
      defaultDBName: '{{ template "postgresql.database" . }}'
      superuser:
        username: '{{ template "postgresql.username" . }}'
        password: '{{ template "postgresql.password" . }}'
    release:
      driver: secrets
      name: example-release
      namespace: default
value: null

And the Mattermost installation umbrella workflow steps require these rendered values, to:

• create DB user
• create DB

AC

• Summarize options that we have to solve this issue
• Brainstorm and agree with team what we want to check as a part of PoC
• If possible, create a PoC of a given solution

Ideas how to solve it:

1. Change the way how we run nested workflow: maybe the Engine shouldn't render one huge workflow, but schedule and run separate Argo Workflows instead? Every workflow would have dedicated upload/download steps
   • Investigated as a part of #546
2. Save rendered value under different property in the artifact (e.g. .rendered) and use that in umbrella workflow
   • Ignore it during TypeInstance upload
   • It probably won't work for other backends, which base on actual TypeInstance existing in Hub
3. Hack: Add "fetcher" container which fetches Helm template storage backend and renders the template
   • We would need to fake TypeInstance ID, because there's no TypeInstance yet
   • It won't work for other backends, which base on actual TypeInstance existing in Hub
   • Probably no-go

Reason and use cases

See https://github.com/capactio/capact/issues/604