[FEATURE]: Support runtime outputs for Helm chart values

[namron86]

Feature Request

Background / Motivation

Today, we use a jungle of bash scripts to pass Cloudformation and Terraform outputs to our Helm charts. Typically, these values are indeterminate until deploy time (e.g. AWS RDS endpoint URLs). We were looking to use Garden to streamline handling these outputs.

What should the user be able to do?

A Garden user should be able to reference runtime outputs in their Helm chart values. e.g.

Terraform that creates an SNS topic

----
kind: Module
type: terraform
name: tf-infra

---
kind: Module
type: helm
.
.
.
serviceResource:
    containerModule: docker-image
dependencies:
    - tf-infra
values:
    microservice:
        appConfig:
            environment:
                SNS_TOPIC_ARN: ${runtime.services.tf-infra.outputs.sns_topic_arn}
              # or
              # SNS_TOPIC_ARN: ${providers.terraform.outputs.sns_topic_arn}
        image:
            repository: ${modules.docker-image.outputs.deployment-image-name}
            tag: ${modules.docker-image.version}

Why do they want to do this? What problem does it solve?

It allows runtime outputs from Terraform (or other modules such as exec) to be resolvable at deploy time.

Suggested Implementation(s)

• Allow the Helm module values: stanza to resolve runtime outputs.
• Add an additional setValues: stanza to the Helm module to allow runtime variables to be passed to Helm via --set at deploy time.

How important is this feature for you/your team?

🥀 Crucial, Garden is unusable for us without it

We want to take a machete to our jungle of bash. Please help! :)

[namron86]

As I understand it there is a workaround that involves invoking the Terraform plugin directly to get these outputs to initially work. However, this involves adding (and possibly duplicating) the parameters required for that stack to work to the providers stanza. Additionally, it only increases the friction involved in using Garden.

Finally, it doesn't address using outputs from exec type modules.

[edvald]

Thanks for flagging this @namron86! I'm very happy to give this priority, if anything it's remarkable that this hasn't come up sooner.

[edvald]

Just for a bit of context, the reason for the current implementation was to allow plugins like conftest-kubernetes to validate rendered charts ahead of deployment. So it's not quite trivial to address, but it's worth figuring out for sure.

[edvald]

Ok, this should be ready in our next release.

[namron86]

You're the man @edvald . Thank you so much for your responsiveness.

[edvald]

🎩 👌 😉