cloudposse / atmos

👽 Terraform Orchestration Tool for DevOps. Keep environment configuration DRY with hierarchical imports of configurations, inheritance, and WAY more. Native support for Terraform and Helmfile.
https://atmos.tools
Apache License 2.0
794 stars 96 forks source link

Release 1.48.0 breaks existing stacks that use strings with Go template formatting #470

Closed elventear closed 1 year ago

elventear commented 1 year ago

Describe the Bug

We have stacks that use strings with Go template formatting that are to be passed verbatim to downstream consumers. Atmos fails when processing those stacks because it is trying to evaluate the template context that should not be treated as a template.

Atmos fails with the error:

template: [path]: executing "[path]" at <.keyName>: map has no entry for key "keyName"

Expected Behavior

Atmos should not fail.

Steps to Reproduce

Add go template strings to a stack file.

Screenshots

No response

Environment

No response

Additional Context

Seems the latest release changed the behavior that allowed for this pattern to work without issues here:

https://github.com/cloudposse/atmos/compare/v1.47.0...v1.48.0#diff-1b8991c18c7d7059b6415e247c1c89b92525261e7fd579623cbb40a65f6a615dL173

aknysh commented 1 year ago

@elventear

please use the following format to disable Go template processing for the imported manifests where the Go templates are not for Atmos

  - path: catalog/xxx/yyy
    skip_templates_processing: true

https://atmos.tools/core-concepts/stacks/imports#imports-schema

Before 1.48.0, we checked the context - if it was not provided at all, the Go templates were not processed. We need the context for other functionality, that's why we added skip_templates_processing attribute.

Let me know if that works for you.

elventear commented 1 year ago

That didn't work for us. I added the import object as the documentation suggests and the error keeps popping up.

We are using https://github.com/cloudposse/github-action-atmos-affected-trigger-spacelift, I am wondering if the logic that action uses to check the affected stacks is not using the import object logic.

I added:

- path: <path_to_yaml>
  context: {}
  skip_templates_processing: true
  ignore_missing_template_values: false
aknysh commented 1 year ago

hmm, not sure why skip_templates_processing: true did not work for you, it def works for us. Also, the GitHub workflow needs to pin all the GHAs it uses, not to use latest (otherwise, it will be impossible to do any major changes). We'll discuss that. Thanks

elventear commented 1 year ago

I just tested interactively in my terminal with atmos describe affected and it fails exactly the same while previous versions work fine (This is on macOS/amd64). Is there a way to get some debug outputs of what atmos is doing?

elventear commented 1 year ago

The file that has the templates is imported a few levels deep from the stack. I added skip_templates_processing: true to all the files that import the leaf node that atmos is complaining about, and the issue persists.

aknysh commented 1 year ago

skip_templates_processing def works for us.

Anyway, we decided to put back the checking for an empty context, will be in the next release (prob tomorrow). Meanwhile, if you are using https://github.com/cloudposse/github-action-atmos-affected-trigger-spacelift , add this

 with:
          atmos-version: 1.47.0
elventear commented 1 year ago

We are pinning the version already.

But I would like to adopt the functionality to skip template processing as seems valuable for maintainability.

My goal is to help debug the issue.

aknysh commented 1 year ago

@elventear please use this release https://github.com/cloudposse/atmos/releases/tag/v1.49.0

elventear commented 1 year ago

I tested and it works. Using the skip_templates_processing in our case seems very valuable, if there is anything I can do to help debug the issue let me know.