cloudposse / atmos

👽 Terraform Orchestration Tool for DevOps. Keep environment configuration DRY with hierarchical imports of configurations, inheritance, and WAY more. Native support for Terraform and Helmfile.
https://atmos.tools
Apache License 2.0
774 stars 95 forks source link

Fast outputs of components by reading remote state directly without `terraform output` #180

Open nitrocode opened 2 years ago

nitrocode commented 2 years ago

Have a question? Please checkout our Slack Community or visit our Slack Archive.

Slack Community

Describe the Feature

The current way to get component outputs is to use this method

atmos terraform outputs eks --stack ue2-sandbox

which will run the following commands

atmos generate backend
terraform init -reconfigure
terraform output

This can take a long time and atmos already knows the s3 bucket, the s3 bucket key, and the workspace

A faster method could be used for atmos terraform stack outputs or something where it simply does the following

aws s3 cp s3://<bucket-name>/<component workspace key prefix>/<component name>/terraform.tfstate - | jq .outputs
{
  "transit_gateway_arn": {
    "value": "arn:aws:ec2:us-east-2:snip:transit-gateway/tgw-snip",
    "type": "string"
  },
  "transit_gateway_id": {
    "value": "tgw-snip",
    "type": "string"
  },
  "transit_gateway_route_table_id": {
    "value": "tgw-rtb-snip",
    "type": "string"
  }
}

or use some jq magic to turn it into this

transit_gateway_arn = "arn:aws:ec2:us-east-2:snip:transit-gateway/tgw-snip"
transit_gateway_id = "tgw-snip"
transit_gateway_route_table_id = "tgw-rtb-snip"
osterman commented 2 years ago

@nitrocode maybe this is a good feature to prototype with atmos custom CLI commands?

https://github.com/cloudposse/atmos/pull/168

I don't think we can implement this in atmos core because it assumes (a) using our tfstate backend (b) using S3 (c) using AWS. Atmos needs to be cloud agnostic.

osterman commented 2 years ago

Maybe this relates to @Gowiem's request: https://github.com/cloudposse/atmos/issues/178