Open smallinsky opened 1 year ago
@smallinsky
I'm looking into this, but it looks like we do have --exec
flag:
$ tsh --help aws
usage: tsh aws [<flags>] [<command>...]
Access AWS API.
Flags:
...
--exec Execute different commands (e.g., terraform) under Teleport credentials
This gives you correct randomly changing credentials on each run:
$ tsh aws --exec sh -- -c export | grep -i aws
export AWS_ACCESS_KEY_ID="9872bafd-353a-4f06-8f76-d4c6d28e63e6"
export AWS_CA_BUNDLE="/Users/tener/.tsh/keys/boson.tener.io/tener-app/boson.tener.io/awsconsole-prod-localca.pem"
export AWS_SECRET_ACCESS_KEY="1ed2ee08-1619-4993-8841-dee6385af7a6"
$ tsh aws --exec sh -- -c export | grep -i aws
export AWS_ACCESS_KEY_ID="d2d6e00f-dbb2-47d6-a445-6a99f4336a2d"
export AWS_CA_BUNDLE="/Users/tener/.tsh/keys/boson.tener.io/tener-app/boson.tener.io/awsconsole-prod-localca.pem"
export AWS_SECRET_ACCESS_KEY="3e224feb-753a-45a2-a706-a224fafcea2a"
So you can simply do: tsh aws --exec terraform init
.
Your thoughts?
So you can simply do: tsh aws --exec terraform init.
@Tener
Have you tested this flow ?
I don't think that it will work since the issue is related to AWS_CA_BUNDLE
env not respected by one of internal library that is used in terraform codebase to fetch module dependency from aws s3 source.
In terms of technical flow there is not difference between tsh proxy aws --app
and tsh aws --exec
so these commands should give the same result.
All right, I was confused by the workaround step no. 3 exporting the AWS_CA_BUNDLE
. Thanks for this.
Looking at the Terraform code, the fetch in module init
is done with Hashicorp's go-getter
library. Unfortunately, there is no direct support for AWS_CA_BUNDLE
in that library.
Reading the S3Getter
code points towards another possible workaround: specify the AWS profile using aws_profile
URL param.
module "mymodule" {
source = "s3::https://***.s3.eu-central-1.amazonaws.com/mymodule/1.0.0.zip?aws_profile=app-aws"
}
Assuming you are fine with changing the .tf
files to override the aws_profile
param, this gives you a basis for a stable workaround.
Unfortunately, I don't see a way to make this work without user intervention. I'm not sure if there is a point in changing tsh
behavior, as users would have to adapt their .tf
files anyway.
I have raised PR to add support for AWS_CA_BUNDLE
in go-getter
, but I'm not sure if/when it is going to be merged: https://github.com/hashicorp/go-getter/pull/457.
Nevertheless, you can easily rebuild Terraform with my change:
git clone https://github.com/hashicorp/terraform.git terraform-git
cd terraform-git
go mod edit -replace github.com/hashicorp/go-getter=github.com/Tener/go-getter@b21a9339890e6c2a20c4dd638763de8755c7c665
go mod tidy
go install .
Given custom terraform installed to GOPATH, this will work:
tsh aws exec -- `go env GOPATH`/bin/terraform init
Assuming you are fine with changing the .tf files to override the aws_profile param, this gives you a basis for a stable workaround.
Unfortunately, I don't see a way to make this work without user intervention.
The go-getter
seems to support the ca_bundle from AWS profile configuration. I was thinking about creating a teleport profile file in tsh directory during tsh app login aws-console-app
call.
Does that sound reasonable ?
The
go-getter
seems to support the ca_bundle from AWS profile configuration. I was thinking about creating a teleport profile file in tsh directory duringtsh app login aws-console-app
call.
We could prepare this profile, but that is a heavy change. Currently, we use env variables to pass the config, including secrets. It isn't immediately clear what we should put in the profile - only the CA bundle or all of the other settings, too? There are arguments to be made either way.
The bigger problem is that even with the profile in place, we have no way to make terraform init
use it on our own. The two mechanisms available are:
?aws_profile=app-aws
AWS_SDK_LOAD_CONFIG
and other env variables, e.g. AWS_PROFILE
.There is no way we should be modifying .tf
files, so the first one is out. I don't think we should use the second either, as AWS_SDK_LOAD_CONFIG
is a heavy hammer to wield and is likely to cause numerous side issues.
What
When terraform module is initialized from S3 bucket using Teleport
tsh proxy aws
The module http flow doesn't respect the
AWS_CA_BUNDLE
env var. In the result the following error is returned:Related: https://github.com/hashicorp/terraform/issues/28551
Workaround
1) Create a aws profile with local teleport CA:
2) Start aws local proxy
3) Export local proxy env:
Export
AWS_SDK_LOAD_CONFIG
and AWS_PROFILE created in step 1.4) Run terraform script: