Allow current host to delegate provisioning AWS to Cloud 9

queglay commented 4 years ago

Current Terraform Version

0.12.29

Use-cases

Using terraform on a local machine to start AWS resources means you need secret keys configured. Alternatively using terraform in a cloud 9 instance does not need secret keys since it can use the role, but then provisioning onsite resources behind a NAT can also be difficult from there or open up a vulnerability that otherwise wouldn't be necessary.

Attempted Solutions

None

Proposal

A Cloud 9 instance in a private subnet can sit behind a bastion host already. It would be an excellent ability to delegate AWS provisioning to a Cloud 9 instance from any onsite host using Terraform.

Advantages:

The Role in the AWS account for the instance allows a Cloud 9 instance to have all abilities terraform requires without secret keys being passed around. Terraform running onsite can connect to cloud 9 instance to provision resources in AWS. Only an SSH key is required to access the cloud 9 instance.
Terraform can still provision onsite resources easily and securely using the onsite node, since it is already behind the NAT gateway. This avoids the need for any onsite bastion or port forwarding if we wanted to provision onsite resources from Cloud 9.
Additionally, multiple nodes could establish outbound connections from any network to terraform cloud, to register themselves as runners. This means no inbound communication would be needed to be configured on any network for the runners to operate. Resources could be provisioned from any of these runners. Another advantage here would be that AWS Cloud 9 instances have the privileges required to create AWS resources, and onsite nodes can be used to provision, as described above without port forwarding or bastions since all communication can be egress only.

apparentlymart commented 4 years ago

Hi @queglay! Thanks for sharing this feature request.

What you've described here sounds very interesting. I've not previously used Cloud9 so I'm trying my best to understand what this would look like based only on a few minutes reading the Cloud9 marketing material.

From how the service is described, it seems like what you are describing would be realized by running the Terraform CLI in the Cloud9 terminal. Does Cloud9 allow the installation of custom software in there, or is it limited only to the pre-packaged language runtimes they list in the documentation?

queglay commented 4 years ago

Correct, you can install TF to a Cloud 9 host. It runs Amazon Linux or Ubuntu. The main difference being you have a web browser providing you with a shell and an IDE after you have done your standard 2FA login to AWS, so you don't need to manage ssh keys for it, or provide AWS secret keys to perform CLI actions / TF deployments. I have tested this to start a vault deployment and it worked well.

I particularly think the last point opens possibilities, I should have placed it first: Similar to a gitlab runner, you may have a terraform host behind NAT onsite, and Cloud 9 instances also behind NAT in AWS. They could authenticate as runners to terraform cloud (a service based connection).

Only authentication to terraform cloud would be required from that point on. From there we could use Terraform to run local provisioners on those hosts, with Ansible or whatever too. It makes bridging the gap between sites more secure (establish outgoing from NAT, no bastion required) and simplifies the first secret transfer problem (no secret key transfer or storage of it, no user management of ssh keys required to connect the runners).

queglay commented 4 years ago

Maybe it is actually already covered by this business feature, not sure, though it seems about right.

https://www.terraform.io/docs/cloud/workspaces/agent.html

I was hoping to share my open source VFX rendering project this way, hopefully it could become open one day if it is indeed what the feature request is.

apparentlymart commented 4 years ago

Hi @queglay,

The primary goal of Terraform Cloud Agents is to allow those who use the Terraform Cloud workflow features (automatic triggering runs from VCS, for example) to still run Terraform on their own infrastructure, rather than in the execution environments provided by Terraform Cloud. If your goal is to run Terraform directly without Terraform cloud anyway then I wouldn't expect all of that extra indirection to be helpful to your goal.

I think what I'm not really following here is what is missing for running Terraform in Cloud9 today. It sounds like you can already install Terraform in Cloud9 and run it there, which would seem to realize all of the benefits you described without any need to route requests indirectly via Terraform Cloud. It seems similar to the typical non-Cloud deployment pattern of having Terraform run in automation orchestrated by, for example, a CI or workflow automation system.

I'm sure I'm missing the point here though, which I'm sorry for. Could you say a little more about what part you find missing today that prevents you from running Terraform in Cloud9? Thanks!

queglay commented 4 years ago

Terraform can run in cloud9, after further thought on this and trying to clarify it, I'm seeking to extend how useful this could be to improve deploying a hybrid environment.

This example demonstrates what I mean, If I want to easily run a hybrid deployment, consider setting up a VPN server and client as an example. A) If we start the deployment of the AWS resources from an onsite terraform host, we need to setup secret keys to do that. B) If instead we start the deployment in cloud9 (or really any AWS instance with an appropriate role), the permissions are provided by the instance role automatically, great!

...but if we also want to configure some onsite resources (like the vpn client to automatically connect to our vpn access server we just setup), then we would need to have an SSH bastion for our onsite subnet to do that. we just lost some of the advantage provided by A) in opting for B).

So the suggestion solves this problem: If both our cloud9 instance (or really any instance in AWS actually with an appropriate role), and an onsite instance could both phone home to terraform cloud, both could operate as runners sitting behind NAT, no port forwarding or bastions required. If terraform can delegate to both of these runners in a deployment, then we get the benefits of both A) and B).

AWS resources can be created without a secret key floating around, since we can delegate to that AWS runner/cloud9.
Onsite resources can be provisioned from terraform without needing to port forward or create a bastion (quite an annoyance, and potential vulnerability).
Both instances can sit behind NAT, and no bastions required.

back story: I think this opens up possibilities in creating software that allows end users to use operate cloud resources more natively. I am producing open source rendering infrastructure for feature film VFX, and I want people to be responsible for their own services. I don't want to necesarily have to manage those services for them. Solutions like this can help alleviate the learning curve for artists to create hybrid infrastructure to render with and not needing to create another man in the middle for secrets management. This is usually resolved by another service provider, which I'd like to only be optional for artists, rather than required for their special sauce in secrets management to use cloud.

hashicorp / terraform