hashicorp / terraform-cdk

Define infrastructure resources using programming constructs and provision them using HashiCorp Terraform
https://www.terraform.io/cdktf
Mozilla Public License 2.0
4.88k stars 454 forks source link

Asset Construct #648

Closed skorfmann closed 3 years ago

skorfmann commented 3 years ago

Community Note

Goal

When dealing with scenarios where files are either copied or generated in a construct, we need a helper Construct to move a given path into the synthesized stack output.

Implementation

Essentially, the Asset Construct needs to know about the stack its living in, so that it can copy the files accordingly during synthesis. The written asset files should be written into a folder with a globally unique name to avoid accidential naming collisions.

Simple File

Given the following example:

const fileAsset = new Asset(this, 'SampleSingleFileAsset', {
  path: path.join(__dirname, 'file-asset.txt')
});

I'd expect the following file to be present

cdktf.out/stacks/stack-a/assets/205jraiwt0295922/file-asset.txt

Simple Folder

Given the following example:

const folderAsset = new Asset(this, 'SampleLocalTerraformModule', {
  path: path.join(__dirname, 'my-terraform-module')
});

I'd expect the following file to be present

cdktf.out/stacks/stack-a/assets/klrja0925aifokwf/my-terraform-module/main.tf
cdktf.out/stacks/stack-a/assets/klrja0925aifokwf/my-terraform-module/variables.tf

Zipped Folder

Given the following example:

const archivedFolderAsset = new Asset(this, 'SampleLocalTerraformModule', {
  path: path.join(__dirname, 'my-terraform-module'),
  type: AssetType.ARCHIVE
});

I'd expect the following file to be present

cdktf.out/stacks/stack-a/assets/klrja0925aifokwf/my-terraform-module/archive.zip

Interface

new Asset(this, 'foo', {
  path: string // file or folder
  type: AssetType
  assetHash?: string
})
interface AssetType {
  FILE,
  FOLDER,
  ZIP
}
const asset = new Asset()

asset.outputPath // e.g. 'assets/${assetHash}/my-terraform-module/archive.zip'
asset.assetType // AssetType.ZIP
asset.assetHash // some sort of hash - to be defined what makes sense here. Perhaps configurable?

Deployment Workflows

Local

For local cdktf deploy workflows the actual location of the asset files are not that important. However, I think it makes sense to have one common concept, regardless of the deployment type. Thefore, I'd go with the same concept as the remote workflow.

Remote

For remote deployments via Terraform Cloud, the entire stack output folder gets archived and uploaded to Terraform Cloud / Enterprise. For this reason, it's important that the archive contains all the relevant files and that file references are relative to the stack output folder.

Use Cases

This Asset Construct will serve as the foundation for a variety of use-cases, e.g.:

Implementation in AWS CDK

The AWS CDK has a somewhat similar concept. However, since they rely on S3 as input for Cloudformation it's not fully transferable to our use-cases. Nevertheless, it shouldn't hurt to keep the use-case of remote asset handling in mind - but I wouldn't tackle this right away.

Things to keep in mind

When using 3rd party dependencies, these have to be inlined for jsii. For this reason, it makes sense to stick to the nodejs stdlib as much as possible (or in doubt copy relevant parts from the dependency) [only relevant for cdktf package]

Also - consider all of this more like a guidance for the end goal - Please feel free to change and adapt while implementing and learning more about the topic.

References

github-actions[bot] commented 1 year ago

I'm going to lock this issue because it has been closed for 30 days. This helps our maintainers find and focus on the active issues. If you've found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.