hashicorp / terraform-cdk

Define infrastructure resources using programming constructs and provision them using HashiCorp Terraform
https://www.terraform.io/cdktf
Mozilla Public License 2.0
4.88k stars 456 forks source link

Allow skipping bugged resources, or allowlist used resources for the imported provider. #2892

Open yuriy-yarosh opened 1 year ago

yuriy-yarosh commented 1 year ago

Community Note

Description

It would save too much time and effort if CDKTF adopters could skip unused resources imports for the existing providers, with a respective resource whitelist in the cdktf.json config file.

{
  "language": "golang",
  "app": "echo CDKTF breaks too often and eats too much RAM, JSII should've been written in Rust",
  "terraformProviders": [{
    "name": "aws" ,
    "version": "~> 5.0.1", # both 5.0.1 and 4.67.0 are broken now because of the following resources
    "include": [],
    "exclude": ["aws_quicksight_template", "aws_appsync_domain_name"]
  }]
}
export interface TerraformDependencyConstraint {
  readonly name: string; // name of the module / provider
  readonly source?: string; // path / url / registry identifier for the module / provider
  readonly version?: string; // version constraint
  readonly include?: string[]; // get and import only the specified resources, import everything if empty or unspecified
  readonly exclude?: string[]; // ignore any buggy or too complex resources, that are breaking builds, or take too much time
type RequirementDefinition = string | TerraformDependencyConstraint;

References

It's pointless and cumbersome to regenerate and re-import everything from scratch, over and over, without any incremental builds (as per #2116 ).

Existing generated modules are bloated with JSII bundles, about 1.5Gb (as per #2829 ).

And existing providers are conflicting from time to time with JSII naming conventions (as per AWS Provider #31615 and #2879).

CDK8S has it's escape hatches, and patching yaml's during import is a common practice there, due to various compatibility issues (as per CDK8S #1323 and #970 sometimes the whole codgen is very inconsistent #1326).

It's possible to workaround cdk8s import issues with some manual yaml patching. But the respective CDKTF escape hatches are rather undocumented and would be fairly inconvenient. It would be really weird to make a git subtree out of tf provider, with the respective patched release, just to workaround some broken codegen.

yuriy-yarosh commented 1 year ago

right now

It's literally taking around 90 min right now, OOM's on 4 threads Archlinux box with 32Gb of RAM.