provider add: unable to add aws provider due to tree too large

iarlyy commented 10 months ago

Expected Behavior

Able to add aws provider to my project and start crafting stacks

Actual Behavior

cdktf provider add aws fails with module source tree too large. I downloaded the zip file myself through github ui and after unpacking it found this:

 97M    cdktf-provider-aws-18.2.0.zip (file downloaded through github ui)
1.2G    cdktf-provider-aws-18.2.0 (unzipped folder)

provider add execution output:

 » cdktf provider add aws             
(node:48869) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
(Use `node --trace-deprecation ...` to show where the warning was created)
[2023-12-21T18:34:07.955] [INFO] default - Checking whether pre-built provider exists for the following constraints:
  provider: aws
  version : latest
  language: go
  cdktf   : 0.19.2

[2023-12-21T18:34:10.169] [INFO] default - Found pre-built provider.
Adding package github.com/cdktf/cdktf-provider-aws-go/aws @ 18.2.0
[2023-12-21T18:34:10.422] [ERROR] default - go: downloading github.com/cdktf/cdktf-provider-aws-go/aws/v18 v18.2.0

go: downloading github.com/cdktf/cdktf-provider-aws-go/aws/v18 v18.2.0
[2023-12-21T18:34:16.357] [ERROR] default - go: github.com/cdktf/cdktf-provider-aws-go/aws/v18@v18.2.0: create zip: module source tree too large (max size is 524288000 bytes)

go: github.com/cdktf/cdktf-provider-aws-go/aws/v18@v18.2.0: create zip: module source tree too large (max size is 524288000 bytes)
non-zero exit code 1
Error: non-zero exit code 1
    at ChildProcess.<anonymous> (/opt/homebrew/Cellar/cdktf/0.19.2/libexec/lib/node_modules/cdktf-cli/bundle/bin/cmds/handlers.js:92:71732)
    at Object.onceWrapper (node:events:634:26)
    at ChildProcess.emit (node:events:519:28)
    at ChildProcess.emit (node:domain:488:12)
    at maybeClose (node:internal/child_process:1105:16)
    at ChildProcess._handle.onexit (node:internal/child_process:305:5)
Collecting Debug Information...
/opt/homebrew/Cellar/cdktf/0.19.2/libexec/lib/node_modules/cdktf-cli/node_modules/yoga-layout-prebuilt/yoga-layout/build/Release/nbind.js:53
        throw ex;
        ^

Error: non-zero exit code 1
    at ChildProcess.<anonymous> (/opt/homebrew/Cellar/cdktf/0.19.2/libexec/lib/node_modules/cdktf-cli/bundle/bin/cmds/handlers.js:92:71732)
    at Object.onceWrapper (node:events:634:26)
    at ChildProcess.emit (node:events:519:28)
    at ChildProcess.emit (node:domain:488:12)
    at maybeClose (node:internal/child_process:1105:16)
    at ChildProcess._handle.onexit (node:internal/child_process:305:5) {
  stderr: 'go: downloading github.com/cdktf/cdktf-provider-aws-go/aws/v18 v18.2.0\n' +
    'go: github.com/cdktf/cdktf-provider-aws-go/aws/v18@v18.2.0: create zip: module source tree too large (max size is 524288000 bytes)\n'
}

Node.js v21.5.0

Steps to Reproduce

just run cdktf provider add aws, I tried v18.2.0 and 18.1.0

Versions

v18.2.0

Providers

I am trying to add my first provider

Gist

No response

Possible Solutions

In the thread linked below, they discuss about the docs folder being bundled together, and i can confirm, this folder is 1.1G large for the go provider.

https://github.com/hashicorp/terraform-cdk/issues/2829

Workarounds

No response

Anything Else?

No response

References

https://github.com/hashicorp/terraform-cdk/issues/2829

Help Wanted

[ ] I'm interested in contributing a fix myself

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment

xiehan commented 10 months ago

Hi there, thanks for the report! We can look into this after the holidays since the entire team is off until the new year.

In the meantime, I wonder if a workaround would be to try to generate the provider bindings locally rather than attempting to use the prebuilt package, as there's no zip file involved in the former. I know it's not an ideal solution because generating the bindings for the AWS provider will take quite a bit of time and memory, but it would at least unblock you and allow you to start writing code while we look to see if we can get the size of the Go provider down. You can always switch back to the prebuilt provider later when we've resolved the issue.

The easiest way to do that is to use cdktf provider add aws --force-local. You can find more documentation and examples at https://cdk.tf/imports

kalos92 commented 9 months ago

Same problem here, the only solution was to downgrade to the 18.0.9 version

github-actions[bot] commented 8 months ago

I'm going to lock this issue because it has been closed for 30 days. This helps our maintainers find and focus on the active issues. If you've found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

hashicorp / terraform-cdk