GoogleCloudPlatform / cloud-foundation-fabric

End-to-end modular samples and landing zones toolkit for Terraform on GCP.
Apache License 2.0
1.51k stars 868 forks source link

fast/extras/0-cicd-github creates corrupted PNG (base 64 encoded content) #2503

Closed lyricnz closed 1 month ago

lyricnz commented 2 months ago

Describe the bug

Applying fast/extras/0-cicd-github creates corrupted PNG output and ultimately fails with error.

Environment

❯ terraform -version
Terraform v1.5.6
on darwin_amd64

Also tried with opentofu (same result)

❯ tofu --version
OpenTofu v1.8.1
on darwin_amd64
❯ git rev-parse --short HEAD
0420dec3

To Reproduce Create a tfvars with any repo:

repositories = {
  fast_00_bootstrap = {
    create_options = {
      description = "FAST bootstrap."
      features = {
        issues = true
      }
    }
    populate_from = "../../stages/0-bootstrap"
  }

Execute: terraform init + apply

Look at PNG in the output - they are base64 encoded text, not the original binary content.

Expected behavior Files in the new repos to be intact, and mostly identical to the original.

Result Terraform output and/or error messages

github_repository_file.default["fast_02_networking/IAM.md"]: Creation complete after 36s [id=fast_02_networking/IAM.md]
github_repository_file.default["fast_00_bootstrap/IAM.md"]: Creation complete after 35s [id=fast_00_bootstrap/IAM.md]
github_repository_file.default["fast_01_resman/outputs-gcs.tf"]: Creation complete after 36s [id=fast_01_resman/outputs-gcs.tf]
github_repository_file.default["fast_00_bootstrap/variables.tf"]: Creation complete after 36s [id=fast_00_bootstrap/variables.tf]
github_repository_file.default["fast_00_bootstrap/diagram.png"]: Creation complete after 36s [id=fast_00_bootstrap/diagram.png]
github_repository_file.default["fast_00_bootstrap/outputs.tf"]: Creation complete after 36s [id=fast_00_bootstrap/outputs.tf]
╷
│ Error: unsupported content encoding: none
│
│   with github_repository_file.default["fast_02_networking/diagram-ncc.png"],
│   on main.tf line 159, in resource "github_repository_file" "default":
│  159: resource "github_repository_file" "default" {
│
╵

Additional context Add any other context about the problem here

lyricnz commented 2 months ago

I can't see how https://registry.terraform.io/providers/integrations/github/latest/docs/resources/repository_file actually supports uploading a binary file (there's no flag to let it know that the content is base64 encoded).

This commit totally doesn't work (for me) https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/commit/a514ce0ef5743b065c1f1788f0b9ec3d9d3650b0#diff-75d6741a4ff5654842d3a983ac5376d3485db0914e89f004825fda996162c569 @jayBana

lyricnz commented 2 months ago

See https://github.com/integrations/terraform-provider-github/issues/710

I can't see anything in https://github.com/integrations/terraform-provider-github/blob/main/github/resource_github_repository_file.go either

wiktorn commented 2 months ago

My guess is that the actual error message comes from around here.

But due how the code looks as of now you should not get this exact message. This suggests that you use v5.43.0 or earlier of github provider. Is this the case? Can you try with newer version?

lyricnz commented 2 months ago

My last attempt was with opentofu:

❯ tofu --version
OpenTofu v1.8.1
on darwin_amd64
+ provider registry.opentofu.org/hashicorp/tls v4.0.5
+ provider registry.opentofu.org/integrations/github v4.31.0

PS: it "works fine" if I comment out the *.png line in main.tf - but obviously doesn't include those files in the output repo.

Will reinstall terraform, and see which version it installed. Sec.

lyricnz commented 2 months ago

Even with latest terraform, and running terraform init it installed that old version:

❯ terraform --version
Terraform v1.9.4
on darwin_amd64
+ provider registry.terraform.io/hashicorp/tls v4.0.5
+ provider registry.terraform.io/integrations/github v4.31.0
lyricnz commented 2 months ago

Updating required_providers to "~> 6.0"

❯ terraform --version
Terraform v1.9.4
on darwin_amd64
+ provider registry.terraform.io/hashicorp/tls v4.0.5
+ provider registry.terraform.io/integrations/github v6.2.3

Then applying the stock terraform, I didn't get any error, but the PNG is still corrupt:

image
ludoo commented 2 months ago

Given there seems to be little to fix on our side, can we close this issue?

ludoo commented 2 months ago

Maybe it would be worth it to mention this in the stage README, what do you think?

lyricnz commented 2 months ago

Yes, documenting "this process will corrupt any binary files, including PNG". In the alternative, maybe a non-terraform solution would work better?