integrations / terraform-provider-github

Terraform GitHub provider
https://www.terraform.io/docs/providers/github/
MIT License
888 stars 725 forks source link

[BUG]: github_branch data source constructs incorrect API URL for public repo #1938

Closed NickFitz closed 1 month ago

NickFitz commented 11 months ago

Expected Behavior

Given the provider configuration:

provider "github" {
  owner = "someorg"
}

the data source:

data "github_branch" "main" {
  repository = "some_public_repository"
  branch = "main"
}

is expected to result in data.github_branch.main having the attributes documented at https://registry.terraform.io/providers/integrations/github/latest/docs/data-sources/branch relating to the GitHub repository at someorg/some_public_repository.

Examining the DEBUG logs shows that the path being requested from the GitHub API is

/repos//some_public_repository/git/ref/heads/main

when it should be

/repos/someorg/some_public_repository/git/ref/heads/main

Setting the GITHUB_OWNER environment variable does not resolve the issue. It appears from the debug output that the absence of a token (which is not required as the repository is public) causes the data source to configure an anonymous owner. But an anonymous owner does not seem to have a name, and therefore the interpolation of the owner name as the organisation name in URL construction fails, resulting in the // seen in the URL.

This appears to be the same, or very similar, behaviour to that identified in issue #942 with the github_repository_file data source. A fix for that issue is in #1494, and from comparing that PR and the corresponding code in github_branch, I believe the fix would be the same or very similar. I also suspect the same issue may be present in other data sources.

Actual Behavior

The attributes of data.github_branch.main are all null due to the malformed path in the API request, which results in a 404 Not Found response.

Terraform Version

Terraform v1.5.7
on darwin_arm64
+ provider registry.terraform.io/hashicorp/aws v5.18.1
+ provider registry.terraform.io/hashicorp/local v2.4.0
+ provider registry.terraform.io/integrations/github v5.39.0

Affected Resource(s)

Terraform Configuration Files

No response

Steps to Reproduce

terraform apply

Debug Output

2023-10-05T11:35:49.128+0100 [WARN]  ValidateProviderConfig from "provider[\"registry.terraform.io/integrations/github\"]" changed the config value, but that value is unused
2023-10-05T11:35:49.129+0100 [DEBUG] provider.terraform-provider-github_v5.39.0: 2023/10/05 11:35:49 [INFO] Selecting owner  from GITHUB_OWNER environment variable
2023-10-05T11:35:49.129+0100 [DEBUG] provider.terraform-provider-github_v5.39.0: 2023/10/05 11:35:49 [INFO] Setting write_delay_ms to 1000
2023-10-05T11:35:49.129+0100 [DEBUG] provider.terraform-provider-github_v5.39.0: 2023/10/05 11:35:49 [DEBUG] Setting read_delay_ms to 0
2023-10-05T11:35:49.129+0100 [DEBUG] provider.terraform-provider-github_v5.39.0: 2023/10/05 11:35:49 [DEBUG] Setting parallel_requests to false
2023-10-05T11:35:49.129+0100 [DEBUG] provider.terraform-provider-github_v5.39.0: 2023/10/05 11:35:49 [INFO] No token present; configuring anonymous owner.
2023-10-05T11:35:49.129+0100 [DEBUG] ReferenceTransformer: "module.some_project.data.github_branch.main" references: []
2023-10-05T11:35:49.130+0100 [DEBUG] provider.terraform-provider-github_v5.39.0: 2023/10/05 11:35:49 [DEBUG] GitHub API Request Details:
2023-10-05T11:35:49.130+0100 [DEBUG] provider.terraform-provider-github_v5.39.0: ---[ REQUEST ]---------------------------------------
2023-10-05T11:35:49.130+0100 [DEBUG] provider.terraform-provider-github_v5.39.0: GET /repos//some_public_repository/git/ref/heads/main HTTP/1.1
2023-10-05T11:35:49.130+0100 [DEBUG] provider.terraform-provider-github_v5.39.0: Host: api.github.com
2023-10-05T11:35:49.130+0100 [DEBUG] provider.terraform-provider-github_v5.39.0: User-Agent: go-github/v55.0.0
2023-10-05T11:35:49.130+0100 [DEBUG] provider.terraform-provider-github_v5.39.0: Accept: application/vnd.github.v3+json,application/vnd.github.stone-crop-preview+json
2023-10-05T11:35:49.130+0100 [DEBUG] provider.terraform-provider-github_v5.39.0: X-Github-Api-Version: 2022-11-28
2023-10-05T11:35:49.130+0100 [DEBUG] provider.terraform-provider-github_v5.39.0: Accept-Encoding: gzip
2023-10-05T11:35:49.130+0100 [DEBUG] provider.terraform-provider-github_v5.39.0: 
2023-10-05T11:35:49.130+0100 [DEBUG] provider.terraform-provider-github_v5.39.0: 
2023-10-05T11:35:49.130+0100 [DEBUG] provider.terraform-provider-github_v5.39.0: -----------------------------------------------------
2023-10-05T11:35:49.378+0100 [DEBUG] provider.terraform-provider-github_v5.39.0: 2023/10/05 11:35:49 [DEBUG] GitHub API Response Details:
2023-10-05T11:35:49.378+0100 [DEBUG] provider.terraform-provider-github_v5.39.0: ---[ RESPONSE ]--------------------------------------
2023-10-05T11:35:49.379+0100 [DEBUG] provider.terraform-provider-github_v5.39.0: HTTP/2.0 404 Not Found
2023-10-05T11:35:49.379+0100 [DEBUG] provider.terraform-provider-github_v5.39.0: Access-Control-Allow-Origin: *
2023-10-05T11:35:49.379+0100 [DEBUG] provider.terraform-provider-github_v5.39.0: Access-Control-Expose-Headers: ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset
2023-10-05T11:35:49.379+0100 [DEBUG] provider.terraform-provider-github_v5.39.0: Content-Security-Policy: default-src 'none'
2023-10-05T11:35:49.379+0100 [DEBUG] provider.terraform-provider-github_v5.39.0: Content-Type: application/json; charset=utf-8
2023-10-05T11:35:49.379+0100 [DEBUG] provider.terraform-provider-github_v5.39.0: Date: Thu, 05 Oct 2023 10:35:49 GMT
2023-10-05T11:35:49.379+0100 [DEBUG] provider.terraform-provider-github_v5.39.0: Referrer-Policy: origin-when-cross-origin, strict-origin-when-cross-origin
2023-10-05T11:35:49.379+0100 [DEBUG] provider.terraform-provider-github_v5.39.0: Server: GitHub.com
2023-10-05T11:35:49.379+0100 [DEBUG] provider.terraform-provider-github_v5.39.0: Strict-Transport-Security: max-age=31536000; includeSubdomains; preload
2023-10-05T11:35:49.379+0100 [DEBUG] provider.terraform-provider-github_v5.39.0: Vary: Accept-Encoding, Accept, X-Requested-With
2023-10-05T11:35:49.379+0100 [DEBUG] provider.terraform-provider-github_v5.39.0: X-Content-Type-Options: nosniff
2023-10-05T11:35:49.379+0100 [DEBUG] provider.terraform-provider-github_v5.39.0: X-Frame-Options: deny
2023-10-05T11:35:49.379+0100 [DEBUG] provider.terraform-provider-github_v5.39.0: X-Github-Api-Version-Selected: 2022-11-28
2023-10-05T11:35:49.379+0100 [DEBUG] provider.terraform-provider-github_v5.39.0: X-Github-Media-Type: github.v3; format=json, github.stone-crop-preview; format=json
2023-10-05T11:35:49.379+0100 [DEBUG] provider.terraform-provider-github_v5.39.0: X-Github-Request-Id: CF2C:80DC:208312D:20DFC80:651E9185
2023-10-05T11:35:49.379+0100 [DEBUG] provider.terraform-provider-github_v5.39.0: X-Ratelimit-Limit: 60
2023-10-05T11:35:49.379+0100 [DEBUG] provider.terraform-provider-github_v5.39.0: X-Ratelimit-Remaining: 56
2023-10-05T11:35:49.379+0100 [DEBUG] provider.terraform-provider-github_v5.39.0: X-Ratelimit-Reset: 1696504862
2023-10-05T11:35:49.379+0100 [DEBUG] provider.terraform-provider-github_v5.39.0: X-Ratelimit-Resource: core
2023-10-05T11:35:49.379+0100 [DEBUG] provider.terraform-provider-github_v5.39.0: X-Ratelimit-Used: 4
2023-10-05T11:35:49.379+0100 [DEBUG] provider.terraform-provider-github_v5.39.0: X-Xss-Protection: 0
2023-10-05T11:35:49.379+0100 [DEBUG] provider.terraform-provider-github_v5.39.0: 
2023-10-05T11:35:49.379+0100 [DEBUG] provider.terraform-provider-github_v5.39.0: {
2023-10-05T11:35:49.379+0100 [DEBUG] provider.terraform-provider-github_v5.39.0:  "message": "Not Found",
2023-10-05T11:35:49.379+0100 [DEBUG] provider.terraform-provider-github_v5.39.0:  "documentation_url": "https://docs.github.com/rest"
2023-10-05T11:35:49.379+0100 [DEBUG] provider.terraform-provider-github_v5.39.0: }
2023-10-05T11:35:49.379+0100 [DEBUG] provider.terraform-provider-github_v5.39.0: -----------------------------------------------------
2023-10-05T11:35:49.379+0100 [DEBUG] provider.terraform-provider-github_v5.39.0: 2023/10/05 11:35:49 [DEBUG] Missing GitHub branch /some_public_repository (refs/heads/main)
2023-10-05T11:35:49.379+0100 [DEBUG] provider.stdio: received EOF, stopping recv loop: err="rpc error: code = Unavailable desc = error reading from server: EOF"
2023-10-05T11:35:49.380+0100 [DEBUG] provider: plugin process exited: path=.terraform/providers/registry.terraform.io/integrations/github/5.39.0/darwin_arm64/terraform-provider-github_v5.39.0 pid=16129
2023-10-05T11:35:49.380+0100 [DEBUG] provider: plugin exited

Panic Output

No response

Code of Conduct

nickfloyd commented 11 months ago

Hey @NickFitz thank you for hunting this down and apologies for the trouble. I am going to assign the labels hacktoberfest and Up for Grabs to signal to the community that this is something that has been verified and can be worked. Please feel free to pick the issue up if you have time! Thank you again for all that you do for this community ❤️ !

Nmishin commented 11 months ago

I can confirm that behaviour, will check it

evilensky commented 10 months ago

I believe I'm running into this issue with a private repository:

I look up an existing repository:

data "github_repository" "repository" {
  full_name = "org/repo"
}

to try and create a branch:

resource "github_branch" "branch" {
  repository = data.github_repository.repo.full_name
  branch     = "config-update"
}

And the URL that's produced and produces a 404:

https://api.github.com/repos/user/org/repo/git/ref/heads/main

The $user value correctly identifies the user of the token which I am using in my provider instantiation.

Stuj1 commented 10 months ago

We also hit this type of issue on a private repo, trying to generate actions environment secrets:

terraform {
  required_providers {
    github = {
      source  = "integrations/github"
      version = "~> 5.0"
    }
  }
}

provider "github" {
  # A fine-grained personal access token with organisation access
  token = var.git_repo_access_token
}

data "github_repository" "repo" {
  full_name = "myorg/{var.git_repo_access_token}"
}

resource "github_actions_environment_secret" "environment_secrets" {
  repository      = data.github_repository.repo.name
  environment     = github_repository_environment.repo_environment.environment
  secret_name     = key
  plaintext_value = value
}

The personal access token used for auth has access granted to the organisation. Applying the plan generates 404 error - the url contains the token creator's username.

Error: GET https://api.github.com/repos/USER/myorg/repo_name: 404 Not Found []

github-actions[bot] commented 1 month ago

👋 Hey Friends, this issue has been automatically marked as stale because it has no recent activity. It will be closed if no further activity occurs. Please add the Status: Pinned label if you feel that this issue needs to remain open/active. Thank you for your contributions and help in keeping things tidy!