Open kunalnanda opened 3 years ago
I'm facing this but only when trying to create files in nested directories.
For example this works: ./circle/config.yaml
And this fails: ./github/workflows/build.yaml
I can only create 1 directory deep.
EDIT: Fixed my issue, in my case it's because to update ./github/worklows
you need the workflows
scope on your token. Had nothing to do with nested directory.
Can we get more detailed debug logging posted to the issue? Adding TF_LOG=DEBUG
will help us determine which request is causing the 404.
@jcudit Here is the debug log:
2021-09-29T11:24:49.630+1000 [DEBUG] Adding temp file log sink: /var/folders/q_/bp_xl53j0ms3k5y0b7d_c8g40000gn/T/terraform-log994684574
2021-09-29T11:24:49.631+1000 [INFO] Terraform version: 1.0.5
2021-09-29T11:24:49.631+1000 [INFO] Go runtime version: go1.16.4
2021-09-29T11:24:49.631+1000 [INFO] CLI args: []string{"/usr/local/Cellar/tfenv/2.2.2/versions/1.0.5/terraform", "apply", "--auto-approve"}
2021-09-29T11:24:49.631+1000 [DEBUG] Attempting to open CLI config file: /Users/blah/.terraformrc
2021-09-29T11:24:49.631+1000 [INFO] Loading CLI configuration from /Users/blah/.terraformrc
2021-09-29T11:24:49.634+1000 [INFO] Loading CLI configuration from /Users/blah/.terraform.d/credentials.tfrc.json
2021-09-29T11:24:49.634+1000 [DEBUG] ignoring non-existing provider search directory terraform.d/plugins
2021-09-29T11:24:49.635+1000 [DEBUG] ignoring non-existing provider search directory /Users/blah/.terraform.d/plugins
2021-09-29T11:24:49.635+1000 [DEBUG] ignoring non-existing provider search directory /Users/blah/Library/Application Support/io.terraform/plugins
2021-09-29T11:24:49.635+1000 [DEBUG] ignoring non-existing provider search directory /Library/Application Support/io.terraform/plugins
2021-09-29T11:24:49.636+1000 [INFO] CLI command args: []string{"apply", "--auto-approve"}
2021-09-29T11:24:49.652+1000 [DEBUG] Service discovery for app.terraform.io at https://app.terraform.io/.well-known/terraform.json
2021-09-29T11:24:50.943+1000 [DEBUG] Retrieve version constraints for service tfe.v2.1 and product terraform
2021-09-29T11:24:53.116+1000 [DEBUG] checking for provisioner in "."
2021-09-29T11:24:53.116+1000 [DEBUG] checking for provisioner in "/usr/local/Cellar/tfenv/2.2.2/versions/1.0.5"
2021-09-29T11:24:53.117+1000 [INFO] Failed to read plugin lock file .terraform/plugins/darwin_amd64/lock.json: open .terraform/plugins/darwin_amd64/lock.json: no such file or directory
2021-09-29T11:24:53.401+1000 [INFO] backend/remote: starting Apply operation
I was also facing this issue, adding auto_init = true
to the github_repository
fixed it for me. I did have to manually delete the repo and reapply the terraform config to get it to work.
The issue seems to have been that the repository was empty and had no branches. Adding auto_init = true
created an initial commit with empty README (docs here), after which terraform apply
was able to successfully create the files in the main branch. I didn't see it directly, but I would imagine that the underlying API call to create the files contains the branch name in the path, which would explain the 404 response received when the branch did not exist.
I also used the github_branch
data source to access the main
branch that was created by setting auto_init = true
and replaced any literal "main"
with data.github_branch.main.branch
.
Seeing the same thing with the following TF
resource "github_repository_file" "test" {
repository = "RentTheRunway/bell_labs"
branch = "master"
file = ".testfile"
content = "**/*.tfstate"
}
Produces the following debug logs
2021/11/02 17:25:45 [DEBUG] EvalApply: ProviderMeta config value set
2021/11/02 17:25:45 [DEBUG] module.repositories.github_repository_file.test: applying the planned Create change
2021-11-02T17:25:45.445-0400 [DEBUG] plugin.terraform-provider-github_v4.17.0: 2021/11/02 17:25:45 [DEBUG] Github API Request Details:
2021-11-02T17:25:45.445-0400 [DEBUG] plugin.terraform-provider-github_v4.17.0: ---[ REQUEST ]---------------------------------------
2021-11-02T17:25:45.445-0400 [DEBUG] plugin.terraform-provider-github_v4.17.0: GET /repos//RentTheRunway/bell_labs/branches/master HTTP/1.1
2021-11-02T17:25:45.445-0400 [DEBUG] plugin.terraform-provider-github_v4.17.0: Host: api.github.com
2021-11-02T17:25:45.445-0400 [DEBUG] plugin.terraform-provider-github_v4.17.0: User-Agent: go-github
2021-11-02T17:25:45.445-0400 [DEBUG] plugin.terraform-provider-github_v4.17.0: Accept: application/vnd.github.v3+json
2021-11-02T17:25:45.445-0400 [DEBUG] plugin.terraform-provider-github_v4.17.0: Accept-Encoding: gzip
2021-11-02T17:25:45.445-0400 [DEBUG] plugin.terraform-provider-github_v4.17.0:
2021-11-02T17:25:45.445-0400 [DEBUG] plugin.terraform-provider-github_v4.17.0:
2021-11-02T17:25:45.445-0400 [DEBUG] plugin.terraform-provider-github_v4.17.0: -----------------------------------------------------
2021-11-02T17:25:45.445-0400 [DEBUG] plugin.terraform-provider-github_v4.17.0: 2021/11/02 17:25:45 [TRACE] Acquiring lock for GitHub API request ("RentTheRunway/bell_labs:master")
2021-11-02T17:25:45.755-0400 [DEBUG] plugin.terraform-provider-github_v4.17.0: 2021/11/02 17:25:45 [TRACE] Releasing lock for GitHub API request ("RentTheRunway/bell_labs:master")
2021-11-02T17:25:45.756-0400 [DEBUG] plugin.terraform-provider-github_v4.17.0: 2021/11/02 17:25:45 [DEBUG] Github API Response Details:
2021-11-02T17:25:45.756-0400 [DEBUG] plugin.terraform-provider-github_v4.17.0: ---[ RESPONSE ]--------------------------------------
2021-11-02T17:25:45.756-0400 [DEBUG] plugin.terraform-provider-github_v4.17.0: HTTP/2.0 404 Not Found
2021-11-02T17:25:45.756-0400 [DEBUG] plugin.terraform-provider-github_v4.17.0: Access-Control-Allow-Origin: *
2021-11-02T17:25:45.756-0400 [DEBUG] plugin.terraform-provider-github_v4.17.0: Access-Control-Expose-Headers: ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, Deprecation, Sunset
2021-11-02T17:25:45.756-0400 [DEBUG] plugin.terraform-provider-github_v4.17.0: Content-Security-Policy: default-src 'none'
2021-11-02T17:25:45.756-0400 [DEBUG] plugin.terraform-provider-github_v4.17.0: Content-Type: application/json; charset=utf-8
2021-11-02T17:25:45.756-0400 [DEBUG] plugin.terraform-provider-github_v4.17.0: Date: Tue, 02 Nov 2021 21:25:45 GMT
2021-11-02T17:25:45.756-0400 [DEBUG] plugin.terraform-provider-github_v4.17.0: Referrer-Policy: origin-when-cross-origin, strict-origin-when-cross-origin
2021-11-02T17:25:45.756-0400 [DEBUG] plugin.terraform-provider-github_v4.17.0: Server: GitHub.com
2021-11-02T17:25:45.756-0400 [DEBUG] plugin.terraform-provider-github_v4.17.0: Strict-Transport-Security: max-age=31536000; includeSubdomains; preload
2021-11-02T17:25:45.756-0400 [DEBUG] plugin.terraform-provider-github_v4.17.0: Vary: Accept-Encoding, Accept, X-Requested-With
2021-11-02T17:25:45.756-0400 [DEBUG] plugin.terraform-provider-github_v4.17.0: X-Content-Type-Options: nosniff
2021-11-02T17:25:45.756-0400 [DEBUG] plugin.terraform-provider-github_v4.17.0: X-Frame-Options: deny
2021-11-02T17:25:45.756-0400 [DEBUG] plugin.terraform-provider-github_v4.17.0: X-Github-Media-Type: github.v3; format=json
2021-11-02T17:25:45.756-0400 [DEBUG] plugin.terraform-provider-github_v4.17.0: X-Github-Request-Id: E962:0E29:24C546:471F11:6181ACD9
2021-11-02T17:25:45.756-0400 [DEBUG] plugin.terraform-provider-github_v4.17.0: X-Ratelimit-Limit: 60
2021-11-02T17:25:45.756-0400 [DEBUG] plugin.terraform-provider-github_v4.17.0: X-Ratelimit-Remaining: 49
2021-11-02T17:25:45.756-0400 [DEBUG] plugin.terraform-provider-github_v4.17.0: X-Ratelimit-Reset: 1635890283
2021-11-02T17:25:45.756-0400 [DEBUG] plugin.terraform-provider-github_v4.17.0: X-Ratelimit-Resource: core
2021-11-02T17:25:45.756-0400 [DEBUG] plugin.terraform-provider-github_v4.17.0: X-Ratelimit-Used: 11
2021-11-02T17:25:45.756-0400 [DEBUG] plugin.terraform-provider-github_v4.17.0: X-Xss-Protection: 0
2021-11-02T17:25:45.756-0400 [DEBUG] plugin.terraform-provider-github_v4.17.0:
2021-11-02T17:25:45.756-0400 [DEBUG] plugin.terraform-provider-github_v4.17.0: {
2021-11-02T17:25:45.756-0400 [DEBUG] plugin.terraform-provider-github_v4.17.0: "message": "Not Found",
2021-11-02T17:25:45.756-0400 [DEBUG] plugin.terraform-provider-github_v4.17.0: "documentation_url": "https://docs.github.com/rest"
UG] plugin.terraform-provider-github_v4.17.0: }
2021-11-02T17:25:45.756-0400 [DEBUG] plugin.terraform-provider-github_v4.17.0: -----------------------------------------------------
2021/11/02 17:25:45 [DEBUG] module.repositories.github_repository_file.test: apply errored, but we're indicating that via the Error pointer rather than returning it: unexpected status code: 404 Not Found
2021/11/02 17:25:45 [ERROR] eval: *terraform.EvalApplyPost, err: unexpected status code: 404 Not Found
2021/11/02 17:25:45 [ERROR] eval: *terraform.EvalSequence, err: unexpected status code: 404 Not Found
it has something to do with the set PAT. If i dont set it, url is malformed like that GET /repos//RentTheRunway/bell_labs/branches/master HTTP/1.1
if i set it it fills the username /repos/lerljaku/RentTheRunway/bell_labs/branches/master HTTP/1.1 but it doesnt work if the repo owner is organization instead of user
EDIT:
when passing filling owner to github provider, it works then
provider "github" {
token = var.gh_pat
owner = var.gh_org
}
I actually had this error because in my state I have the resource pointing to a branch that doesn't exist anymore, even if the new one I specify was there. I solved deleting the resource from the state and then re-apply.
I ran into this error and discovered the cause in my case was that there are two Github providers: hashicorp/github
and integrations/github
. The integrations/github
provider works fine, the hashicorp/github
provider throws 404 errors. So if your module doesn't explicitly require integrations/github
, it'll use the hashicorp provider and blow up.
Lost 3 hours of my life to this and the solution was to provide owner
either explicitly in the provider or through GITHUB_OWNER
😭
Documentation is misleading mentioning that owner
is optional.
This problem happens when there's a mismatch between both implementations (integrations and hashicorp).
How to reproduce:
integrations/terraform-provider-github
with owner property different from the current userintegrations/terraform-provider-github
(so it will use the hashicorp/terraform-provider-github
by defalt)Expected behaviour:
Current behaviour:
github_repository_file
from a module don't see the integrations/terraform-provider-github
owner and try to create the file in the current user repositoryI have the same problem. My repo was empty, so i recreate it with non empty readme file and the problem has gone.
In my case,
So what I ended up doing was log in to my postgres database and manually update the state JSON. Since I had successfully been able to use github_repository_file
resource with other repositories, I copied a good value and tweaked it for the repository that wasn't working. Then, I deleted my local terraform state (rm -f .terraform/terraform.tfstate
) and reinitialized from my backend (terraform init --backend-config=${CONN_STR}
).
Here's the JSON snippet I added to my state:
{
"version": 4,
"terraform_version": "1.1.9",
"resources": [
{
"mode": "managed",
"type": "github_repository_file",
"name": "readme",
"provider": "provider[\"registry.terraform.io/integrations/github\"]",
"instances": [
{
"index_key": "my-awesome-repo",
"schema_version": 0,
"attributes": {
"branch": "main",
"commit_author": "my-git-username",
"commit_email": "me@example.com",
"commit_message": "Created by Terraform",
"commit_sha": "af1d88db0dc7a616eca89d88ef25235de7cfe41e",
"content": "my-awesome-repo",
"file": "README.md",
"id": "my-awesome-repo/README.md",
"overwrite_on_create": false,
"repository": "my-awesome-repo",
"sha": "2aae6c35c94fcfb415dbe95f408b9ce91ee846ed"
},
"sensitive_attributes": [],
"private": "cyQstM==",
"dependencies": [
"github_repository.my-awesome-repo"
]
}
]
}
]
}
Note, commit_sha
should be the commit that represents the file content, usually the most recent commit to the file in GitHub.
Note, sha
is the SHA1 hash of the file contents. To get the known value in GitHub, use the following curl command against the API:
GITHUB_TOKEN="ghp_your_token"
OWNER="your-repo-owner"
REPO="your-repo-name"
COMMIT="commit-hash-of-your-file"
curl -X GET -H "Authorization: token ${GITHUB_TOKEN}" "https://api.github.com/repos/${OWNER}/${REPO}/git/blobs/${COMMIT}"
The JSON response includes the sha
value.
I have the same issue, have checked all the above fixes/work arounds and nothing seems to work.
I am trying to create 20+ files and some of them get created but the remaining end up with the same error
unexpected status code: 404 Not Found
But if I apply the changes again then the remaining files get created. I have also checked the API rate limit and it looks good.
I am using the following config cdktf 0.13.1 terraform v1.3.2 github provider -> registry.terraform.io/integrations/github 4.31.0
@shubham030 can you share a small snippet of HCL that reproduces the issue I can try?
For me, I get the error when trying to import the file into state. The file was previously created by terraform, then deleted from state. Now, I get 404
when trying to re-import.
EDIT: turns out I needed to prepend the repo-name/
for the import id (eg: path/to/file:branch
).
I have tried all provided workarounds above and the only one that has gotten me to a better place is manually creating the file outside of terraform and allowing it to overwrite it on next apply.
Terraform version: 1.4.5
terraform {
required_providers {
github = {
source = "integrations/github"
version = "5.3.0"
}
}
}
This morning I created a new Personal Access Token and verified the problem is no longer present after granting it full permissions. There seems to be some permissions issue related to creating files in a repo for the first time, versus updating them in place.
I also verified my projects have branch protections in place - so perhaps it's less related to workflow permissions and more related to admin permissions to bypass these.
Hello, i am facing the same problem, i change the permissions of the token to full permissions but didn't work
I had this issue and realized the cryptic 404 on one repo despite having applied this same resource in a module to all my other github repos for a year was because that one repo only had the master
branch whereas others had the main
branch.
The 404 hint actually meant that the branch specified in the github_repository_file
resource was main
which didn't exist in that repo.
I solved it by replacing an explicit branch name of main
with data.github_repository.default_branch
which dynamically determines the default branch name.
You can see the real code example here as I deploy my CODEOWNERS file like this:
https://github.com/HariSekhon/Terraform/blob/master/github_repo/codeowners.tf#L27
and the data source chained from the repo creation to generate proper implicit depends_on:
https://github.com/HariSekhon/Terraform/blob/master/github_repo/repo.tf#L67
For me the issue was related to token permissions. I didn't narrow down the exact permissions, but trying just repo create and delete was not enough. I gave it full repo control too.
The boiler plate code works with it just fine.
I ended up with code that referred to multiple examples in this thread:
terraform {
backend "local" {}
required_providers {
github = {
source = "integrations/github" #integrations/terraform-provider-github
version = "~> 5.0"
}
}
}
provider "github" {
token = "${var.gh_access_token}"
owner = "${var.gh_owner}"
}
Still facing this issue
In my case something weird happened, I have a fine-grained PAT with content write permission for the target repo which has worked so far without issues, however when I tried to use both the github_repository_file
data source and resource the Error: unexpected status code: 404 Not Found
was returned, after trying must of the things mentioned above without success I decided to remove all the permissions from the PAT and then started added them again, the weird thing is that the permission combination that worked was properties and content write permissions, this is weird so I then remove the properties permission and guess what? it worked, so it work with the original permission I had at the beginning but I had to disable the same permissions, after this the issues didn't show up again.
Terraform Version
Affected Resource(s)
Terraform Configuration Files
Expected Behavior
The repositories should be created, and a default .gitignore file should be created with contents from the given template file.
Actual Behavior
The repositories are created. The .gitignore file fails with the following error:
This error is repeated 27 times, one for each repository.
Steps to Reproduce
Please list the steps required to reproduce the issue, for example:
terraform apply
Important Factoids
I am running this against my personal Github account. My Github token has the following permissions: