integrations / terraform-provider-github

Terraform GitHub provider
https://www.terraform.io/docs/providers/github/
MIT License
886 stars 722 forks source link

Error when creating a new branch for an empty repository #577

Open anouarchattouna opened 3 years ago

anouarchattouna commented 3 years ago

Hello,

We are facing this error while trying to create a new branch for a new repository :

github_branch.branch: Creating...

Error: Error querying GitHub branch reference my-organization/first-repository (refs/heads/master): GET https://api.github.com/repos/my-organization/first-repository/git/refs/heads/master: 409 Git Repository is empty. []

  on main.tf line 15, in resource "github_branch" "branch":
  15: resource "github_branch" "branch" {

Terraform Version

Terraform v0.12.29
+ provider.github v2.9.2

Affected Resource(s)

Terraform Configuration Files

provider "github" {
  version = "~> 2.9"
  token        = "***"
  organization = "my-organization"
}

resource "github_repository" "repository" {
  name        = "first-repository"
  description = "This is the first repository"
  private     = true

  delete_branch_on_merge = true
}

resource "github_branch" "branch" {
  repository = github_repository.repository.name
  branch     = "main"
}

Expected Behavior

Create the repository and the branch

Actual Behavior

The branch is not created because the repository is empty

Regards

onurg commented 3 years ago

The branch is not created because the repository is not "initialized" - unfortunately we hit the same behavior but this is something coming from the github api - when the repo is created first, you need to initialize it to be able to create a branch.

it's a feature, not a bug :/

HorizonNet commented 3 years ago

It's actually the way how Git works. When a repository is created it is initialized with an empty Git repository. In an empty Git repository no commit exists in the beginning. A branch is actually a pointer to a commit, which also cannot exist as there is no commit yet. Only with the first commit to a repository you will get a branch, which is named after the value in your Git configuration.

jcudit commented 3 years ago

The auto_init parameter may be helpful for creating a branch when the repository is created. We also have a new default_branch resource that may be helpful?

anouarchattouna commented 3 years ago

I think that source_branch argument should default to main

majormoses commented 3 years ago

I think that source_branch argument should default to main

Perhaps we should remove the default or make it look that up (if not specified) since that is a per repository setting. While I think switching to match githubs new defaults are a good thing. That said because of the wide impact of all these changes many orgs have opted to still keep using "master" until all the tooling better supports it (not just terraform). Either way making any change to a default behavior should be considered a "breaking change" and only made in a major version.

jspiro commented 3 years ago

Yes, you need to look up the organization default.

Also, you have a further impact/problem. If you try to create a branch for what would be the default branch (which makes sense), then it will fail, forcing you to do an import (which is a nightmare). This should be smarter, perhaps always looking up if the branch exists or not and not holding state–just declaring the branch SHOULD exist if it doesn't.

And then an end of life problem. We now have archive_on_destroy but no equivalent for branches. So when the repo is neatly archived, all the branches will be gone (or at least all the non-default branches depending on the above implementation).

Then there's the problem of changing default branches. If you don't track your default branch, then it won't create a new one, it'll just destroy the old one. If you do track it, then it will destroy the old one and create a new one, moving the HEAD of that branch. Nasty problem. (Edge case, but nasty.)

It seems like adding the new branch and default branch resources was not thought out terribly well. They're of very limited, and largely dangerous use, when all they do is prevent an error upon creation of a new repo and branch protection of a branch not existing. That is easily solved by creating the branch and then re-applying.

jspiro commented 3 years ago

Perhaps a partially better solution would be for the default branch resource to track and create a branch, and perhaps the github API supports the renaming like it does in the UI? At the least renaming it would know which branch to rename and retain the existing SHA.

ayk33 commented 2 years ago

Can we get an update on this? This issue is a blocker.

majormoses commented 2 years ago

Can we get an update on this? This issue is a blocker.

While this bug is annoying it's not hard to work around using modules with good defaults for your org/team. Defaults are hard because people have different needs, you can make a decision for your team/org much easier than we can as a community.

Even outside of this issue I highly suggest taking the time to build good modules for your github org. For scale, maintenance, and for protection from the edges of the apis+providers.

berserker5000 commented 2 years ago

Hi there, any update on this?

ghost commented 2 years ago

The auto-initoption in the resource "github_repository" does not solve the problem.

resource "github_repository" "example" {
  name        = "demo-repo"
  description = "My awesome codebase"
  auto_init   = true
  visibility = "private"

}

resource "github_branch" "dev" {
  repository = github_repository.example.name
  branch     = "dev"
  depends_on = [
    github_repository.example
  ]
}

still gives:

Error: Error querying GitHub branch reference / (refs/heads/main): GET https://api.github.com/repos///git/ref/heads/main: 409 Git Repository is empty. [] │ │ with github_branch.dev, │ on sample.tf line 57, in resource "github_branch" "dev": │ 57: resource "github_branch" "dev" {

nukezzzz commented 2 years ago

Hi got the same issue , Added source_branch = organization_default_branch to resource "github_branch" , hope it helps !

seanorama commented 1 year ago

None of the above workarounds have worked for us.

github_branch.default: Creating...
╷
│ Error: error querying GitHub branch reference myorg/github (refs/heads/main): GET https://github.ourdomain.tld/api/v3/repos/myorg/github/git/ref/heads/main: 409 Git Repository is empty. []
│
│   with github_branch.default,
│   on main.tf line 55, in resource "github_branch" "default":
│   55: resource "github_branch" "default" {

Simplified without variables:

terraform {
  required_providers {
    github = {
      source  = "integrations/github"
      version = "~> 5.17"
    }
  }
}

provider "github" {
  owner    = var.github_owner
  base_url = var.github_base_url
}

resource "github_repository" "default" {
  name        = "github"
  description = "description here"

  auto_init              = true
  visibility             = "public"
}

resource "github_branch" "default" {
  repository    = "github"
  branch        = "master"
  source_branch = "main"
  depends_on = [
    github_repository.default
  ]
}
GitHub Enterprise Server 3.6.3
spkane commented 1 year ago

I have found that the compromise for creating brand-new repos looks like this:

locals {
  default_branch = "main"
}

resource "github_repository" "test" {
  name                   = "test"
  description            = "test"
  visibility             = "private"
  // Creates the 'main' branch with a small 'README.md'
  auto_init              = true
  has_issues             = true
  has_downloads          = true
  vulnerability_alerts   = true
  allow_merge_commit     = false
  delete_branch_on_merge = true
}

// auto-init in the repo gets us this branch already.
// We could import this, but having this in here will breaks the run and
// it doesn't really do anything super useful at this point.
//
//resource "github_branch" "test_default" {
//  repository = github_repository.test.name
//  branch     = "main"
//}

resource "github_branch_default" "test_default" {
  repository = github_repository.test.name
  branch     = local.default_branch
}

resource "github_branch_protection" "test" {
  repository_id           = github_repository.test.node_id
  pattern                 = local.default_branch
  push_restrictions       = []
  required_linear_history = true
  required_pull_request_reviews {
    dismiss_stale_reviews           = true
    dismissal_restrictions          = []
    require_code_owner_reviews      = false
    required_approving_review_count = 1
    restrict_dismissals             = false
  }
}

resource "github_team_repository" "test" {
  repository = github_repository.test.name
  team_id    = github_team.All.id
  permission = "push"
}

The main thing is to use auto_init = true for the repo creation and then avoid using github_branch for the main branch.

github-actions[bot] commented 9 months ago

👋 Hey Friends, this issue has been automatically marked as stale because it has no recent activity. It will be closed if no further activity occurs. Please add the Status: Pinned label if you feel that this issue needs to remain open/active. Thank you for your contributions and help in keeping things tidy!

mering commented 9 months ago

Still an issue.

ernestyouniverse commented 8 months ago

It's still an issue. Please tell me if the following is lunacy.:

So I have a variable with a map, like this format, called repositories:

{
    my-repository-name: {
        name: 'my-repository-name',
        default_branch_name    = "<<SOME STRING>>"
        ...
    },
    ...
}

i go through the var.repositories like this for the first step:

resource "github_repository" "repository" {
    for_each = var.repositories
    ...

Than as the very next step i go through each repositories for github_branch, excluding those, whose default_branch_name is an empty string.

resource "github_branch" "repo_branch" {
  for_each = { for repository_name, repository_config in var.repositories : repository_name => repository_config if repository_config.default_branch_name != "" }

  repository = each.value.name
  branch     = each.value.default_branch_name
}

resource "github_branch_default" "repo_default_branch" {
  for_each = { for repository_name, repository_config in var.repositories : repository_name => repository_config if repository_config.default_branch_name != "" }

  repository = each.value.name
  branch     = github_branch.repo_branch[each.key].branch
}

This way when I add a new repository, i don't add a default branch. It creates an empty repository with all the setups. Than I initialize my repository, main branch gets inited. Than I add any sort of default branch into the repository (i prefer development as default), run it, than manually go to the created repository and delete the "main" branch.

It's really not too cool tho... doing it until there isn't some real solution is worked out.

shyam2180 commented 8 months ago

Hi friends, Give this a try; it worked for me when creating a new branch for an empty repository

terraform { required_providers { github = { source = "hashicorp/github" version = "5.42.0" } } }

provider "github" { token = "Your_Github_Token" }

locals { default_branch = "main" }

resource "github_repository" "repository" { name = "Your_name" description = "automation" visibility = "private" auto_init = true }

resource "time_sleep" "wait_for_repository" { depends_on = [github_repository.repository]

create_duration = "30s" # Adjust the duration as needed }

resource "github_branch" "default" { repository = github_repository.repository.name branch = local.default_branch source_branch = "main" depends_on = [time_sleep.wait_for_repository] }

resource "github_branch" "production" { repository = github_repository.repository.name branch = "prod" depends_on = [time_sleep.wait_for_repository] }

resource "github_branch" "development" { repository = github_repository.repository.name branch = "dev" depends_on = [time_sleep.wait_for_repository] }

resource "github_branch" "testing" { repository = github_repository.repository.name branch = "uat" depends_on = [time_sleep.wait_for_repository] }

mering commented 8 months ago

@shyam2180 The important part is auto_init = true as noted in https://github.com/integrations/terraform-provider-github/issues/577#issuecomment-1423037653.

rorylogue commented 7 months ago

To clarify, there isn't a fix/workaround for when a repository is already created and I want to create a new branch main? (with master as default branch already)

error querying GitHub branch reference 404 Not Found [] - Same errors as above.

spkane commented 7 months ago

To clarify, there isn't a fix/workaround for when a repository is already created and I want to create a new branch main? (with master as default branch already)

error querying GitHub branch reference 404 Not Found [] - Same errors as above.

You should be able to create a new branch on an existing repo. This error looks a bit like you are trying to reference the branch before creating it.

This issue is very specifically about the fact that you can not create a new branch before the first commit has been made to the initial default branch (manually or via auto_init = true)

moltar commented 6 months ago

I think there's possibly a race condition between auto init option and default branch resources.

When I deploy repo with init flag set, and default branch, the deployment fails with:

RepositoryInit  github_repository.test_Repository_321889EA: Creation complete after 7s [id=test]
RepositoryInit  ╷
                │ Error: PATCH https://api.github.com/repos/org/test: 404 Not Found []
                │ 
                │   with github_branch_default.test_Repository_DefaultBranch_11BBFCBE (test/Repository/DefaultBranch),
                │   on cdk.tf.json line 31, in resource.github_branch_default.test_Repository_DefaultBranch_11BBFCBE (test/Repository/DefaultBranch):
                │   31:       }

However, if I comment out the default branch, deploy, then uncomment the default branch and deploy again, then the deployment succeeds.

moltar commented 6 months ago

Setting explicit dependency on the repo seems to work 👍🏼

CDKTF code:

defaultBranch.dependsOn = [ dependable(repo) ]
webstean commented 3 months ago

This is definitely still an issue - I suspect a race condition as per https://github.com/integrations/terraform-provider-github/issues/577#issuecomment-1932812567

resource "github_repository" "github-working" {
  for_each = local.gitrepo
  provider = github.individual

  name        = each.value.name
  description = each.value.description

  auto_init              = true
  is_template            = false
  visibility             = "public"
}

resource "time_sleep" "github_create_wait_30_seconds" {
  create_duration = "30s"
  depends_on = [github_repository.github-working]
}

resource "github_branch" "github-working" {
  for_each = github_repository.github-working
  provider = github.individual

  repository = each.value.name
  branch     = "main"
  depends_on = [time_sleep.github_create_wait_30_seconds, github_repository.github-working]
}

resource "github_branch_default" "github-working" {
  for_each = github_repository.github-working
  provider = github.individual

  repository = each.value.name
  branch     = "main"
  depends_on = [github_branch.github-working]
}

Results in:

│ Error: error querying GitHub branch reference webstean/github-codespaces-dotfiles (refs/heads/main): GET https://api.github.com/repos/webstean/github-codespaces-dotfiles/git/ref/heads/main: 409 Git Repository is empty. []
│ 
│   with github_branch.github-working["dotfiles"],
│   on github-repo.tf line 66, in resource "github_branch" "github-working":
│   66: resource "github_branch" "github-working" {
│