[BUG]: inconsistent state after 422 Unprocessable Entity

[matthiasr]

Expected Behavior

The Terraform state matches the API state, or differences are seen on refresh.

Actual Behavior

While investigating what turned out to be #2145, I got my Terraform state into a situation where it thinks the github_repository is in the desired state, but it is in fact not. It seems that after receiving a 422, the state combined the current etag with the desired state of the field I was toggling. Subsequent refreshes did not see the issue, because the etag matches the current status, and the refresh gets a 304 Not Modified.

Terraform Version

❯ terraform -v Terraform v1.6.1 on darwin_arm64

• provider registry.terraform.io/integrations/github v5.45.0

Affected Resource(s)

• github_repository

Terraform Configuration Files

terraform {
  required_providers {
    github = {
      source = "integrations/github"
    }
  }
}

provider "github" {}

resource "github_repository" "repository" {
  name        = "import_test"
  archived    = false
  description = "testing import"
  has_issues  = true
  has_wiki    = true

  # Added this as a possible workaround for #2145 but it did not make a difference. The trace shows that these values are included in the PATCH request either way.
  security_and_analysis {
    secret_scanning {
      status = "disabled"
    }
    secret_scanning_push_protection {
      status = "disabled"
    }
  }
}

import {
  to = github_repository.repository
  id = "import_test"
}

Steps to Reproduce

Toggle between has_wiki = true and has_wiki = false, with terraform apply -auto-approve. Eventually the plan becomes "No changes. Your infrastructure matches the configuration." but checking the GitHub API, the value is not what it should be (true in my case).

In terraform state show, it shows true, in gh api repos/$owner/$repo, false. However, the etag in the state matches the etag from a previous refresh when it was false, according to Terraform trace output:

Terraform and API state

``` ❯ gh api repos/matthiasr/import_test { "id": 757470430, "node_id": "R_kgDOLSYU3g", "name": "import_test", "full_name": "matthiasr/import_test", "private": true, "owner": { "login": "matthiasr", "id": 131590, "node_id": "MDQ6VXNlcjEzMTU5MA==", "avatar_url": "https://avatars.githubusercontent.com/u/131590?v=4", "gravatar_id": "", "url": "https://api.github.com/users/matthiasr", "html_url": "https://github.com/matthiasr", "followers_url": "https://api.github.com/users/matthiasr/followers", "following_url": "https://api.github.com/users/matthiasr/following{/other_user}", "gists_url": "https://api.github.com/users/matthiasr/gists{/gist_id}", "starred_url": "https://api.github.com/users/matthiasr/starred{/owner}{/repo}", "subscriptions_url": "https://api.github.com/users/matthiasr/subscriptions", "organizations_url": "https://api.github.com/users/matthiasr/orgs", "repos_url": "https://api.github.com/users/matthiasr/repos", "events_url": "https://api.github.com/users/matthiasr/events{/privacy}", "received_events_url": "https://api.github.com/users/matthiasr/received_events", "type": "User", "site_admin": false }, "html_url": "https://github.com/matthiasr/import_test", "description": "testing import", "fork": false, "url": "https://api.github.com/repos/matthiasr/import_test", "forks_url": "https://api.github.com/repos/matthiasr/import_test/forks", "keys_url": "https://api.github.com/repos/matthiasr/import_test/keys{/key_id}", "collaborators_url": "https://api.github.com/repos/matthiasr/import_test/collaborators{/collaborator}", "teams_url": "https://api.github.com/repos/matthiasr/import_test/teams", "hooks_url": "https://api.github.com/repos/matthiasr/import_test/hooks", "issue_events_url": "https://api.github.com/repos/matthiasr/import_test/issues/events{/number}", "events_url": "https://api.github.com/repos/matthiasr/import_test/events", "assignees_url": "https://api.github.com/repos/matthiasr/import_test/assignees{/user}", "branches_url": "https://api.github.com/repos/matthiasr/import_test/branches{/branch}", "tags_url": "https://api.github.com/repos/matthiasr/import_test/tags", "blobs_url": "https://api.github.com/repos/matthiasr/import_test/git/blobs{/sha}", "git_tags_url": "https://api.github.com/repos/matthiasr/import_test/git/tags{/sha}", "git_refs_url": "https://api.github.com/repos/matthiasr/import_test/git/refs{/sha}", "trees_url": "https://api.github.com/repos/matthiasr/import_test/git/trees{/sha}", "statuses_url": "https://api.github.com/repos/matthiasr/import_test/statuses/{sha}", "languages_url": "https://api.github.com/repos/matthiasr/import_test/languages", "stargazers_url": "https://api.github.com/repos/matthiasr/import_test/stargazers", "contributors_url": "https://api.github.com/repos/matthiasr/import_test/contributors", "subscribers_url": "https://api.github.com/repos/matthiasr/import_test/subscribers", "subscription_url": "https://api.github.com/repos/matthiasr/import_test/subscription", "commits_url": "https://api.github.com/repos/matthiasr/import_test/commits{/sha}", "git_commits_url": "https://api.github.com/repos/matthiasr/import_test/git/commits{/sha}", "comments_url": "https://api.github.com/repos/matthiasr/import_test/comments{/number}", "issue_comment_url": "https://api.github.com/repos/matthiasr/import_test/issues/comments{/number}", "contents_url": "https://api.github.com/repos/matthiasr/import_test/contents/{+path}", "compare_url": "https://api.github.com/repos/matthiasr/import_test/compare/{base}...{head}", "merges_url": "https://api.github.com/repos/matthiasr/import_test/merges", "archive_url": "https://api.github.com/repos/matthiasr/import_test/{archive_format}{/ref}", "downloads_url": "https://api.github.com/repos/matthiasr/import_test/downloads", "issues_url": "https://api.github.com/repos/matthiasr/import_test/issues{/number}", "pulls_url": "https://api.github.com/repos/matthiasr/import_test/pulls{/number}", "milestones_url": "https://api.github.com/repos/matthiasr/import_test/milestones{/number}", "notifications_url": "https://api.github.com/repos/matthiasr/import_test/notifications{?since,all,participating}", "labels_url": "https://api.github.com/repos/matthiasr/import_test/labels{/name}", "releases_url": "https://api.github.com/repos/matthiasr/import_test/releases{/id}", "deployments_url": "https://api.github.com/repos/matthiasr/import_test/deployments", "created_at": "2024-02-14T15:04:42Z", "updated_at": "2024-02-14T16:45:28Z", "pushed_at": "2024-02-14T15:04:42Z", "git_url": "git://github.com/matthiasr/import_test.git", "ssh_url": "git@github.com:matthiasr/import_test.git", "clone_url": "https://github.com/matthiasr/import_test.git", "svn_url": "https://github.com/matthiasr/import_test", "homepage": "", "size": 0, "stargazers_count": 0, "watchers_count": 0, "language": null, "has_issues": true, "has_projects": false, "has_downloads": false, "has_wiki": false, "has_pages": false, "has_discussions": false, "forks_count": 0, "mirror_url": null, "archived": false, "disabled": false, "open_issues_count": 0, "license": null, "allow_forking": true, "is_template": false, "web_commit_signoff_required": false, "topics": [], "visibility": "private", "forks": 0, "open_issues": 0, "watchers": 0, "default_branch": "main", "permissions": { "admin": true, "maintain": true, "push": true, "triage": true, "pull": true }, "temp_clone_token": "REDACTED", "allow_squash_merge": true, "allow_merge_commit": true, "allow_rebase_merge": true, "allow_auto_merge": false, "delete_branch_on_merge": false, "allow_update_branch": false, "use_squash_pr_title_as_default": false, "squash_merge_commit_message": "COMMIT_MESSAGES", "squash_merge_commit_title": "COMMIT_OR_PR_TITLE", "merge_commit_message": "PR_TITLE", "merge_commit_title": "MERGE_MESSAGE", "security_and_analysis": { "secret_scanning": { "status": "disabled" }, "secret_scanning_push_protection": { "status": "disabled" }, "dependabot_security_updates": { "status": "disabled" }, "secret_scanning_validity_checks": { "status": "disabled" } }, "network_count": 0, "subscribers_count": 1 } ❯ cd ../../environments/import_test ❯ fc -R ❯ terraform state show github_repository.repository # github_repository.repository: resource "github_repository" "repository" { allow_auto_merge = false allow_merge_commit = true allow_rebase_merge = true allow_squash_merge = true allow_update_branch = false archived = false auto_init = false default_branch = "main" delete_branch_on_merge = false description = "testing import" etag = "W/\"f1c0febe2576d83ae62f1446de4b8724734dd0df6c1df07bfccaca6b7c0fc8fa\"" full_name = "matthiasr/import_test" git_clone_url = "git://github.com/matthiasr/import_test.git" has_discussions = false has_downloads = false has_issues = true has_projects = false has_wiki = true html_url = "https://github.com/matthiasr/import_test" http_clone_url = "https://github.com/matthiasr/import_test.git" id = "import_test" is_template = false merge_commit_message = "PR_TITLE" merge_commit_title = "MERGE_MESSAGE" name = "import_test" node_id = "R_kgDOLSYU3g" private = true repo_id = 757470430 squash_merge_commit_message = "COMMIT_MESSAGES" squash_merge_commit_title = "COMMIT_OR_PR_TITLE" ssh_clone_url = "git@github.com:matthiasr/import_test.git" svn_url = "https://github.com/matthiasr/import_test" topics = [] visibility = "private" vulnerability_alerts = false web_commit_signoff_required = false security_and_analysis { secret_scanning { status = "disabled" } secret_scanning_push_protection { status = "disabled" } } } ```

As far as I can tell, the sequence of events is

1. Refresh, get new etag (f1c0febe2576d83ae62f1446de4b8724734dd0df6c1df07bfccaca6b7c0fc8fa) and has_wiki value (false)
2. Attempt to PATCH, including security_and_analysis fields
3. Rejected with 422
4. Save updated state with etag=f1c0… and has_wiki=true
5. On next refresh, use etag from state
6. Get 304 Not Modified
7. Conclude that there is nothing to do

Debug Output

Trace log, too long to put inline:

https://gist.github.com/matthiasr/14fd71acf3a041d319c4fb1abc2c8125

Panic Output

No response

Code of Conduct

• [X] I agree to follow this project's Code of Conduct

[matthiasr]

Bubbling up from the trace log because it looks suspect, but I don't know how to interpret it:

2024-02-14T16:46:08.727Z [WARN]  Provider "registry.terraform.io/integrations/github" produced an unexpected new value for github_repository.repository during refresh.
      - .has_wiki: was cty.True, but now cty.False
      - .etag: was cty.StringVal("W/\"ca9d39c274f2ff56598c64f2b3580dcac520b94892c8693f7ff752ee0ecd4595\""), but now cty.StringVal("W/\"f1c0febe2576d83ae62f1446de4b8724734dd0df6c1df07bfccaca6b7c0fc8fa\"")

[matthiasr]

More curiosities: The backend behavior is inconsistent between fields:

• For has_wiki, on 422, the update does not happen in the API, but is recorded in the state.
• For archived, the repository does get archived, even though the API request returns a 422. This is also recorded in the state.

I updated the gist with a trace log from the latter. Note how the PATCH includes both "archived": true and "has_wiki": true. Afterwards, from the API, I get "has_wiki": false, and "archived": true,:

API and terraform state after archiving

``` ❯ gh api repos/matthiasr/import_test { "id": 757470430, "node_id": "R_kgDOLSYU3g", "name": "import_test", "full_name": "matthiasr/import_test", "private": true, "owner": { "login": "matthiasr", "id": 131590, "node_id": "MDQ6VXNlcjEzMTU5MA==", "avatar_url": "https://avatars.githubusercontent.com/u/131590?v=4", "gravatar_id": "", "url": "https://api.github.com/users/matthiasr", "html_url": "https://github.com/matthiasr", "followers_url": "https://api.github.com/users/matthiasr/followers", "following_url": "https://api.github.com/users/matthiasr/following{/other_user}", "gists_url": "https://api.github.com/users/matthiasr/gists{/gist_id}", "starred_url": "https://api.github.com/users/matthiasr/starred{/owner}{/repo}", "subscriptions_url": "https://api.github.com/users/matthiasr/subscriptions", "organizations_url": "https://api.github.com/users/matthiasr/orgs", "repos_url": "https://api.github.com/users/matthiasr/repos", "events_url": "https://api.github.com/users/matthiasr/events{/privacy}", "received_events_url": "https://api.github.com/users/matthiasr/received_events", "type": "User", "site_admin": false }, "html_url": "https://github.com/matthiasr/import_test", "description": "testing import", "fork": false, "url": "https://api.github.com/repos/matthiasr/import_test", "forks_url": "https://api.github.com/repos/matthiasr/import_test/forks", "keys_url": "https://api.github.com/repos/matthiasr/import_test/keys{/key_id}", "collaborators_url": "https://api.github.com/repos/matthiasr/import_test/collaborators{/collaborator}", "teams_url": "https://api.github.com/repos/matthiasr/import_test/teams", "hooks_url": "https://api.github.com/repos/matthiasr/import_test/hooks", "issue_events_url": "https://api.github.com/repos/matthiasr/import_test/issues/events{/number}", "events_url": "https://api.github.com/repos/matthiasr/import_test/events", "assignees_url": "https://api.github.com/repos/matthiasr/import_test/assignees{/user}", "branches_url": "https://api.github.com/repos/matthiasr/import_test/branches{/branch}", "tags_url": "https://api.github.com/repos/matthiasr/import_test/tags", "blobs_url": "https://api.github.com/repos/matthiasr/import_test/git/blobs{/sha}", "git_tags_url": "https://api.github.com/repos/matthiasr/import_test/git/tags{/sha}", "git_refs_url": "https://api.github.com/repos/matthiasr/import_test/git/refs{/sha}", "trees_url": "https://api.github.com/repos/matthiasr/import_test/git/trees{/sha}", "statuses_url": "https://api.github.com/repos/matthiasr/import_test/statuses/{sha}", "languages_url": "https://api.github.com/repos/matthiasr/import_test/languages", "stargazers_url": "https://api.github.com/repos/matthiasr/import_test/stargazers", "contributors_url": "https://api.github.com/repos/matthiasr/import_test/contributors", "subscribers_url": "https://api.github.com/repos/matthiasr/import_test/subscribers", "subscription_url": "https://api.github.com/repos/matthiasr/import_test/subscription", "commits_url": "https://api.github.com/repos/matthiasr/import_test/commits{/sha}", "git_commits_url": "https://api.github.com/repos/matthiasr/import_test/git/commits{/sha}", "comments_url": "https://api.github.com/repos/matthiasr/import_test/comments{/number}", "issue_comment_url": "https://api.github.com/repos/matthiasr/import_test/issues/comments{/number}", "contents_url": "https://api.github.com/repos/matthiasr/import_test/contents/{+path}", "compare_url": "https://api.github.com/repos/matthiasr/import_test/compare/{base}...{head}", "merges_url": "https://api.github.com/repos/matthiasr/import_test/merges", "archive_url": "https://api.github.com/repos/matthiasr/import_test/{archive_format}{/ref}", "downloads_url": "https://api.github.com/repos/matthiasr/import_test/downloads", "issues_url": "https://api.github.com/repos/matthiasr/import_test/issues{/number}", "pulls_url": "https://api.github.com/repos/matthiasr/import_test/pulls{/number}", "milestones_url": "https://api.github.com/repos/matthiasr/import_test/milestones{/number}", "notifications_url": "https://api.github.com/repos/matthiasr/import_test/notifications{?since,all,participating}", "labels_url": "https://api.github.com/repos/matthiasr/import_test/labels{/name}", "releases_url": "https://api.github.com/repos/matthiasr/import_test/releases{/id}", "deployments_url": "https://api.github.com/repos/matthiasr/import_test/deployments", "created_at": "2024-02-14T15:04:42Z", "updated_at": "2024-02-14T17:41:32Z", "pushed_at": "2024-02-14T15:04:42Z", "git_url": "git://github.com/matthiasr/import_test.git", "ssh_url": "git@github.com:matthiasr/import_test.git", "clone_url": "https://github.com/matthiasr/import_test.git", "svn_url": "https://github.com/matthiasr/import_test", "homepage": "", "size": 0, "stargazers_count": 0, "watchers_count": 0, "language": null, "has_issues": true, "has_projects": false, "has_downloads": false, "has_wiki": false, "has_pages": false, "has_discussions": false, "forks_count": 0, "mirror_url": null, "archived": true, "disabled": false, "open_issues_count": 0, "license": null, "allow_forking": true, "is_template": false, "web_commit_signoff_required": false, "topics": [], "visibility": "private", "forks": 0, "open_issues": 0, "watchers": 0, "default_branch": "main", "permissions": { "admin": true, "maintain": true, "push": true, "triage": true, "pull": true }, "temp_clone_token": "REDACTED", "network_count": 0, "subscribers_count": 1 } ❯ terraform state show github_repository.repository # github_repository.repository: resource "github_repository" "repository" { allow_auto_merge = false allow_merge_commit = true allow_rebase_merge = true allow_squash_merge = true allow_update_branch = false archived = true auto_init = false default_branch = "main" delete_branch_on_merge = false description = "testing import" etag = "W/\"1f2d247405c095b75ee7067274f6e0c2bad8365f12de057ee097ee966b29da80\"" full_name = "matthiasr/import_test" git_clone_url = "git://github.com/matthiasr/import_test.git" has_discussions = false has_downloads = false has_issues = true has_projects = false has_wiki = true html_url = "https://github.com/matthiasr/import_test" http_clone_url = "https://github.com/matthiasr/import_test.git" id = "import_test" is_template = false merge_commit_message = "PR_TITLE" merge_commit_title = "MERGE_MESSAGE" name = "import_test" node_id = "R_kgDOLSYU3g" private = true repo_id = 757470430 squash_merge_commit_message = "COMMIT_MESSAGES" squash_merge_commit_title = "COMMIT_OR_PR_TITLE" ssh_clone_url = "git@github.com:matthiasr/import_test.git" svn_url = "https://github.com/matthiasr/import_test" topics = [] visibility = "private" vulnerability_alerts = false web_commit_signoff_required = false security_and_analysis { advanced_security { status = "disabled" } secret_scanning { status = "disabled" } secret_scanning_push_protection { status = "disabled" } } } ```

[matthiasr]

Oh no, I've been extra confusing myself by choosing has_wiki which cannot actually be set to true on private repositories (which I have been using for testing). Please hold while I retest …

[matthiasr]

Yes, that was probably it. Also, I can no longer reproduce #2145 – the API no longer returns the security_and_analysis field for this repository.