[BUG]: 422 Secret scanning is not available for this repository.

[kiranbaddi]

Expected Behavior

After updating the list of approver_ids for the github_repository_environment resource, the list of approvers for the approve production should be updated. We are doing it for multiple repositories. All repositories for private.

Actual Behavior

Terraform apply throws the following error for one of the newly created repositories

╷
│ Error: PATCH https://api.github.com/repos/retracedgmbh/constants-service: 422 Secret scanning is not available for this repository. []
│ 
│   with module.new--repository.github_repository.repository,
│   on modules/repositories/main.tf line 107, in resource "github_repository" "repository":
│  107: resource "github_repository" "repository" {

Terraform Version

❯Terraform v1.5.6 on linux_amd64

• provider registry.terraform.io/integrations/github v5.38.0

Affected Resource(s)

-github_environment.

Terraform Configuration Files

resource "github_repository_environment" "env_approve_production" {
  environment = "approve-production"
  repository  = var.name

  reviewers {
    users = var.approver_ids
  }

  depends_on = [github_repository.repository]
}

### Steps to Reproduce

Add new revieres and run 
``` $ terraform apply ```

### Debug Output

```shell
│ Error: PATCH https://api.github.com/repos/retracedgmbh/new-repository: 422 Secret scanning is not available for this repository. []
│
│   with module.new-repository.github_repository.repository,
│   on modules/repositories/main.tf line 107, in resource "github_repository" "repository":
│  107: resource "github_repository" "repository" {
│
╵
Operation failed: failed running terraform apply (exit 1)

Panic Output

No response

Code of Conduct

• [X] I agree to follow this project's Code of Conduct

[laurenty]

I also ran into this issue today using version v5.38.0 of the provider. Upgrading to v5.45.0 did not solve the issue. Terraform version: 1.5.7

[tanguy-platsec]

On our side we encountered the same issue. I believe the main affected resource is the github_repository. Maybe it's related to a change in the backend/API caused by this new feature? Kind of annoying to see all the terraform applies fail because of a random change we did not ask for, without any action on our side (no updgrade of the provider or anything suspicious).

[dinoshauer]

Also ran into this, on provider version 5.45.0 and terraform version 1.7.0 I managed to work around it with this patch:

  security_and_analysis {
    secret_scanning_push_protection {
      status = "disabled"
    }

    secret_scanning {
      status = "disabled"
    }
  }

[mohramadan911]

we ran into the same just on 13 Feb

[muawiakh]

Also running in the same issue and since we are using github_repository via a module, it would require a module upgrade across all callees.

[anttijman]

We have this same issue as well. Can not remove nor modify a repository resource

[richjyoung]

Same, existing repos with no changes planned are fine, attempting to apply changes to an imported repository fails with this error. We've never enabled this feature.

[Homulvas]

Also ran into this, on provider version 5.45.0 and terraform version 1.7.0 I managed to work around it with this patch:

  security_and_analysis {
    secret_scanning_push_protection {
      status = "disabled"
    }

    secret_scanning {
      status = "disabled"
    }
  }

This didn't help in our case. Even after setting explicit security_and_analysis values for all repos any change to the existing repos results in the same error.

[gh-risharma]

How to resolve this issue? Any solution?

[pmmalinov01]

reapply it. It helped us.

[gh-risharma]

reapply it. It helped us.

without changing anything? Just a re-apply would help?

[kpocius]

~When you get the error, applying again helps, but as soon as you make any other changes to the repo, you get the error again and have to apply a second time.~ Nope, looks like it's completely impossible to make any changes. This is becoming critical issue fast -- we had to revert to making all changes manually :(

[pmmalinov01]

reapply it. It helped us.

without changing anything? Just a re-apply would help?

It helped us. For some stupid reason

[lukedoesinfra]

Provider version 5.45.0.

Just dumped the debug logs from the PATCH api call and can clearly see the security_and_analysis section being set but still getting a 422 back from Github.

<snip>
2024-02-14T14:07:41.413Z [DEBUG] provider.terraform-provider-github_v5.45.0:  "security_and_analysis": {
2024-02-14T14:07:41.413Z [DEBUG] provider.terraform-provider-github_v5.45.0:   "secret_scanning": {
2024-02-14T14:07:41.413Z [DEBUG] provider.terraform-provider-github_v5.45.0:    "status": "disabled"
2024-02-14T14:07:41.413Z [DEBUG] provider.terraform-provider-github_v5.45.0:   },
2024-02-14T14:07:41.413Z [DEBUG] provider.terraform-provider-github_v5.45.0:   "secret_scanning_push_protection": {
2024-02-14T14:07:41.413Z [DEBUG] provider.terraform-provider-github_v5.45.0:    "status": "disabled"
2024-02-14T14:07:41.413Z [DEBUG] provider.terraform-provider-github_v5.45.0:   }
2024-02-14T14:07:41.413Z [DEBUG] provider.terraform-provider-github_v5.45.0:  }
2024-02-14T14:07:41.413Z [DEBUG] provider.terraform-provider-github_v5.45.0: }

[mBlomsterberg]

Facing the same issue here. Workflows are blocked until fixed.

Tried adding:

security_and_analysis {
    secret_scanning_push_protection {
      status = "disabled"
    }
    secret_scanning {
      status = "disabled"
    }
    advanced_security {
      status = "disabled" 
   }
  }

AND reapplying fixed it.

[gh-risharma]

reapply it. It helped us.

without changing anything? Just a re-apply would help?

It helped us. For some stupid reason

not helping for me. still getting the same error. any other fixes for it?

+ terraform apply git_tfplan
module.github_repo["terraform-aws-gh-dp-glue"].github_repository.this: Modifying... [id=terraform-aws-gh-dp-glue]
╷
│ Error: PATCH https://api.github.com/repos/gh-org-data-platform/terraform-aws-gh-dp-glue: 422 Secret scanning is not available for this repository. []
│ 
│   with module.github_repo["terraform-aws-gh-dp-glue"].github_repository.this,
│   on repository/main.tf line 2, in resource "github_repository" "this":
│    2: resource "github_repository" this {
│ 

[Laffs2k5]

Observed the same issue today in our integration test suite for our github terraform project. No changes in terraform version, code or config, no change in provider (5.45.0) since last successful test.

There must be a change in the API's behavior here, can't think of any other cause 🤷‍♂️

We ended up having to perform changes manually.

[gh-risharma]

what is the cause of this error? like why suddenly this error is coming up? anyone??

[meliamzz]

what is the cause of this error? like why suddenly this error is coming up? anyone??

Maybe this new feature as mentioned by @tanguy-platsec The engineering team is working on a fix apparently.

[lukedoesinfra]

what is the cause of this error? like why suddenly this error is coming up? anyone??

Maybe this new feature . The engineering team is working on a fix apparently.

Can you link the reply from Github?

[rwblokzijl]

what is the cause of this error? like why suddenly this error is coming up? anyone??

Maybe this new feature . The engineering team is working on a fix apparently.

Can you link the reply from Github?

Github support for enterprise answered me the same thing:

We're sorry to hear about this. We've relayed this information onto our engineers to investigate further. Hopefully, they'll be able to identify if there's been a change on our end and address it as soon as possible. We understand this is causing significant business impact and want to assure you our team are looking into it. We'll let you know as soon as there's been any progress...

[hikerspath]

Same, had to disable on private repos.

[gh-risharma]

what is the cause of this error? like why suddenly this error is coming up? anyone??

Maybe this new feature . The engineering team is working on a fix apparently.

Can you link the reply from Github?

Github support for enterprise answered me the same thing:

We're sorry to hear about this. We've relayed this information onto our engineers to investigate further. Hopefully, they'll be able to identify if there's been a change on our end and address it as soon as possible. We understand this is causing significant business impact and want to assure you our team are looking into it. We'll let you know as soon as there's been any progress...

Any idea by what time it will be fixed?

[rwblokzijl]

what is the cause of this error? like why suddenly this error is coming up? anyone??

Maybe this new feature . The engineering team is working on a fix apparently.

Can you link the reply from Github?

Github support for enterprise answered me the same thing:

We're sorry to hear about this. We've relayed this information onto our engineers to investigate further. Hopefully, they'll be able to identify if there's been a change on our end and address it as soon as possible. We understand this is causing significant business impact and want to assure you our team are looking into it. We'll let you know as soon as there's been any progress...

Any idea by what time it will be fixed?

[gh-risharma]

I'm getting this weird error, the first job after this change , the plan contains - no changess. but as soon as i apply the pln appears with 1 change and the apply is failed with this error. what is happening?

Facing the same issue here. Workflows are blocked until fixed.

Tried adding:

security_and_analysis {
    secret_scanning_push_protection {
      status = "disabled"
    }
    secret_scanning {
      status = "disabled"
    }
    advanced_security {
      status = "disabled" 
   }
  }

AND reapplying fixed it.

[matthiasr]

~I filed #2148 because I believe it is a separate issue, but it may be confusing attempts to work around this issue because it makes it look like everything is OK, a subsequent plan is "clean", but values haven't actually been updated.~

No, nevermind, that was PEBKAC, the attribute I was testing with is not actually mutable for private repositories.

[matthiasr]

While I was testing, the GitHub API stopped including a security_and_analysis field in the GET repos response. I have proof that it previously did, but now it no longer does.

However, if I have it in my Terraform state (terraform state show github_repository.repository includes security_and_analysis), the provider includes it in the PATCH request, which provokes the 422.

It seems that I can un-stick myself when I

• remove the security_and_analysis block from my code
• terraform state rm the github_repository resource
• then re-terraform import it (or use the import block)

Then, terraform state show no longer includes this block, the PATCH to update other fields does not include it, and thus I can reliably toggle other fields.

As an alternative to state rm/import, manually editing the state file to remove this field from the resource also works, but don't 😄

[matthiasr]

To re-state what I believe happened:

• For a while, GET responses for repositories that are ineligible for secret scanning / advanced security included disabled values for these fields.
• Terraform, via the GitHub provider, saved those into the state, on refresh or import.
• Subsequently, any attempt to update the repository through Terraform includes them in the PATCH request
• These requests are rejected with a 422 response but the otherwise requested changes do take effect.
• Now, GET responses no longer include this field. Importing a repository no longer imports a broken state.
• However, a refresh does not remove the field from the state.
• PATCH requests that include the field are still being rejected.
• It is still possible to add the fields into the state by adding them to the resource, even with all disabled values, which also breaks it.

[kfcampbell]

@matthiasr you are correct in this assessment! The bug at the API level has now been fixed.

I wonder if there's an elegant way to handle this at the provider level? At the moment I'm blanking on how we might resolve the issue without relying on a manual workaround in each case.

[matthiasr]

It seems like fixing this in the provider would require teaching the provider about the business decisions on which exact repositories can have this configuration. Which will not depend only on the repository but also on the organization.

Also, a provider fix will take time to roll out. Aside from the time to make a release, I suspect any provider with security_and_analysis support (so, any from the last ~2 years) would have saved these fields, forcing everyone to crash update now who isn't already near the newest version but ran a plan with refresh while the API returned the fields.

In my opinion, the best way out here would be a backend fix:

Since the API did return this field in GET repo responses, and evidently clients exist who store these and expect to be able to echo them back into the API, the API should allow security_and_analysis with only disabled values, even if disabled is the only valid value for the given repository. I think it would also need to handle only some of the subfields being set. When one or more are set to disabled, this is functionally reaffirming the reality of the repository; the API should not error in that case.

Secondly, and less urgently, the current behavior of returning 422 but actually doing the thing is very very confusing and potentially dangerous: the API is telling the client the entity was unprocessable – but it does process it, despite saying it won't.

[matthiasr]

I checked, and all of our managed github_repository resources now have security_and_analysis in the state, when none of them should. So implementing one of the workarounds I gave above will be major state surgery one way or another, which I would really rather not do.

[matthiasr]

On the provider level, maybe something that would help is not to include everything from the state in the PATCH, only the fields that are actually specified in the configuration – or even better, only those that need to be changed according to the plan.

At the same time, on refresh, clear fields from the state that are no longer returned from the API? This part may be complicated a little by the etag which means in some cases, the provider doesn't even get updated content (without the fields). Even if the API fix changed all the etags, a refresh after the fix would write the new etag to the state without deleting the security_and_analysis fields. Maybe for a while, the provider would have to not send etags during refresh at all, and always get the full response?

[hikerspath]

Working today for me! No disabling necessary. Thanks GitHub!

[matthiasr]

Same for me – I got a little confused because all repositories did have a security_and_analysis attribute in the state (but it was an empty array for most). The ones that actually have anything are the public repositories.

In case anyone needs it, a command to check for "problem states" is

terraform state pull | jq -r '.resources[] | select(.mode == "managed" and .type == "github_repository") | .instances[] | select(.attributes.private) | select((.attributes.security_and_analysis | length) > 0) | .attributes.id'

If you still have some, try terraform refresh.

[azmodude]

Is anyone experiencing this problem using GitHub Enterprise Server on premise? I am getting hit by this when importing existing repositories via terraform import. I am sure it worked before, but we have recently updated to 3.11.5 - maybe some change was introduced there.

No matter what my .tf file says, the imported state always shows the

   security_and_analysis {
        advanced_security {
            status = "disabled"
        }
        secret_scanning {
            status = "disabled"
        }
        secret_scanning_push_protection {
            status = "disabled"
        }
    }

block. Subsequent changes to those imported repositories fail with the 422 Secret scanning is not available for this repository. error. No state rm and importing again and/or refresh seems to help.

[Kristijan]

Is anyone experiencing this problem using GitHub Enterprise Server on premise? I am getting hit by this when importing existing repositories via terraform import. I am sure it worked before, but we have recently updated to 3.11.5 - maybe some change was introduced there.

No matter what my .tf file says, the imported state always shows the

   security_and_analysis {
        advanced_security {
            status = "disabled"
        }
        secret_scanning {
            status = "disabled"
        }
        secret_scanning_push_protection {
            status = "disabled"
        }
    }

block. Subsequent changes to those imported repositories fail with the 422 Secret scanning is not available for this repository. error. No state rm and importing again and/or refresh seems to help.

I'm running into this issue with GitHub Enterprise Server 3.9.6. None of the above workarounds have helped.

However, for me, it's not a persistent error. I run my TF plan, and hit the 422 Secret scanning is not available for this repository error. Then I run the same plan again 30 minutes later, and it works fine.

[TahreemSaleem]

We encountered the same issue using Terraform version v1.5.7 and provider version v6.2.1. We resolved it by adding the following configuration in the github_repository resource: security_and_analysis {}

Additionally, we made sure that security_and_analysis is not present in the lifecycle { ignore_changes: ... } block of that resource.

Terraform made the following changes during terraform plan.

~ security_and_analysis {
      - advanced_security {
          - status = "disabled" -> null
        }
      - secret_scanning {
          - status = "disabled" -> null
        }
      - secret_scanning_push_protection {
          - status = "disabled" -> null
        }
   }