I changed how GITHUB_TOKEN works on my local machine to allow switching between projects more easily.
In short, I'm prefixing them and select the correct token using direnv.
Effectively, GITHUB_TOKEN is now empty on my local machine and will only be set if a project uses some form of direnv or me manually exporting a certain github token in my shell.
Reasoning is some project work with classic tokens while we're all (trying to) transistion towards fine grained tokens.
In a repo where I didn't setup direnv the github terraform provider was trying to connect without any credentials.
We're using the provider like this and I know we could fix this by setting token in the provider.
provider "github" {
owner = "Q42"
}
My feature request is to make logging more clear.
Upon debugging with TF_LOG_PROVIDER=DEBUG, I encountered a long list the following warnings (line endings are mine):
2024-01-08T17:50:26.273+0100 [DEBUG] provider.terraform-provider-github_v5.41.0:
2024/01/08 17:50:26 [INFO] Removing repository path /some-file/kustomization.yaml from state
because the branch no longer exists in GitHub
For each of the github_repository_file resources in the terraform state the check failed and it was considered removed from the state.
The result of this was that the planned change involved recreating all of the github_repository_file since the refreshed state was assuming the files weren't there anymore. I don't know what would happen if this was applied.
To make things a bit worse for my particular case: I ran into a github rate limit:
2024-01-08T17:50:26.364+0100 [DEBUG] provider.terraform-provider-github_v5.41.0:
2024/01/08 17:50:26 [DEBUG] Rate limit 60 reached, sleeping for 59m50.635986s
(until 2024-01-08 18:50:17.000001 +0100 CET m=+3602.992082893) before retrying
in the end gave it all away, but that's just logged once and it doesn't mention GITHUB_TOKEN. It's also only logged when enabling more debugging. I didn't see this until I already knew what was going on.
Fixing logging in this particular might be an approach to warn users faster:
output the warning in the normal stdout
output the fact that anonymous access causes the rate limit (in debug logging)
refresh improvements that could be made
interpret the 404 better when refreshing state: do you even have repo access? In the anonymous case the 404 is not because the file doesn't exists anymore.
Something I also thought of is to optin or opt out of the ability to use the github provider with anonymous access.
I think most developers would use the github provider to actually change things so they wouldn't need anonymous access.
From a user's perspective (but hey, I'm biased right now) allow_anonymous_access should preferably default to false to prevent this kind of issue. But that's a backwards incompatible case.
Defaulting allow_anonymous_access to true could also work (maybe change the default when a new mayor version comes along?) if the provider documentation recommends setting it to true.
SDK Version
provider.terraform-provider-github_v5.41.0
API Version
No response
Relevant log output
2024-01-08T17:50:14.178+0100 [DEBUG] provider.terraform-provider-github_v5.41.0:
2024/01/08 17:50:14 [INFO] No token present; configuring anonymous owner.
2024-01-08T17:50:26.273+0100 [DEBUG] provider.terraform-provider-github_v5.41.0:
2024/01/08 17:50:26 [INFO] Removing repository path /some-file/kustomization.yaml from state
because the branch no longer exists in GitHub
Code of Conduct
[X] I agree to follow this project's Code of Conduct
Describe the need
I changed how GITHUB_TOKEN works on my local machine to allow switching between projects more easily. In short, I'm prefixing them and select the correct token using
direnv. Effectively, GITHUB_TOKEN is now empty on my local machine and will only be set if a project uses some form ofdirenvor me manually exporting a certain github token in my shell. Reasoning is some project work with classic tokens while we're all (trying to) transistion towards fine grained tokens.In a repo where I didn't setup direnv the github terraform provider was trying to connect without any credentials. We're using the provider like this and I know we could fix this by setting
tokenin the provider.My feature request is to make logging more clear.
Upon debugging with TF_LOG_PROVIDER=DEBUG, I encountered a long list the following warnings (line endings are mine):
For each of the
github_repository_fileresources in the terraform state the check failed and it was considered removed from the state. The result of this was that the planned change involved recreating all of thegithub_repository_filesince the refreshed state was assuming the files weren't there anymore. I don't know what would happen if this was applied.To make things a bit worse for my particular case: I ran into a github rate limit:
The log entry:
in the end gave it all away, but that's just logged once and it doesn't mention
GITHUB_TOKEN. It's also only logged when enabling more debugging. I didn't see this until I already knew what was going on.Fixing logging in this particular might be an approach to warn users faster:
refresh improvements that could be made
Something I also thought of is to optin or opt out of the ability to use the github provider with anonymous access. I think most developers would use the github provider to actually change things so they wouldn't need anonymous access.
From a user's perspective (but hey, I'm biased right now)
allow_anonymous_accessshould preferably default tofalseto prevent this kind of issue. But that's a backwards incompatible case. Defaultingallow_anonymous_accesstotruecould also work (maybe change the default when a new mayor version comes along?) if the provider documentation recommends setting it to true.SDK Version
provider.terraform-provider-github_v5.41.0
API Version
No response
Relevant log output
Code of Conduct