runatlantis / atlantis

Terraform Pull Request Automation
https://www.runatlantis.io
Other
7.82k stars 1.06k forks source link

Atlantis unable to authenticate to GitHub #2526

Closed ayushkr04 closed 4 days ago

ayushkr04 commented 2 years ago

We have been running Atlantis for 2 years now without any issues. Since today morning, suddenly, we have started encountering error:

fatal: could not read Password for 'https://<<< GIT TOKEN HERE No such device or address

There has been no change in the token or secrets.

The gitconfig in values.yaml is defined as below:

vcsSecretName: 'atlantis-secrets'

gitconfig: | [url "https://<<< GIT TOKEN HERE >>>@github.com"] insteadOf = https://github.com [url "https://<<< GIT TOKEN HERE >>>@github.com"] insteadOf = ssh://git@github.com

Let me know if you need any additional information.

jamengual commented 2 years ago

I don't know if tokens expire or not but did you try creating a new one?

On Fri, Sept 16, 2022, 8:00 a.m. Ayush Kumar @.***> wrote:

We have been running Atlantis for 2 years now without any issues. Since today morning, suddenly, we have started encountering error:

fatal: could not read Password for 'https://<<< GIT TOKEN HERE No such device or address

There has been to change in the token or secrets.

The gitconfig in values.yaml is defined as below:

vcsSecretName: 'atlantis-secrets'

gitconfig: | [url "https://<<< GIT TOKEN HERE @."] insteadOf = https://github.com [url "https://<<< GIT TOKEN HERE @."] insteadOf = @.***

Let me know if you need any additional information.

— Reply to this email directly, view it on GitHub https://github.com/runatlantis/atlantis/issues/2526, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAQ3ERGHWPUB5LSX5EC7HK3V6SDRRANCNFSM6AAAAAAQONDV4A . You are receiving this because you are subscribed to this thread.Message ID: @.***>

ayushkr04 commented 2 years ago

I tried logging in with the token to GH, and I could successfully login ..

jamengual commented 2 years ago

did you reset the token in atlantis?

If this does not work generate a new one and see if it works.

you could be getting throttled or something else.

On Fri, Sept 16, 2022, 9:25 a.m. Ayush Kumar @.***> wrote:

I tried logging in with the token to GH, and I could successfully login ..

— Reply to this email directly, view it on GitHub https://github.com/runatlantis/atlantis/issues/2526#issuecomment-1249558851, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAQ3ERB2ROB434NFY2QKVUTV6SNPHANCNFSM6AAAAAAQONDV4A . You are receiving this because you commented.Message ID: @.***>

ayushkr04 commented 2 years ago

I will to try create a new webhook for atlantis as suggested and add that to the repo But out of curiosity, if the current webhook token would have expired/non-functional, it would have thrown error. Right ? But I do get a OK response.

jamengual commented 2 years ago

yes, but as you said, 2 years running and suddenly this happens? then definitely something happen to that token for some reason external to Atlantis

On Fri, Sept 16, 2022, 10:18 a.m. Ayush Kumar @.***> wrote:

I will to create a new webhook for atlantis and add that to the repo But out of curiosity, if the current webhook token would have expired/non-functional, I would have thrown error. Right ? But I do get a OK response.

— Reply to this email directly, view it on GitHub https://github.com/runatlantis/atlantis/issues/2526#issuecomment-1249602428, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAQ3ERCVGJLM4KMJI4PXQKLV6STWXANCNFSM6AAAAAAQONDV4A . You are receiving this because you commented.Message ID: @.***>

ayushkr04 commented 2 years ago

@jamengual I tried updating the webhook as well but no luck. Well I don't think, but could it be because of the upgrade I had made from 0.8.2 to 0.15.1 ?

Below are the logs from the pod just in case if that's helpful.

2022/09/19 08:36:51+0000 [INFO] GoogleCloudPlatform//test-project#167: Creating dir "/atlantis-data/repos/GoogleCloudPlatform/test-project/167/default" 2022/09/19 08:36:51+0000 [INFO] GoogleCloudPlatform//test-project#167: Successfully parsed atlantis.yaml file 2022/09/19 08:36:51+0000 [INFO] GoogleCloudPlatform//test-project#167: 1 projects are to be planned based on their when_modified config 2022/09/19 08:36:52+0000 [INFO] GoogleCloudPlatform//test-project#167: Acquired lock with id "GoogleCloudPlatform/test-project/dns/peered-zones/default" 2022/09/19 08:36:53+0000 [EROR] GoogleCloudPlatform//test-project#167: Running "/usr/local/bin/terraform init -input=false -no-color -upgrade" in "/atlantis-data/repos/GoogleCloudPlatform//test-project/167/default/dns/peered-zones": exit status 1 2022/09/19 08:36:53+0000 [INFO] GoogleCloudPlatform//test-project#167: Deleting plans because there were errors and automerge requires all plans succeed

ayushkr04 commented 2 years ago

Below is the complete output of atlantis plan:

running "/atlantis-data/bin/terraform0.13.4 init -input=false -no-color -upgrade" in "/atlantis-data/repos/GoogleCloudPlatform/test-project/167/default/dns/peered-zones": exit status 1 Upgrading modules... Downloading git::ssh://[git@github.com]/GoogleCloudPlatform/terraform-modules.git for peered-dns-zones...

Error: Failed to download module

Could not download module "peered-dns-zones" (main.tf:1) source code from "git::ssh://[git@github.com]/GoogleCloudPlatform/terraform-modules.git": error downloading 'ssh://[git@github.com]/GoogleCloudPlatform/terraform-modules.git': /usr/bin/git exited with 128: Cloning into '.terraform/modules/peered-dns-zones'... fatal: could not read Password for 'https://<;<< GIT TOKEN HERE

@github.com': No such device or address

File contents: /etc/secret-gitconfig/gitconfig

[url "https://<<< GIT TOKEN HERE >>>@github.com"] insteadOf = https://github.com [url "https://<<< GIT TOKEN HERE >>>@github.com"] insteadOf = ssh://git@github.com

Helm Chart:


gitconfig: | [url "https://<<< GIT TOKEN HERE >>>@github.com"] insteadOf = https://github.com [url "https://<<< GIT TOKEN HERE >>>@github.com"] insteadOf = ssh://git@github.com

comjf commented 2 years ago

I'm not exactly sure what is up here, or if your issue is the same as mine, but this github issue helped me trace through a recent git related breakage for me as well.

There was a behavior change in my most recent upgrade: ATLANTIS_VERSION v0.19.8 -> v0.19.9-pre.20220912

I have an external data script that runs:

git ls-remote git@github.com:${ORG}/${PROJECT}.git

This no longer works, I had to look through some of these linked issues and change it to:

git ls-remote ssh://git@github.com/${ORG}/${PROJECT}.git

Probably just a breaking change from a dependency if I had to guess. I'm not sure how many people will be similarly affected when the pre-release image is launched, but it may warrant fixing for backwards compatibility.

Sharing in the hopes that it helps someone. Cheers and love the project!

jamengual commented 2 years ago

can you add the link to the github issue?

Thanks.

On Wed, Sep 21, 2022 at 1:46 PM James Flowers @.***> wrote:

I'm not exactly sure what is up here, or if your issue is the same as mine, but this github issue helped me trace through a recent git related breakage for me as well.

There was a behavior change in my most recent upgrade: ATLANTIS_VERSION v0.19.8 -> v0.19.9-pre.20220912

I have an external data script that runs:

git ls-remote @.***:${ORG}/${PROJECT}.git

This no longer works, I had to look through some of these linked issues and change it to:

git ls-remote @./${ORG}/${PROJECT}.git @./$%7BORG%7D/$%7BPROJECT%7D.git>

Probably just a breaking change from a dependency if I had to guess. I'm not sure how many people will be similarly affected when the pre-release image is launched, but it may warrant fixing for backwards compatibility.

Sharing in the hopes that it helps someone. Cheers and love the project!

— Reply to this email directly, view it on GitHub https://github.com/runatlantis/atlantis/issues/2526#issuecomment-1254210205, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAQ3ERGVFWNZXHPJXU7DXHTV7NXY5ANCNFSM6AAAAAAQONDV4A . You are receiving this because you were mentioned.Message ID: @.***>