Cannot connect to GitHub from inside the devcontainer

tschaffter commented 2 years ago

I'm trying to push to a branch in Rong's fork from inside the devcontainer. First, the following dialog appears on Windows:

After entering my username and password, the terminal shows the following message. I need to press Ctrl+C to exit the command.

$ git push rrchai HEAD:rong-test
Logon failed, use ctrl+c to cancel basic credential prompt.
fatal: could not read Username for 'https://github.com': terminal prompts disabled

tschaffter commented 2 years ago

Running the command directly from WSL where I have SSH key for GH configured simply works.

tschaffter commented 2 years ago

@rrchai @vpchung Have you ever bumped into this issue?

tschaffter commented 2 years ago

Update

I cloned my fork using the HTTPS url insitead of the SSH url.

I remember now that GH disable authentication using username/password. In a terminal outside of VS Code, I can successfully authenticate when pushing using my GH username and PAT. An error message is displayed if I use my password.
From VS Code, the GH popup doesn't seem to work even if I enter my PAT instead of my password in the password field. Whatever I enter, the terminal in VS Code will tell me twice Logon failed, use ctrl+c to cancel basic credential prompt. but then it will performed the action as if I was authenticated. I realized that I can also click twice on the Cancel button of the popup to get the same behavior.

Ideally I should be able to cache my GH credentials after cloning the fork with the HTTPS link.

tschaffter commented 2 years ago

Do you use GitHub CLI or Git Credential Manager to store your GH credentials for repos cloned with the HTTPS protocol, @vpchung @rrchai ?

vpchung commented 2 years ago

@tschaffter I have never encountered this error.

As for GH credentials, I'm actually using the macOS keychain, but I will probably switch to GCM moving forward.

vpchung commented 2 years ago

Just switched to GCM and am still able to perform git actions smoothly!

tschaffter commented 2 years ago

I installed GCM locally (WSL) and it didn't kick in when working remotely in the devcontainer (expected)
I installed GCM in the devcontainer and it kicked in when I tried to push to Rong's branch.

$ git push rrchai HEAD:rong-test
fatal: No credential store has been selected.

Set the GCM_CREDENTIAL_STORE environment variable or the credential.credentialStore Git configuration setting to one of the following options:

  secretservice : freedesktop.org Secret Service (requires graphical interface)
  gpg           : GNU `pass` compatible credential storage (requires GPG and `pass`)
  cache         : Git's in-memory credential cache
  plaintext     : store credentials in plain-text files (UNSECURE)

See https://aka.ms/gcm/credstores for more information.

fatal: No credential store has been selected.

Set the GCM_CREDENTIAL_STORE environment variable or the credential.credentialStore Git configuration setting to one of the following options:

  secretservice : freedesktop.org Secret Service (requires graphical interface)
  gpg           : GNU `pass` compatible credential storage (requires GPG and `pass`)
  cache         : Git's in-memory credential cache
  plaintext     : store credentials in plain-text files (UNSECURE)

See https://aka.ms/gcm/credstores for more information.

yarn run v1.22.18
$ nx affected:test

 >  NX   Affected criteria defaulted to --base=main --head=HEAD

    ✔  nx run web-challenge-search:test  [existing outputs match the cache, left as is]
    ✔  nx run web-ui:test  [existing outputs match the cache, left as is]
    ✔  nx run web-user-profile:test  [existing outputs match the cache, left as is]
    ✔  nx run web-org-profile:test  [existing outputs match the cache, left as is]
    ✔  nx run web-org-search:test  [existing outputs match the cache, left as is]
    ✔  nx run web-challenge:test  [existing outputs match the cache, left as is]
    ✔  nx run web-not-found:test  [existing outputs match the cache, left as is]
    ✔  nx run web-signup:test  [existing outputs match the cache, left as is]
    ✔  nx run web-about:test  [existing outputs match the cache, left as is]
    ✔  nx run web-login:test  [existing outputs match the cache, left as is]
    ✔  nx run web-home:test  [existing outputs match the cache, left as is]

 ———————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————

 >  NX   Successfully ran target test for 11 projects (120ms)

   Nx read the output from the cache instead of running the command for 11 out of 11 tasks.

Done in 1.56s.
Enumerating objects: 5, done.
Counting objects: 100% (5/5), done.
Delta compression using up to 4 threads
Compressing objects: 100% (3/3), done.
Writing objects: 100% (3/3), 319 bytes | 319.00 KiB/s, done.
Total 3 (delta 2), reused 0 (delta 0), pack-reused 0
remote: Resolving deltas: 100% (2/2), completed with 2 local objects.
To https://github.com/rrchai/challenge-registry
   7375cea..fcb25ac  HEAD -> rong-test

I still need to find how to solve the issue reported in the log. If this works, we should then add GCM to the definition of the devcontainer. I'm also puzzled as to why I can push to rong's branch without providing credentials (at least I think I didn't).

tschaffter commented 2 years ago

The solution to the above issue is described here:

git config --global credential.credentialStore plaintext

At the time of the first push, GCM asked me to authenticate. GCM automatically used the credentials for subsequent pushes.

vpchung commented 2 years ago

Interesting... I didn't install GCM in the devcontainer but the prompt to relogin with GCM did prompt for me when I attempted to push (from the SSH remote of the devcontainer). After logging in, I was able to successfully push into Rong's PR.

tschaffter commented 2 years ago

@vpchung Where did you install GCM? Were you prompted to enter your credentials when being inside the devcontainer on the EC2 instance?

vpchung commented 2 years ago

It was installed on my local machine, and yes, I was prompted to enter my credentials while inside the devcontainer on the instance. Hence my confusion....

Maybe it's a Mac thing....?

tschaffter commented 2 years ago

Push to GitHub remote from remote devcontainer

Attempt 1: fresh container

When committing for the first time in the devcontainer:

vscode@1887abfa3791:/workspaces/challenge-registry$ git commit -S -m 'Test edit'
Author identity unknown

*** Please tell me who you are.

Run

  git config --global user.email "you@example.com"
  git config --global user.name "Your Name"

to set your account's default identity.
Omit --global to set the identity only in this repository.

fatal: unable to auto-detect email address (got 'vscode@1887abfa3791.(none)')

When attempting to push to the branch of another developer:

$ git push
fatal: The upstream branch of your current branch does not match
the name of your current branch.  To push to the upstream branch
on the remote, use

    git push rrchai HEAD:rong-test

To push to the branch of the same name on the remote, use

    git push rrchai HEAD

To choose either option permanently, see push.default in 'git help config'.

Using the first push command, the following popup is shown. For some reason I can cancel the popup twice and the push will still complete (how?). This popup is broken to me. Whatever I input is not accepted. Note that GH no longer accept authentication with the combo username+password. Inputing a PAT doesn't work easier.

Note that GCM is installed on WSL but it didn't kick in.

This shows that I was able to push to the remote repo right after failing to authenticate twice:

vscode@1887abfa3791:/workspaces/challenge-registry$ git push --set-upstream origin test-gcm
Logon failed, use ctrl+c to cancel basic credential prompt.
fatal: could not read Username for 'https://github.com': terminal prompts disabled
Logon failed, use ctrl+c to cancel basic credential prompt.
fatal: could not read Username for 'https://github.com': terminal prompts disabled
yarn run v1.22.18
$ nx affected:test

 >  NX   Affected criteria defaulted to --base=main --head=HEAD

 >  NX   Successfully ran target test for 0 projects (10ms)

Done in 1.62s.
Total 0 (delta 0), reused 0 (delta 0), pack-reused 0
remote: 
remote: Create a pull request for 'test-gcm' on GitHub by visiting:
remote:      https://github.com/tschaffter/challenge-registry/pull/new/test-gcm
remote: 
To https://github.com/tschaffter/challenge-registry
 * [new branch]      test-gcm -> test-gcm
Branch 'test-gcm' set up to track remote branch 'test-gcm' from 'origin'.

Attempt 2: Configure git to use plaintext credentials in devcontainer

First, looking at the git config for the repo in the devcontainer:

$ git config --list | grep credential
credential.helper=!f() { /home/vscode/.vscode-server/bin/da15b6fd3ef856477bf6f4fb29ba1b7af717770d/node /tmp/vscode-remote-containers-25127d5bc909798e35bb5afa12ebc23db5cd142b.js $*; }; f
credential.helper=!f() { /home/vscode/.vscode-server/bin/da15b6fd3ef856477bf6f4fb29ba1b7af717770d/node /tmp/vscode-remote-containers-25127d5bc909798e35bb5afa12ebc23db5cd142b.js $*; }; f
...

This may be what is used by VS Code to show the GitHub login popup.

Run this command in the devcontainer:

git config --global credential.credentialStore plaintext

See the git config again:

$ git config --list | grep credential
credential.helper=!f() { /home/vscode/.vscode-server/bin/da15b6fd3ef856477bf6f4fb29ba1b7af717770d/node /tmp/vscode-remote-containers-25127d5bc909798e35bb5afa12ebc23db5cd142b.js $*; }; f
credential.helper=!f() { /home/vscode/.vscode-server/bin/da15b6fd3ef856477bf6f4fb29ba1b7af717770d/node /tmp/vscode-remote-containers-25127d5bc909798e35bb5afa12ebc23db5cd142b.js $*; }; f
credential.credentialstore=plaintext

The login popup is still displayed.

Attempt 3: Installing GCM in devcontainer

$ sudo dpkg -i gcmcore-linux_amd64.2.0.696.deb 
Selecting previously unselected package gcmcore.
(Reading database ... 25324 files and directories currently installed.)
Preparing to unpack gcmcore-linux_amd64.2.0.696.deb ...
Unpacking gcmcore (2.0.696) ...
Setting up gcmcore (2.0.696) ...
vscode@1887abfa3791:/workspaces/challenge-registry$ git-credential-manager-core configure
Configuring component 'Git Credential Manager'...
Configuring component 'Azure Repos provider'...

Removing the property added in Attempt 2:

git config --global --unset credential.credentialStore

See config:

$ git config --list | grep credential
credential.helper=!f() { /home/vscode/.vscode-server/bin/da15b6fd3ef856477bf6f4fb29ba1b7af717770d/node /tmp/vscode-remote-containers-25127d5bc909798e35bb5afa12ebc23db5cd142b.js $*; }; f
credential.helper=!f() { /home/vscode/.vscode-server/bin/da15b6fd3ef856477bf6f4fb29ba1b7af717770d/node /tmp/vscode-remote-containers-25127d5bc909798e35bb5afa12ebc23db5cd142b.js $*; }; f
credential.helper=
credential.helper=/usr/local/share/gcm-core/git-credential-manager-core
credential.https://dev.azure.com.usehttppath=true

It looks like GCM added/overwrote credential.helper.

UPDATE: The git config credential.credentialStore is required, otherwise we get the following message:

$ git push
fatal: No credential store has been selected.

Set the GCM_CREDENTIAL_STORE environment variable or the credential.credentialStore Git configuration setting to one of the following options:

  secretservice : freedesktop.org Secret Service (requires graphical interface)
  gpg           : GNU `pass` compatible credential storage (requires GPG and `pass`)
  cache         : Git's in-memory credential cache
  plaintext     : store credentials in plain-text files (UNSECURE)

See https://aka.ms/gcm/credstores for more information.

fatal: No credential store has been selected.

Set the GCM_CREDENTIAL_STORE environment variable or the credential.credentialStore Git configuration setting to one of the following options:

  secretservice : freedesktop.org Secret Service (requires graphical interface)
  gpg           : GNU `pass` compatible credential storage (requires GPG and `pass`)
  cache         : Git's in-memory credential cache
  plaintext     : store credentials in plain-text files (UNSECURE)

See https://aka.ms/gcm/credstores for more information.

yarn run v1.22.18
$ nx affected:test

 >  NX   Affected criteria defaulted to --base=main --head=HEAD

 >  NX   Successfully ran target test for 0 projects (9ms)

Done in 1.56s.
Enumerating objects: 5, done.
Counting objects: 100% (5/5), done.
Delta compression using up to 4 threads
Compressing objects: 100% (2/2), done.
Writing objects: 100% (3/3), 293 bytes | 293.00 KiB/s, done.
Total 3 (delta 2), reused 1 (delta 1), pack-reused 0
remote: Resolving deltas: 100% (2/2), completed with 2 local objects.
To https://github.com/tschaffter/challenge-registry
   25f2292..ac84875  test-gcm -> test-gcm

Once again, I was able to push without providing credentials?

After adding the plaintext property back to git config, GCM properly kicks in:

$ git push
Select an authentication method for 'https://github.com/':
  1. Device code (default)
  2. Personal access token
option (enter for default): 2
Enter GitHub personal access token for 'https://github.com/'...
Token: 
yarn run v1.22.18
$ nx affected:test

 >  NX   Affected criteria defaulted to --base=main --head=HEAD

 >  NX   Successfully ran target test for 0 projects (8ms)

Done in 1.48s.
Enumerating objects: 5, done.
Counting objects: 100% (5/5), done.
Delta compression using up to 4 threads
Compressing objects: 100% (3/3), done.
Writing objects: 100% (3/3), 291 bytes | 291.00 KiB/s, done.
Total 3 (delta 2), reused 0 (delta 0), pack-reused 0
remote: Resolving deltas: 100% (2/2), completed with 2 local objects.
To https://github.com/tschaffter/challenge-registry
   ac84875..0ee49c2  test-gcm -> test-gcm

Additional pushes no longer asks for GH credentials.

Conclusion

Attempt 3 where GCM is installed in the devcontainer is an acceptable solution. GCM can be preinstalled in the devcontainer so that developers will simply have to input their credentials once when automatically prompted.

@vpchung told me that she was able to get her remote devcontainer to use the credentials stored in the GCM instance she installed locally (Mac). This would be the ideal solution as developers could "stream" their credentials to the remote devcontainer thanks to VS Code.

tschaffter commented 2 years ago

@rrchai Were you able to push to GitHub from the remote devcontainer? Does the content of the above post rings some bells? GCM seems to be the recommended to store Git credentials nowadays. If you are not using it yet, could you try installed it locally (on your mac), then see you if can seamlessly push to your fork remote without entering credentials because VS Code somehow can make use of your local GCM?

rrchai commented 2 years ago

@tschaffter I could push the code to my fork. It did ask me about git config initially.

That's the history of the commands I ran:

vscode@59a84a85647d:/workspaces/challenge-registry$ git fetch origin
vscode@59a84a85647d:/workspaces/challenge-registry$ git checkout rong-test
Branch 'rong-test' set up to track remote branch 'rong-test' from 'origin'.
Switched to a new branch 'rong-test'
vscode@59a84a85647d:/workspaces/challenge-registry$ git add -u
vscode@59a84a85647d:/workspaces/challenge-registry$ git commit -m "test"
Author identity unknown

*** Please tell me who you are.

Run

  git config --global user.email "you@example.com"
  git config --global user.name "Your Name"

to set your account's default identity.
Omit --global to set the identity only in this repository.

fatal: unable to auto-detect email address (got 'vscode@59a84a85647d.(none)')
vscode@59a84a85647d:/workspaces/challenge-registry$ git config --global user.email "rchai@sagebase.org"
vscode@59a84a85647d:/workspaces/challenge-registry$ git config --global user.name "Rongrong Chai"
vscode@59a84a85647d:/workspaces/challenge-registry$ git config --local init.defaultBranch main
vscode@59a84a85647d:/workspaces/challenge-registry$ git add -u
vscode@59a84a85647d:/workspaces/challenge-registry$ git status
On branch rong-test
Your branch is up to date with 'origin/rong-test'.

Changes to be committed:
  (use "git restore --staged <file>..." to unstage)
        modified:   README.md

vscode@59a84a85647d:/workspaces/challenge-registry$ git commit -m "test"
yarn run v1.22.18
$ nx affected:lint --uncommitted
 >  NX   Successfully ran target lint for 0 projects (9ms)

Done in 1.55s.
[rong-test ec26200] test
 1 file changed, 1 insertion(+), 1 deletion(-)
vscode@59a84a85647d:/workspaces/challenge-registry$ git push origin rong-test
yarn run v1.22.18
$ nx affected:test

 >  NX   Affected criteria defaulted to --base=main --head=HEAD

    ✔  nx run web-user-profile:test (2s)
    ✔  nx run web-challenge-search:test (19s)
    ✔  nx run web-org-profile:test (18s)
    ✔  nx run web-ui:test (26s)
    ✔  nx run web-org-search:test (20s)
    ✔  nx run web-signup:test (3s)
    ✔  nx run web-not-found:test (18s)
    ✔  nx run web-login:test (2s)
    ✔  nx run web-challenge:test (37s)
    ✔  nx run web-about:test (15s)
    ✔  nx run web-home:test (24s)

 —————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————

 >  NX   Successfully ran target test for 11 projects (1m)

Done in 70.28s.
Enumerating objects: 5, done.
Counting objects: 100% (5/5), done.
Delta compression using up to 4 threads
Compressing objects: 100% (3/3), done.
Writing objects: 100% (3/3), 286 bytes | 286.00 KiB/s, done.
Total 3 (delta 2), reused 0 (delta 0), pack-reused 0
remote: Resolving deltas: 100% (2/2), completed with 2 local objects.
To https://github.com/rrchai/challenge-registry
   b6e7ef5..ec26200  rong-test -> rong-test

rrchai commented 2 years ago

@tschaffter

git push rrchai HEAD:rong-test

Are you testing to push something to my fork? Do you want me to try to push something into your fork?

Updated:

I just tried ^. It didn't allow me to push due to lack of permission, which makes sense to me.

vscode@59a84a85647d:/workspaces/challenge-registry$ git remote add thomas https://github.com/tschaffter/challenge-registry
vscode@59a84a85647d:/workspaces/challenge-registry$ git fetch thomas
remote: Enumerating objects: 174, done.
remote: Counting objects: 100% (152/152), done.
remote: Compressing objects: 100% (64/64), done.
remote: Total 174 (delta 93), reused 142 (delta 83), pack-reused 22
Receiving objects: 100% (174/174), 107.65 KiB | 21.53 MiB/s, done.
Resolving deltas: 100% (102/102), completed with 13 local objects.
From https://github.com/tschaffter/challenge-registry
 * [new branch]      doc-testing                   -> thomas/doc-testing
 * [new branch]      evaluate-keycloak             -> thomas/evaluate-keycloak
 * [new branch]      gcm-test-2                    -> thomas/gcm-test-2
 * [new branch]      main                          -> thomas/main
 * [new branch]      setup-ec2-for-devcontainers   -> thomas/setup-ec2-for-devcontainers
 * [new branch]      test-gcm                      -> thomas/test-gcm
 * [new branch]      test-pr                       -> thomas/test-pr
 * [new branch]      vscode-multi-folder-workspace -> thomas/vscode-multi-folder-workspace
vscode@59a84a85647d:/workspaces/challenge-registry$ git checkout thomas/test-pr
Note: switching to 'thomas/test-pr'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

HEAD is now at 46ac5eb Minor update
vscode@59a84a85647d:/workspaces/challenge-registry$ git branch
* (HEAD detached at thomas/test-pr)
  main
  rong-test
vscode@59a84a85647d:/workspaces/challenge-registry$ git add README.md 
vscode@59a84a85647d:/workspaces/challenge-registry$ git commit -m "test"
yarn run v1.22.18
$ nx affected:lint --uncommitted
 >  NX   Successfully ran target lint for 0 projects (8ms)

Done in 0.98s.
[detached HEAD 834a30d] test
 1 file changed, 18 insertions(+), 19 deletions(-)
vscode@59a84a85647d:/workspaces/challenge-registry$ git push thomas thomas/test-pr
yarn run v1.22.18
$ nx affected:test

 >  NX   Affected criteria defaulted to --base=main --head=HEAD

    ✔  nx run web-about:test (5s)

 —————————————————————————————————————————————————————————————————————————————————————————————————————————————————

 >  NX   Successfully ran target test for 1 projects (5s)

Done in 5.86s.
Total 0 (delta 0), reused 0 (delta 0), pack-reused 0
To https://github.com/tschaffter/challenge-registry
 ! [remote rejected] thomas/test-pr -> thomas/test-pr (permission denied)
error: failed to push some refs to 'https://github.com/tschaffter/challenge-registry'

tschaffter commented 2 years ago

Thanks for sharing your commands. Do you have GCM installed on our laptop? If not, I'm wondering how you and I can push to GH remotes without apparently specifying our credentials. Maybe there is a trace of a previous authentication on the EC2 instances or VS Code is somehow grabbing credentials from our hosts..

rrchai commented 2 years ago

@tschaffter I don't remember if I used GCM before, but I did use Github CLI with its configuration setup. So I guess I have some credentials stored locally. :)

tschaffter commented 2 years ago

Since I'm working from the office I decided to give a try using my MBP. I just installed GCM locally but the devcontainer never showed signed that GCM was used. Upon committing, VS Code triggered a GitHub login dialog that offers more option than the dialog I have on Windows. Here I was able to authenticate myself using a PAT (see screenshot). The credentials are saved somewhere so that I didn't have to enter them again for subsequent pushes.

tschaffter commented 2 years ago

This thread confirms that GCM should not be installed in the devcontainer and that VS Code should be redirecting authentication request to the local instance of GCM.

Yeah, if you don't install the GCM in the container, and you have it configured locally in your .gitconfig, it should be used by the dev container automatically. If you install the GCM in the container, that will take over, and theres no way to pop anything up from inside the container itself. Only text would be allowed.

The redirection is defined in two locations inside the devcontainer:

vscode@fe83bf95e5bd:/workspaces$ cd /home/vscode/ 
vscode@fe83bf95e5bd:~$ cat .gitconfig  
[credential]
        helper = "!f() { /home/vscode/.vscode-server/bin/c3511e6c69bb39013c4a4b7b9566ec1ca73fc4d5/node /tmp/vscode-remote-containers-3e37fb277048489a6f31b854d451e4b0cf64893f.js $*; }; f"
[user]
        email = thomas.schaffter@gmail.com
        name = Thomas Schaffter

vscode@fe83bf95e5bd:~$ git config --list | grep credential
credential.helper=!f() { /home/vscode/.vscode-server/bin/c3511e6c69bb39013c4a4b7b9566ec1ca73fc4d5/node /tmp/vscode-remote-containers-3cec0e5dd2ad1eaba11b3a5cdd0513f9845bdbb8.js $*; }; f
credential.helper=!f() { /home/vscode/.vscode-server/bin/c3511e6c69bb39013c4a4b7b9566ec1ca73fc4d5/node /tmp/vscode-remote-containers-3cec0e5dd2ad1eaba11b3a5cdd0513f9845bdbb8.js $*; }; f

tschaffter commented 2 years ago

So authentication works seamlessly on remote devcontainer on MacOS. On Windows-WSL, a solution to the broken authentication dialog is to installed GCM in the devcontainer.

tschaffter commented 2 months ago

Update 2024-04-12

The latest solution is to configure the GitHub CLI as credential helper, see #2638

Sage-Bionetworks / sage-monorepo