search

git-ecosystem / git-credential-manager

Secure, cross-platform Git credential storage with authentication to GitHub, Azure Repos, and other popular Git hosting services.
Other
6.79k stars 1.77k forks source link

After installing private Github repo as dependency via yarn with fine-grained access tokens, GCM encounters 403 errors #1421

Open Nantris opened 1 year ago

Nantris commented 1 year ago

Version

2.3.2

Operating system

Linux

OS version or distribution

Ubuntu 23.04

Git hosting provider(s)

GitHub

Other hosting provider

No response

(Azure DevOps only) What format is your remote URL?

None

Can you access the remote repository directly in the browser?

Yes, I can access the repository

Expected behavior

Using fine-grained access tokens to define a yarn or npm dependency should not cause git-credential-manager to fail with 403 errors:

remote: Write access to repository not granted.
fatal: unable to access 'https://github.com/Username/ProjectName.git/': The requested URL returned error: 403

Unfortunately in our case this is the only way we can access that repository besides merging it into our monorepo which we do not want to do, but this seems to force us.

Actual behavior

I get the error message posted above and it persists until the machine restarts of I run echo "url=https://github.com" | git credential reject

git credential-manager diagnose yields no warnings or other complaints.

The same problem affected the 2.0.x version I was previously on.

Logs

Diagnose log at 2023-09-21T01:39:06Z AppPath: /usr/local/bin/git-credential-manager InstallDir: /usr/local/share/gcm-core/ Version: 2.3.2 ------------ Diagnostic: Environment Skipped: False Success: True Exception: None Log: OSType: Linux OSVersion: Ubuntu 23.04 Reading environment variables... OK Variables: KONSOLE_DBUS_SERVICE=:1.44 KDE_APPLICATIONS_AS_SCOPE=1 XDG_SESSION_PATH=/org/freedesktop/DisplayManager/Session1 KDE_SESSION_UID=1000 LESS=-R SHELL_SESSION_ID=6eb2289323fb495a8571193aec5f25ed HOME=/home/me KONSOLE_DBUS_SESSION=/Sessions/1 OLDPWD=/home/me/Documents/MyProject LOGNAME=me USER=me LSCOLORS=Gxfxcxdxbxegedabagacad QTWEBENGINE_DICTIONARIES_PATH=/usr/share/hunspell-bdic/ XDG_SESSION_DESKTOP=KDE COLORTERM=truecolor PAM_KWALLET5_LOGIN=/run/user/1000/kwallet5.socket ANDROID_SDK=/home/me/Android/Sdk/ XCURSOR_THEME=Breeze_Snow KDE_SESSION_VERSION=5 DESKTOP_SESSION=plasma GTK_RC_FILES=/etc/gtk/gtkrc:/home/me/.gtkrc:/home/me/.config/gtkrc XDG_DATA_DIRS=/usr/share/plasma:/home/me/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share XDG_SESSION_TYPE=x11 XDG_SESSION_ID=3 XAUTHORITY=/home/me/.Xauthority XDG_VTNR=1 XCURSOR_SIZE=24 ANDROID_HOME=/home/me/Android/Sdk/ REACT_EDITOR=subl PAGER=less KDE_FULL_SESSION=true TERM=xterm-256color EXPO_USE_EXOTIC=1 GTK_USE_PORTAL=0 PATH=/usr/lib/git-core:/home/me/.nvm/versions/node/v18.16.0/bin:/home/me/Android/Sdk/:/home/me/Android/Sdk//platform-tools/:/home/me/.local/bin:/home/me/.cargo/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin NVM_BIN=/home/me/.nvm/versions/node/v18.16.0/bin NVM_DIR=/home/me/.nvm SSH_AUTH_SOCK=/tmp/ssh-XXXXXXG7mf7u/agent.1229 PWD=/home/me/Documents/MyProject GIT_TRACE2_PARENT_SID=5bc64cee-c416-44e9-8583-1cf845f2f2c8 NVM_INC=/home/me/.nvm/versions/node/v18.16.0/include/node XDG_CURRENT_DESKTOP=KDE ANDROID_SDK_ROOT=/home/me/Android/Sdk/ SESSION_MANAGER=local/ubuntu:@/tmp/.ICE-unix/1491,unix/ubuntu:/tmp/.ICE-unix/1491 QT_SCREEN_SCALE_FACTORS=Virtual1=1;Virtual2=1;Virtual3=1;Virtual4=1;Virtual5=1;Virtual6=1;Virtual7=1;Virtual8=1; _=/usr/bin/git WINDOWID=65011725 COLORFGBG=15;0 SHELL=/usr/bin/zsh KONSOLE_VERSION=221203 PROFILEHOME= NVM_CD_FLAGS=-q IM_CONFIG_PHASE=1 DISPLAY=:0 GTK2_RC_FILES=/etc/gtk-2.0/gtkrc:/home/me/.gtkrc-2.0:/home/me/.config/gtkrc-2.0 QT_AUTO_SCREEN_SCALE_FACTOR=0 DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus LANGUAGE= XDG_SESSION_CLASS=user SYSTEMD_EXEC_PID=1189 XDG_SEAT_PATH=/org/freedesktop/DisplayManager/Seat0 SHLVL=1 GIT_EXEC_PATH=/usr/lib/git-core XDG_SEAT=seat0 XDG_RUNTIME_DIR=/run/user/1000 SSH_AGENT_PID=1345 XDG_CONFIG_DIRS=/home/me/.config/kdedefaults:/etc/xdg/xdg-plasma:/etc/xdg:/usr/share/kubuntu-default-settings/kf5-settings ZSH=/home/me/.oh-my-zsh MyProject_PORT=2324 LANG=en_US.UTF-8 GPG_AGENT_INFO=/run/user/1000/gnupg/S.gpg-agent:0:1 LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=00:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.avif=01;35:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.webp=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36:*~=00;90:*#=00;90:*.bak=00;90:*.old=00;90:*.orig=00;90:*.part=00;90:*.rej=00;90:*.swp=00;90:*.tmp=00;90:*.dpkg-dist=00;90:*.dpkg-old=00;90:*.ucf-dist=00;90:*.ucf-new=00;90:*.ucf-old=00;90:*.rpmnew=00;90:*.rpmorig=00;90:*.rpmsave=00;90: QT_ACCESSIBILITY=1 ------------ Diagnostic: File system Skipped: False Success: True Exception: None Log: Temporary directory is '/tmp/'... Checking basic file I/O... Writing to temporary file '/tmp/9fad43898339a37740d821ac'... OK Reading from temporary file '/tmp/9fad43898339a37740d821ac'... OK Deleting temporary file '/tmp/9fad43898339a37740d821ac'... OK Testing IFileSystem instance... UserHomePath: /home/me UserDataDirectoryPath: /home/me/.gcm GetCurrentDirectory(): /home/me/Documents/MyProject ------------ Diagnostic: Networking Skipped: False Success: True Exception: None Log: Checking networking and HTTP stack... Creating HTTP client... OK IsNetworkAvailable: True Sending HEAD request to http://example.com...Sending HEAD request to https://example.com... OK OK Acquiring free TCP port... OK Testing local HTTP loopback connections... Creating new HTTP listener for http://localhost:43417/... OK Waiting for loopback connection... OK Writing response... OK Waiting for response data... OK Loopback connection data OK ------------ Diagnostic: Git Skipped: False Success: True Exception: None Log: Getting Git version... OK Git version is '2.39.2' Locating current repository... OK Git repository at '/home/me/Documents/MyProject/.git' Listing all Git configuration... OK Git configuration: file:/home/me/.gitconfig user.email=me.myself@gmail.com file:/home/me/.gitconfig user.name=me file:/home/me/.gitconfig credential.credentialstore=secretservice file:/home/me/.gitconfig credential.helper=/usr/local/bin/git-credential-manager file:.git/config core.repositoryformatversion=0 file:.git/config core.filemode=true file:.git/config core.bare=false file:.git/config core.logallrefupdates=true file:.git/config core.hookspath=.husky file:.git/config remote.origin.url=https://github.com/Me/MyProject.git file:.git/config remote.origin.fetch=+refs/heads/*:refs/remotes/origin/* file:.git/config branch.master.remote=origin file:.git/config branch.master.merge=refs/heads/master ------------ Diagnostic: Credential storage Skipped: False Success: True Exception: None Log: ICredentialStore instance is of type: CredentialStore Writing test credential... OK Reading test credential... OK Deleting test credential... OK ------------ Diagnostic: Microsoft authentication (AAD/MSA) Skipped: False Success: True Exception: None Log: Broker is not enabled. Flow type is: Auto Gathering MSAL token cache data... OK CacheDirectory: /home/me/.local/.IdentityService CacheFileName: msal.cache CacheFilePath: /home/me/.local/.IdentityService/msal.cache KeyringCollection: KeyringSchemaName: KeyringSecretLabel: KeyringAttribute1: (,) KeyringAttribute2: (,) Creating cache helper... OK Verifying MSAL token cache persistence... OK ------------ Diagnostic: GitHub API Skipped: False Success: True Exception: None Log: Using 'https://github.com/' as API target. Querying '/meta' endpoint... OK
Nantris commented 1 year ago

I've all but confirmed that the issue is the fine-grained access tokens. When I replace the token with another fine-grained access token that I gave access to everything, the issue no longer appears.

mjcheetham commented 1 year ago 
remote: Write access to repository not granted.
fatal: unable to access 'https://github.com/Username/ProjectName.git/': The requested URL returned error: 403

Git does not callback to credential helpers when encountering a 403, so there's nothing GCM can do here. The message beginning remote: comes from GitHub.com and indicates that the token used does not have sufficient permissions to write to the repository.

I've all but confirmed that the issue is the fine-grained access tokens. When I replace the token with another fine-grained access token that I gave access to everything, the issue no longer appears.

Using fine-grained (or classic) personal access tokens with GCM means you must have created one manually and entered it in a GCM "token" prompt. GCM has no control over that token.

What scopes and which specific repositories have you granted access to for that token? Typically you need the following repository permissions to be enabled to work with Git:

Nantris commented 1 year ago

@mjcheetham thanks for your reply. I think I wasn't entirely clear with what I'm experiencing. The fine-grained access tokens work fine for their purposes, but GCM is (incorrectly in my view) scoping them up and treating them as the definitive access to our Git repo, even though we're signing in via the popup that invokes the web browser. I never enter any token into GCM at any point, but it seems that simply using the token in Yarn causes GCM to override my more privileged credentials (from the web browser sign in) with the personal access tokens instead.

I'm planning to move away from this model so it likely won't cause my any long term problems, but I wanted to report the issue since it's very strange and took me several hours to realize the nature of the problem.

mjcheetham commented 1 year ago

GCM is (incorrectly in my view) scoping them up and treating them as the definitive access to our Git repo, even though we're signing in via the popup that invokes the web browser.

Oh! To clarify, your scenario so I'm sure I understand..

I am not familiar with npm or yarn and what they are doing with the Git repos that are specified. It is possible that they are performing some extra operations that are not covered by the default scope set that GCM creates. Do you know what extra operations or scopes would be required here? We may consider expanding the default set that GCM creates tokens with for common scenarios.

If you manually generate a token with expanded scopes and enter that in the token box when prompted by GCM, that should work (until said token expires) as GCM does not modify the scope off tokens after they've been created.

Nantris commented 1 year ago

The first two bullet points are definitely right but I'm not quite sure what you mean by this:

  • The token that GCM is creating is scoped too narrowly, resulting in an error.

Crucially, I sign in via the web browser with full access first and everything works - but then after running yarn GCM seems to discard the web-browser-based auth and use the fine-grained personal access token instead, which is what seems to cause the error.

Do you know what extra operations or scopes would be required here?

Unfortunately not.

But I'm not sure I've been entirely clear still. The way we were loading a dependency was like this in our package.json:

"dependency-name": "git+https://username:github_pat_123456789012345678901234567890123456789012345678901234567890@github.com/username/dependency-name.git"

As it's poor security practice it may not be necessary to actually support this, but the fact that it breaks working GCM without being obvious about what the cause is is the bigger problem in my mind because it can cause a lot of wasted time. As a result I tried re-configuring, reinstalling, upgrading GCM all to no avail.

After some research I've settled on an alternative approach which I'm hopeful will work with all of our build processes. It's working for every use I've tested so far.

This actually does not work

.yarnrc/.npmrc

//npm.pkg.github.com/:_authToken=${GITHUB_TOKEN}

And then set GITHUB_TOKEN as an environmental variable using the fine-grained personal access token. This avoids breaking GCM and hardcoding variables into the repo.

Nantris commented 1 year ago

Little update, my workaround doesn't work so this may actually be a problem worth addressing.