microsoft / azure-pipelines-agent

Azure Pipelines Agent 🚀
MIT License
1.7k stars 855 forks source link

[BUG]: "could not read Password" error in `checkout` task when `fetchFilter` is provided #4860

Open afscrome opened 1 week ago

afscrome commented 1 week ago

What happened?

If you add a fetchFilter (from #4693) to the checkout task (as well as enabling the AGENT_USE_FETCH_FILTER_IN_CHECKOUT_TASK knob), then the checkout manages to fetch objects from the remotes, but fails on the final git checkout task with an authentication issue:

There is a warning in the result suggesting changing fetchDepth, however I get the same issue regardless of whether I set fetch depth

Repo pipeline:

job: demo
displayName: 🏗️ Build
steps:
  - checkout: self
    fetchFilter: tree:0
    #fetchDepth: 0

Versions

Current agent version: '3.241.0'

Operating System Ubuntu 22.04.4 LTS

Runner Image Image: ubuntu-22.04 Version: 20240616.1.0 Included Software: https://github.com/actions/runner-images/blob/ubuntu22/20240616.1/images/ubuntu/Ubuntu2204-Readme.md Image Release: https://github.com/actions/runner-images/releases/tag/ubuntu22%2F20240616.1

Runner Image Provisioner 2.0.370.1 Current image version: '20240616.1.0'

Environment type (Please select at least one enviroment where you face this issue)

Azure DevOps Server type

dev.azure.com (formerly visualstudio.com)

Azure DevOps Server Version (if applicable)

No response

Operation system

Ubuntu 22.04.04 LTS

Version controll system

Git

Relevant log output

Starting: Checkout REPO@main to s
==============================================================================
Task         : Get sources
Description  : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.
Version      : 1.0.0
Author       : Microsoft
Help         : [More Information](https://go.microsoft.com/fwlink/?LinkId=798199)
==============================================================================
Syncing repository: REPO (Git)
git version
git version 2.45.2
git lfs version
git-lfs/3.5.1 (GitHub; linux amd64; go 1.21.8)
git init "/home/vsts/work/1/s"
hint: Using 'master' as the name for the initial branch. This default branch name
hint: is subject to change. To configure the initial branch name to use in all
hint: of your new repositories, which will suppress this warning, call:
hint:
hint:   git config --global init.defaultBranch <name>
hint:
hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and
hint: 'development'. The just-created branch can be renamed via this command:
hint:
hint:   git branch -m <name>
Initialized empty Git repository in /home/vsts/work/1/s/.git/
git remote add origin https://ORG@dev.azure.com/ORG/PROJECT/_git/REPO
git config gc.auto 0
git config core.longpaths true
git config --get-all http.https://ORG@dev.azure.com/ORG/PROJECT/_git/REPO.extraheader
git config --get-all http.extraheader
git config --get-regexp .*extraheader
git config --get-all http.proxy
git config http.version HTTP/1.1
git --config-env=http.extraheader=env_var_http.extraheader fetch --force --tags --prune --prune-tags --progress --no-recurse-submodules origin --depth=1 --filter=tree:0 +edb49c62b56ceb393542ebb413c58b39130fd07e:refs/remotes/origin/edb49c62b56ceb393542ebb413c58b39130fd07e
remote: Azure Repos        
remote: 
remote: Found 1 objects to send. (10 ms)        
Receiving objects: 100% (1/1)
Receiving objects: 100% (1/1), 184 bytes | 184.00 KiB/s, done.
From https://dev.azure.com/ORG/PROJECT/_git/REPO
 * [new ref]         edb49c62b56ceb393542ebb413c58b39130fd07e -> origin/edb49c62b56ceb393542ebb413c58b39130fd07e
git --config-env=http.extraheader=env_var_http.extraheader fetch --force --tags --prune --prune-tags --progress --no-recurse-submodules origin --depth=1 --filter=tree:0 +edb49c62b56ceb393542ebb413c58b39130fd07e
remote: Azure Repos        
remote: 
remote: Found 0 objects to send. (19 ms)        
From https://dev.azure.com/ORG/PROJECT/_git/REPO
 * branch            edb49c62b56ceb393542ebb413c58b39130fd07e -> FETCH_HEAD
git checkout --progress --force refs/remotes/origin/edb49c62b56ceb393542ebb413c58b39130fd07e
fatal: could not read Password for 'https://ORG@dev.azure.com': terminal prompts disabled
fatal: could not fetch 7f12c57ea65125ee8da773d3de56ea12bc338daa from promisor remote
##[warning]Git checkout failed on shallow repository, this might because of git fetch with depth '1' doesn't include the checkout commit 'refs/remotes/origin/edb49c62b56ceb393542ebb413c58b39130fd07e'. Please reference documentation (http://go.microsoft.com/fwlink/?LinkId=829603)
##[error]Git checkout failed with exit code: 128
Finishing: Checkout REPO@main to s
afscrome commented 3 days ago

Setting AGENT_USE_FETCH_FILTER_IN_CHECKOUT_TASK no longer seems to be required, but the issue still occurs.

I believe the problem is the final checkout call is missing credentials (presumably --config-env=http.extraheader=env_var_http.extraheader)

git checkout --progress --force refs/remotes/origin/edb49c62b56ceb393542ebb413c58b39130fd07e

If you add a second checkout step before with persistCredential: true, then the treeless clone works correctly.

  steps:
    # Fake checkout in order to get credentials persisted, after which point the second treeless clone works
    - checkout: self
      persistCredentials: true
      path: fake

    - checkout: self
      fetchFilter: tree:0
      fetchDepth: 1
      fetchTags: false