Open MarcoFabbri-orienteed opened 1 year ago
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.
Hi @agilgur5 thank you for marking this as a bug. Do you have news on this issue?
No. Any updates would be in the issue. Please see https://sindresorhus.com/blog/issue-bumping.
You also did not fill out the bug report template, which does not help.
This error is thrown using Argo Workflow 3.4.10: any idea?
The fix to #10600, #10953, was not released until 3.5.0-rc1. So I would suggest trying with 3.5.0
you are right @agilgur5 , let me provide more information
Pre-requisites
:3.5.0
What happened/what you expected to happen? To connect to an Azure devops repository and clone the content as artifacts. Following this generic example https://github.com/argoproj/argo-workflows/blob/master/examples/input-artifact-git.yaml
Version
Version{Version:v3.5.0,BuildDate:2023-10-13T13:06:32Z,GitCommit:bf735a2e861d6b1c686dd4a076afc3468aa89c4a,GitTag:v3.5.0,GitTreeState:clean,GoVersion:go1.21.3,Compiler:gc,Platform:linux/amd64
Paste a small workflow that reproduces the issue. We must be able to run the workflow; don't enter a workflows that uses private images.
apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
generateName: input-artifact-git-
spec:
entrypoint: git-clone
templates:
- name: git-clone
inputs:
artifacts:
- name: argo-source
path: /src
git:
repo: https://dev.azure.com/<projectFolder>/_git/<repoName>
branch: main
usernameSecret:
name: azuredevops-git-secret
key: username
passwordSecret:
name: azuredevops-git-secret
key: password
container:
image: golang:1.10
command: [sh, -c]
args: ["git status && ls && cat VERSION"]
workingDir: /src
The secret was created with
apiVersion: v1
data:
password: <password>
username: YXJnbw==
kind: Secret
metadata:
name: azuredevops-git-secret
namespace: argo
type: Opaque
Logs from the workflow controller
time="2023-11-02T10:00:33.013Z" level=info msg="Processing workflow" namespace=argo workflow=input-artifact-git-rrch5-hmhhj
time="2023-11-02T10:00:33.033Z" level=warning msg="Non-transient error: configmaps \"artifact-repositories\" not found"
time="2023-11-02T10:00:33.033Z" level=info msg="resolved artifact repository" artifactRepositoryRef=default-artifact-repository
time="2023-11-02T10:00:33.033Z" level=info msg="Updated phase -> Running" namespace=argo workflow=input-artifact-git-rrch5-hmhhj
time="2023-11-02T10:00:33.033Z" level=warning msg="Node was nil, will be initialized as type Skipped" namespace=argo workflow=input-artifact-git-rrch5-hmhhj
time="2023-11-02T10:00:33.033Z" level=info msg="was unable to obtain node for , letting display name to be nodeName" namespace=argo workflow=input-artifact-git-rrch5-hmhhj
time="2023-11-02T10:00:33.033Z" level=info msg="Pod node input-artifact-git-rrch5-hmhhj initialized Pending" namespace=argo workflow=input-artifact-git-rrch5-hmhhj
time="2023-11-02T10:00:33.033Z" level=error msg="was unable to obtain node for " namespace=argo workflow=input-artifact-git-rrch5-hmhhj
time="2023-11-02T10:00:33.129Z" level=info msg="Created pod: input-artifact-git-rrch5-hmhhj (input-artifact-git-rrch5-hmhhj)" namespace=argo workflow=input-artifact-git-rrch5-hmhhj
time="2023-11-02T10:00:33.129Z" level=info msg="TaskSet Reconciliation" namespace=argo workflow=input-artifact-git-rrch5-hmhhj
time="2023-11-02T10:00:33.129Z" level=info msg=reconcileAgentPod namespace=argo workflow=input-artifact-git-rrch5-hmhhj
time="2023-11-02T10:00:33.129Z" level=info msg="Workflow to be dehydrated" Workflow Size=2054
time="2023-11-02T10:00:33.140Z" level=info msg="Workflow update successful" namespace=argo phase=Running resourceVersion=148216596 workflow=input-artifact-git-rrch5-hmhhj
time="2023-11-02T10:00:43.131Z" level=info msg="Processing workflow" namespace=argo workflow=input-artifact-git-rrch5-hmhhj
time="2023-11-02T10:00:43.131Z" level=info msg="Task-result reconciliation" namespace=argo numObjs=0 workflow=input-artifact-git-rrch5-hmhhj
time="2023-11-02T10:00:43.131Z" level=info msg="Pod failed: Error (exit code 1): artifact argo-source failed to load: failed to clone \"https://dev.azure.com/*/_git/<repoName>\": unexpected client error: unexpected requesting \"https://dev.azure.com/*/_git/<repoName>/git-upload-pack\" status code: 400" displayName=input-artifact-git-rrch5-hmhhj namespace=argo pod=input-artifact-git-rrch5-hmhhj templateName=git-clone workflow=input-artifact-git-rrch5-hmhhj
time="2023-11-02T10:00:43.131Z" level=info msg="marking node as failed since init container has non-zero exit code" namespace=argo new.phase=Failed workflow=input-artifact-git-rrch5-hmhhj
time="2023-11-02T10:00:43.131Z" level=info msg="node changed" namespace=argo new.message="Error (exit code 1): artifact argo-source failed to load: failed to clone \"https://dev.azure.com/*/_git/<repoName>\": unexpected client error: unexpected requesting \"https://dev.azure.com/*/_git/<repoName>/git-upload-pack\" status code: 400" new.phase=Failed new.progress=0/1 nodeID=input-artifact-git-rrch5-hmhhj old.message= old.phase=Pending old.progress=0/1 workflow=input-artifact-git-rrch5-hmhhj
time="2023-11-02T10:00:43.132Z" level=info msg="TaskSet Reconciliation" namespace=argo workflow=input-artifact-git-rrch5-hmhhj
time="2023-11-02T10:00:43.132Z" level=info msg=reconcileAgentPod namespace=argo workflow=input-artifact-git-rrch5-hmhhj
time="2023-11-02T10:00:43.132Z" level=info msg="Updated phase Running -> Failed" namespace=argo workflow=input-artifact-git-rrch5-hmhhj
time="2023-11-02T10:00:43.132Z" level=info msg="Updated message -> Error (exit code 1): artifact argo-source failed to load: failed to clone \"https://dev.azure.com/*/_git/<repoName>\": unexpected client error: unexpected requesting \"https://dev.azure.com/*/_git/<repoName>\/git-upload-pack\" status code: 400" namespace=argo workflow=input-artifact-git-rrch5-hmhhj
time="2023-11-02T10:00:43.132Z" level=info msg="Marking workflow completed" namespace=argo workflow=input-artifact-git-rrch5-hmhhj
time="2023-11-02T10:00:43.132Z" level=info msg="Workflow to be dehydrated" Workflow Size=3085
time="2023-11-02T10:00:43.137Z" level=info msg="cleaning up pod" action=deletePod key=argo/input-artifact-git-rrch5-hmhhj-1340600742-agent/deletePod
time="2023-11-02T10:00:43.142Z" level=info msg="Workflow update successful" namespace=argo phase=Failed resourceVersion=148216674 workflow=input-artifact-git-rrch5-hmhhj
time="2023-11-02T10:00:43.143Z" level=info msg="Queueing Failed workflow argo/input-artifact-git-rrch5-hmhhj for delete in 168h0m0s due to TTL"
time="2023-11-02T10:00:43.170Z" level=info msg="cleaning up pod" action=labelPodCompleted key=argo/input-artifact-git-rrch5-hmhhj/labelPodCompleted
time="2023-11-02T10:02:09.213Z" level=info msg="Alloc=47344 TotalAlloc=31311568 Sys=127816 NumGC=1407 Goroutines=175"
Logs from in your workflow's init container
time="2023-11-02T10:00:37.183Z" level=info msg="Starting Workflow Executor" version=v3.5.0
time="2023-11-02T10:00:37.186Z" level=info msg="Using executor retry strategy" Duration=1s Factor=1.6 Jitter=0.5 Steps=5
time="2023-11-02T10:00:37.186Z" level=info msg="Executor initialized" deadline="0001-01-01 00:00:00 +0000 UTC" includeScriptOutput=false namespace=argo podName=input-artifact-git-rrch5-hmhhj template="{\"name\":\"git-clone\",\"inputs\":{\"artifacts\":[{\"name\":\"argo-source\",\"path\":\"/src\",\"git\":{\"repo\":\"https://dev.azure.com/*/_git/<repoName>\",\"usernameSecret\":{\"name\":\"azuredevops-git-secret\",\"key\":\"username\"},\"passwordSecret\":{\"name\":\"azuredevops-git-secret\",\"key\":\"password\"},\"branch\":\"main\"}}]},\"outputs\":{},\"metadata\":{},\"container\":{\"name\":\"\",\"image\":\"golang:1.10\",\"command\":[\"sh\",\"-c\"],\"args\":[\"git status \\u0026\\u0026 ls \\u0026\\u0026 cat VERSION\"],\"workingDir\":\"/src\",\"resources\":{}}}" version="&Version{Version:v3.5.0,BuildDate:2023-10-13T13:06:32Z,GitCommit:bf735a2e861d6b1c686dd4a076afc3468aa89c4a,GitTag:v3.5.0,GitTreeState:clean,GoVersion:go1.21.3,Compiler:gc,Platform:linux/amd64,}"
time="2023-11-02T10:00:37.241Z" level=info msg="Start loading input artifacts..."
time="2023-11-02T10:00:37.241Z" level=info msg="Downloading artifact: argo-source"
time="2023-11-02T10:00:37.339Z" level=info msg="Load artifact" artifactName=argo-source duration=97.805551ms error="failed to clone \"https://dev.azure.com/*/_git/<repoName>\": unexpected client error: unexpected requesting \"https://dev.azure.com/*/_git/<repoName>/git-upload-pack\" status code: 400" key=
time="2023-11-02T10:00:37.339Z" level=error msg="executor error: artifact argo-source failed to load: failed to clone \"https://dev.azure.com/*/_git/<repoName>\": unexpected client error: unexpected requesting \"https://dev.azure.com/*/_git/<repoName>/git-upload-pack\" status code: 400"
time="2023-11-02T10:00:37.340Z" level=info msg="Alloc=10152 TotalAlloc=15800 Sys=23909 NumGC=4 Goroutines=5"
time="2023-11-02T10:00:37.340Z" level=fatal msg="artifact argo-source failed to load: failed to clone \"https://dev.azure.com/*/_git/<repoName>\": unexpected client error: unexpected requesting \"https://dev.azure.com/*/_git/<repoName>/git-upload-pack\" status code: 400"
I hope this is enough to understand the issue.
Thanks
Thanks for the update @mfabbri. Could you please add code blocks and syntax highlighting (i.e. ```yaml) to your comment? It looks like it wasn't formatted correctly.
So you're still having this issue in 3.5 then, correct?
@rohankmr414 would you be able to take a look at this? I think some of the go-git
changes were still in-progress when you last worked on this issue (I know #11149 never quite got completed, but don't know the full context)
You're right @agilgur5 : I didn't remember the syntax. Now it's fixed. Please @rohankmr414 , ask me any additional details you need about the test I performed. Thank you both
I think I might have found the issue. #10953 merely bumped go-git
, but it actually requires some code on our side to add to workaround the Azure issue. https://github.com/go-git/go-git/pull/613 did not entirely fix https://github.com/go-git/go-git/issues/64 -- it provided a workaround, and that workaround requires some manual code. There's an example in the go-git
PR that requires implementing; one implementation of it can be seen in https://github.com/kluctl/kluctl/pull/637
That should be fairly simple to add. @mfabbri would you be interested in contributing that? If not, I can probably take this on
Thank you for the proposal @agilgur5 but I don't think I have enough knowledge to proceed with this implementation. Can you take it?
we are also experiencing such issue and waiting for the fix :)
Hi all, the issue 10600 is closed but I still have an authentication issue on Azure devops with the git artifact example
https://github.com/argoproj/argo-workflows/blob/master/examples/input-artifact-git.yaml
artifact argo-source failed to load: failed to clone "https://<org>@dev.azure.com/<org>/folder/_git/customerdata": unexpected client error: unexpected requesting "https://<org>@dev.azure.com/<org>/folder/_git/customerdata/git-upload-pack" status code: 400
This error is thrown using Argo Workflow 3.4.10: any idea?
Thanks
Originally posted by @MarcoFabbri-orienteed in https://github.com/argoproj/argo-workflows/issues/10600#issuecomment-1673171748