Open sigwinch28 opened 10 months ago
sigwinch28
commented
10 months ago I speculate that my malformed scp-style URLs are being intepreted as file:// URLs, which are unsupported by go-git:
https://github.com/go-git/go-git/blob/master/COMPATIBILITY.md#transport-schemes
sigwinch28
commented
10 months ago I speculate that my malformed scp-style URLs are being intepreted as
file://URLs, which are unsupported by go-git:https://github.com/go-git/go-git/blob/master/COMPATIBILITY.md#transport-schemes
This appears to be what's happening. Reproduced using the same version of go-git as argo-workflows:
go.mod:
module github.com/sigwinch28/go-git-scp-repro
go 1.21.3
require github.com/go-git/go-git/v5 v5.4.2
require (
github.com/acomagu/bufpipe v1.0.3 // indirect
github.com/go-git/gcfg v1.5.0 // indirect
github.com/go-git/go-billy/v5 v5.3.1 // indirect
github.com/jbenet/go-context v0.0.0-20150711004518-d14ea06fba99 // indirect
github.com/mitchellh/go-homedir v1.1.0 // indirect
golang.org/x/net v0.0.0-20210326060303-6b1517762897 // indirect
golang.org/x/sys v0.0.0-20210502180810-71e4cd670f79 // indirect
gopkg.in/warnings.v0 v0.1.2 // indirect
)
main.go:
package main
import (
"fmt"
"github.com/go-git/go-git/v5/plumbing/transport"
)
func main() {
wellFormed, err := transport.NewEndpoint("git@github.com:argo/argo-workflows.git")
if err != nil {
panic(err)
}
fmt.Printf("Well-formed: %s\n", wellFormed.String())
illFormed, err := transport.NewEndpoint("git@github.com/argo/argo-workflows.git")
if err != nil {
panic(err)
}
fmt.Printf("Ill-formed: %s\n", illFormed.String())
}
output of go run main.go:
Well-formed: ssh://git@github.com/argo/argo-workflows.git
Ill-formed: file://git@github.com/argo/argo-workflows.gitA workaround for users wanting file:// support is to add the git executable to their executor image, e.g. using the following Dockerfile:
#syntax=docker/dockerfile:1.4
ARG ARGO_WORKFLOWS_VERSION
FROM quay.io/argoproj/argoexec:v${ARGO_WORKFLOWS_VERSION} as argoexec
FROM alpine:3.18
RUN apk update && apk add --no-cache git
COPY --link --from=argoexec /bin/argoexec /bin/argoexecThen running docker build --build-arg ARGO_WORKFLOWS_VERSION=<<your version of argo workflows here>> ., publishing the image somewhere, and updating the workflows controller arguments to use your published image.
When I run the workflow posted above using this modified image, I get the expected repository not found error which is less cryptic.
$ argo submit -n argo-workflows --wait repro.yaml
Name: repro-prnml
Namespace: argo-workflows
ServiceAccount: unset (will run with the default ServiceAccount)
Status: Pending
Created: Sun Nov 05 15:33:23 +0000 (now)
Progress:
repro-prnml Failed at 2023-11-05 15:33:33 +0000 GMT
$ argo logs -n argo-workflows repro-prnml -c init
repro-prnml: time="2023-11-05T15:33:26.662Z" level=info msg="Starting Workflow Executor" version=v3.5.1
repro-prnml: time="2023-11-05T15:33:26.666Z" level=info msg="Using executor retry strategy" Duration=1s Factor=1.6 Jitter=0.5 Steps=5
repro-prnml: time="2023-11-05T15:33:26.666Z" level=info msg="Executor initialized" deadline="0001-01-01 00:00:00 +0000 UTC" includeScriptOutput=false namespace=argo-workflows podName=repro-prnml template="{\"name\":\"git-clone\",\"inputs\":{\"artifacts\":[{\"name\":\"argo-source\",\"path\":\"/src\",\"git\":{\"repo\":\"git@github.com/argoproj/argo-workflows.git\",\"revision\":\"v3.5.1\",\"sshPrivateKeySecret\":{\"name\":\"github-credentials\",\"key\":\"ssh-private-key\"}}}]},\"outputs\":{},\"metadata\":{},\"container\":{\"name\":\"\",\"image\":\"golang:1.10\",\"command\":[\"sh\",\"-c\"],\"args\":[\"git status \\u0026\\u0026 ls\"],\"workingDir\":\"/src\",\"resources\":{}}}" version="&Version{Version:v3.5.1,BuildDate:2023-11-03T18:37:58Z,GitCommit:877c5523066e17687856fe3484c9b2d398e986f5,GitTag:v3.5.1,GitTreeState:clean,GoVersion:go1.21.3,Compiler:gc,Platform:linux/amd64,}"
repro-prnml: time="2023-11-05T15:33:26.783Z" level=info msg="Start loading input artifacts..."
repro-prnml: time="2023-11-05T15:33:26.783Z" level=info msg="Downloading artifact: argo-source"
repro-prnml: time="2023-11-05T15:33:26.786Z" level=info msg="Load artifact" artifactName=argo-source duration=3.34862ms error="failed to clone \"git@github.com/argoproj/argo-workflows.git\": repository not found" key=
repro-prnml: time="2023-11-05T15:33:26.786Z" level=error msg="executor error: artifact argo-source failed to load: failed to clone \"git@github.com/argoproj/argo-workflows.git\": repository not found"
repro-prnml: time="2023-11-05T15:33:26.787Z" level=info msg="Alloc=8344 TotalAlloc=12376 Sys=25189 NumGC=3 Goroutines=4"
repro-prnml: time="2023-11-05T15:33:26.787Z" level=fatal msg="artifact argo-source failed to load: failed to clone \"git@github.com/argoproj/argo-workflows.git\": repository not found"Root cause is #8292 and #7798. It might be more appropriate to make this a regression in support for the file:// scheme for git artifacts
agilgur5
commented
10 months ago Follow-up to this Slack thread.
We could potentially catch this exact error and give a friendlier error message like "file:// URL detected, an image with the git binary must be used"
Pre-requisites
:latestWhat happened/what you expected to happen?
When I use a repo of the form
user@host/repoinstead ofuser@host:repofor git input artifacts (i.e., I accidentally use an incorrect forward slash instead of the correct colon), the pipeline fails with an unintuitive error message ofexec: \"git\": executable file not found in $PATHHere is how I reproduced it:
Version
v3.5.1
Paste a small workflow that reproduces the issue. We must be able to run the workflow; don't enter a workflows that uses private images.
Logs from the workflow controller
Logs from in your workflow's wait container