Closed mlbiam closed 2 years ago
vdemeester
commented
2 years ago @mlbiam thanks for the issue. Is there anymore logs ? The message is a warning and shouldn't prevent the rest of the step to execute. The rest of the step(s) might fail for a valid reason linked to it (most likely because the credentials are not found, …) but it could be worked-around I think (by mounting the secret instead of relying to the annotated secret).
mlbiam
commented
2 years ago @vdemeester You're right. The task runs but fails because it can't talk to the remote git repository. Here are the logs:
$ tkn pipelinerun logs build-hello-pipeline-run -n python-hello-build
task generate-image-tag has failed: "step-git-source-git-resource-5s5r8" exited with code 1 (image: "gcr.io/tekton-releases/github.com/tektoncd/pipeline/cmd/git-init@sha256:39b53980b255ef8af80086ac702f5b113d32f60cc4c8876b096ba37fc5912c81"); for logs run: kubectl -n python-hello-build logs build-hello-pipeline-run-generate-image-tag-4r659-pod-kq5dv -c step-git-source-git-resource-5s5r8
[generate-image-tag : create-dir-result-image-njhbx] 2021/09/28 18:48:27 warning: unsuccessful cred copy: ".ssh" from "/tekton/creds" to "/": unable to create destination directory: mkdir /.ssh: permission denied
[generate-image-tag : git-source-git-resource-5s5r8] {"level":"error","ts":1632854913.79133,"caller":"git/git.go:54","msg":"Error running git [fetch --recurse-submodules=yes --depth=1 origin --update-head-ok --force master]: exit status 128\nNo user exists for uid 1000\r\nfatal: Could not read from remote repository.\n\nPlease make sure you have the correct access rights\nand the repository exists.\n","stacktrace":"github.com/tektoncd/pipeline/pkg/git.run\n\tgithub.com/tektoncd/pipeline/pkg/git/git.go:54\ngithub.com/tektoncd/pipeline/pkg/git.Fetch\n\tgithub.com/tektoncd/pipeline/pkg/git/git.go:149\nmain.main\n\tgithub.com/tektoncd/pipeline/cmd/git-init/main.go:53\nruntime.main\n\truntime/proc.go:225"}
[generate-image-tag : git-source-git-resource-5s5r8] {"level":"fatal","ts":1632854913.7915208,"caller":"git-init/main.go:54","msg":"Error fetching git repository: failed to fetch [master]: exit status 128","stacktrace":"main.main\n\tgithub.com/tektoncd/pipeline/cmd/git-init/main.go:54\nruntime.main\n\truntime/proc.go:225"}
failed to get logs for task generate-image-tag : container step-git-source-git-resource-5s5r8 has failed : [{"key":"StartedAt","value":"2021-09-28T18:48:33.686Z","type":3}]
Tasks Completed: 1 (Failed: 1, Cancelled 0), Skipped: 2Here's the Task: https://github.com/PacktPublishing/Kubernetes---An-Enterprise-Guide-2E/blob/main/chapter14/example-apps/tekton/tekton-task1.yaml
Manually mounting the Secret was my next thought. I wasn't sure if there was another way to go. I figured this is similar to how OpenShift runs (injecting random runAsUser) so maybe there was a way around this warning i was missing.
mlbiam
commented
2 years ago My workaround for now was to disable gatekeeper in namespaces that run pipelines. Looking at the Pod objects being generated, it looks like they're running with suplemental group id, but no user id or group id.
I checked out the Dockerfile for the git-init and checkout images, and they add user and group 65532, but they stay as root. I tried changing my mutation to assign 65532 as the default runAsUser/runAsGroup and no longer got the unsuccessful cred copy error, but the git checkout fails:
{"level":"error","ts":1633090656.5643337,"caller":"git/git.go:54","msg":"Error running git [fetch --recurse-submodules=yes --depth=1 origin --update-head-ok --force main]: exit status 128\nWarning: Permanently added 'gitlab-gitlab-shell.gitlab.svc.cluster.local,10.100.8.4' (ECDSA) to the list of known hosts.\r\ngit@gitlab-gitlab-shell.gitlab.svc.cluster.local: Permission denied (publickey,keyboard-interactive).\r\nfatal: Could not read from remote repository.\n\nPlease make sure you have the correct access rights\nand the repository exists.\n","stacktrace":"github.com/tektoncd/pipeline/pkg/git.run\n\tgithub.com/tektoncd/pipeline/pkg/git/git.go:54\ngithub.com/tektoncd/pipeline/pkg/git.Fetch\n\tgithub.com/tektoncd/pipeline/pkg/git/git.go:149\nmain.main\n\tgithub.com/tektoncd/pipeline/cmd/git-init/main.go:53\nruntime.main\n\truntime/proc.go:225"}
{"level":"fatal","ts":1633090656.5645268,"caller":"git-init/main.go:54","msg":"Error fetching git repository: failed to fetch [main]: exit status 128","stacktrace":"main.main\n\tgithub.com/tektoncd/pipeline/cmd/git-init/main.go:54\nruntime.main\n\truntime/proc.go:225"}My guess is that because user 65532's home dir isn't /tekton/creds it isn't picking up the private key? (i think this would be a great feature and i have a dev environment so i'm happy to help make this work)
afrittoli
commented
2 years ago Thanks @mlbiam - thanks for the updates and offer to help!
As a side note, it looks like you are using PipelineResources which we are likely to be deprecate.
I didn't really dive into this, but I wonder if the fact that we do not set the HOME variable by default anymore might have an impact here @sbwsg
mlbiam
commented
2 years ago @afrittoli
As a side note, it looks like you are using PipelineResources which we are likely to be deprecate.
Can you point me to some background? If PipelineResources is going away, what's the right approach? embedding the config directly into the PipelineRun?
ghost
commented
2 years ago Here's the rationale for PipelineResources being deprecated: https://github.com/tektoncd/community/blob/main/teps/0074-deprecate-pipelineresources.md#motivation
For cloning from a git repo there's a task in the catalog, git-clone, that accepts credentials directly as a workspace. You can pass in a k8s Secret here. When running as a non-root user you will likely also need to set the userHome parameter.
ghost
commented
2 years ago @mlbiam I'm going to close this issue on the basis that pipelineresources are likely to be deprecated. Recommend using the git-clone task from the catalog instead. Please feel free to reopen this issue if you'd like more help around this. Cheers!
/close
tekton-robot
commented
2 years ago @sbwsg: Closing this issue.
Expected Behavior
We are using GateKeeper to enforce pod security policies in a KinD cluster (1.21.1). A mutating webhook adds
runAsUser,runAsGroup, andfsUserto all containers and initContainers if one isn't present. When ourPipelinelaunches we expect everything to run.Actual Behavior
our pipelines show:
Steps to Reproduce the Problem
Additional Info
Kubernetes version:
Output of
kubectl version:Here's the complete
Podlisting: