Closed mtcolman closed 2 years ago
Based on the error messages it seems like there is an issue with the gcr registry that the example taskrun uses. If i change the registry to something else instead of gcr.io/foo/bar
it changes to chains.tekton.dev/signed: "true"
outputs:
- name: builtImage
resourceSpec:
type: image
params:
- name: url
value: gcr.io/foo/bar
Though it still fails the cosign verify-blob
Hey @mtcolman thanks for pointing this out! My guess is the tutorial needs to be updated and Chains is trying to upload the signature for the image to an OCI registry (which it doesn't have auth for, so it fails). But, it first stores the signature and payload on the TaskRun correctly, which is why verify-blob
passes.
Could you provide the output of:
kubectl describe cm -n tekton-chains chains-config
that should confirm my suspicion!
Hey @priyawadhwa I was having similar issues but my cosign verify-blob
also fails to verify. I am using the same taskrun given in the tutorial.
The chains controller is throwing error:
{"level":"error","ts":"2021-11-09T18:00:14.280Z","logger":"watcher","caller":"taskrun/reconciler.go:308","msg":"Returned an error","commit":"9bd7fb8","knative.dev/controller":"github.com.tektoncd.chains.pkg.reconciler.taskrun.Reconciler","knative.dev/kind":"tekton.dev.TaskRun","knative.dev/traceid":"bd8b68a1-f7ef-4893-8ffa-b017ab731dad","knative.dev/key":"default/build-push-run-output-image-bfxtg","targetMethod":"ReconcileKind","error":"1 error occurred:\n\t* uploading: GET https://gcr.io/v2/token?scope=repository%3Afoo%2Fbar%3Apull&service=gcr.io: UNKNOWN: Service 'containerregistry.googleapis.com' is not enabled for consumer 'project:foo'.\n\n","stacktrace":"github.com/tektoncd/chains/vendor/github.com/tektoncd/pipeline/pkg/client/injection/reconciler/pipeline/v1beta1/taskrun.(*reconcilerImpl).Reconcile\n\tgithub.com/tektoncd/chains/vendor/github.com/tektoncd/pipeline/pkg/client/injection/reconciler/pipeline/v1beta1/taskrun/reconciler.go:308\ngithub.com/tektoncd/chains/vendor/knative.dev/pkg/controller.(*Impl).processNextWorkItem\n\tgithub.com/tektoncd/chains/vendor/knative.dev/pkg/controller/controller.go:542\ngithub.com/tektoncd/chains/vendor/knative.dev/pkg/controller.(*Impl).RunContext.func3\n\tgithub.com/tektoncd/chains/vendor/knative.dev/pkg/controller/controller.go:478"}
{"level":"error","ts":"2021-11-09T18:00:14.285Z","logger":"watcher","caller":"controller/controller.go:566","msg":"Reconcile error","commit":"9bd7fb8","knative.dev/controller":"github.com.tektoncd.chains.pkg.reconciler.taskrun.Reconciler","knative.dev/kind":"tekton.dev.TaskRun","duration":0.2414034,"error":"1 error occurred:\n\t* uploading: GET https://gcr.io/v2/token?scope=repository%3Afoo%2Fbar%3Apull&service=gcr.io: UNKNOWN: Service 'containerregistry.googleapis.com' is not enabled for consumer 'project:foo'.\n\n","stacktrace":"github.com/tektoncd/chains/vendor/knative.dev/pkg/controller.(*Impl).handleErr\n\tgithub.com/tektoncd/chains/vendor/knative.dev/pkg/controller/controller.go:566\ngithub.com/tektoncd/chains/vendor/knative.dev/pkg/controller.(*Impl).processNextWorkItem\n\tgithub.com/tektoncd/chains/vendor/knative.dev/pkg/controller/controller.go:543\ngithub.com/tektoncd/chains/vendor/knative.dev/pkg/controller.(*Impl).RunContext.func3\n\tgithub.com/tektoncd/chains/vendor/knative.dev/pkg/controller/controller.go:478"}
{"level":"info","ts":"2021-11-09T18:00:14.285Z","logger":"watcher","caller":"taskrun/taskrun.go:57","msg":"taskrun default/build-push-run-output-image-bfxtg has been reconciled","commit":"9bd7fb8","knative.dev/controller":"github.com.tektoncd.chains.pkg.reconciler.taskrun.Reconciler","knative.dev/kind":"tekton.dev.TaskRun","knative.dev/traceid":"4ab64e15-4382-4b93-8782-cee8678cd3e5","knative.dev/key":"default/build-push-run-output-image-bfxtg"}
{"level":"info","ts":"2021-11-09T18:00:14.285Z","logger":"watcher","caller":"controller/controller.go:550","msg":"Reconcile succeeded","commit":"9bd7fb8","knative.dev/controller":"github.com.tektoncd.chains.pkg.reconciler.taskrun.Reconciler","knative.dev/kind":"tekton.dev.TaskRun","knative.dev/traceid":"4ab64e15-4382-4b93-8782-cee8678cd3e5","knative.dev/key":"default/build-push-run-output-image-bfxtg","duration":0.000104}
{"level":"info","ts":"2021-11-09T18:00:14.285Z","logger":"watcher.event-broadcaster","caller":"record/event.go:282","msg":"Event(v1.ObjectReference{Kind:\"TaskRun\", Namespace:\"default\", Name:\"build-push-run-output-image-bfxtg\", UID:\"6f0edb08-b19a-4106-8c3c-920e82d21a90\", APIVersion:\"tekton.dev/v1beta1\", ResourceVersion:\"373438\", FieldPath:\"\"}): type: 'Warning' reason: 'InternalError' 1 error occurred:\n\t* uploading: GET https://gcr.io/v2/token?scope=repository%3Afoo%2Fbar%3Apull&service=gcr.io: UNKNOWN: Service 'containerregistry.googleapis.com' is not enabled for consumer 'project:foo'.\n\n","commit":"9bd7fb8"}
Here is my configmap for chains:
Name: chains-config
Namespace: tekton-chains
Labels: app.kubernetes.io/component=chains
app.kubernetes.io/instance=default
app.kubernetes.io/part-of=tekton-pipelines
pipeline.tekton.dev/release=devel
version=v0.5.0
Annotations: <none>
Data
====
artifacts.taskrun.format:
----
tekton-provenance
artifacts.taskrun.storage:
----
tekton
Events: <none>
Cosgin Verify Error:
cosign verify-blob -key cosign.pub -signature ./signature ./payload
error: verifying blob [./payload]: failed to verify signature
Hey @mtcolman thanks for pointing this out! My guess is the tutorial needs to be updated and Chains is trying to upload the signature for the image to an OCI registry (which it doesn't have auth for, so it fails). But, it first stores the signature and payload on the TaskRun correctly, which is why
verify-blob
passes.Could you provide the output of:
kubectl describe cm -n tekton-chains chains-config
that should confirm my suspicion!
Hi, @priyawadhwa I subsequently ran through the in-toto tutorial, so my configmap says:
$ kubectl describe cm -n tekton-chains chains-config
Name: chains-config
Namespace: tekton-chains
Labels: app.kubernetes.io/component=chains
app.kubernetes.io/instance=default
app.kubernetes.io/part-of=tekton-pipelines
pipeline.tekton.dev/release=devel
version=v0.5.0
Annotations: <none>
Data
====
artifacts.taskrun.storage:
----
oci
transparency.enabled:
----
true
artifacts.taskrun.format:
----
tekton-provenance
Events: <none>
But yes, I haven't set anything up for pushing to repo in the first tutorial, so I agree with you.
Hey @priyawadhwa I was having similar issues but my
cosign verify-blob
also fails to verify. I am using the same taskrun given in the tutorial.The chains controller is throwing error:
{"level":"error","ts":"2021-11-09T18:00:14.280Z","logger":"watcher","caller":"taskrun/reconciler.go:308","msg":"Returned an error","commit":"9bd7fb8","knative.dev/controller":"github.com.tektoncd.chains.pkg.reconciler.taskrun.Reconciler","knative.dev/kind":"tekton.dev.TaskRun","knative.dev/traceid":"bd8b68a1-f7ef-4893-8ffa-b017ab731dad","knative.dev/key":"default/build-push-run-output-image-bfxtg","targetMethod":"ReconcileKind","error":"1 error occurred:\n\t* uploading: GET https://gcr.io/v2/token?scope=repository%3Afoo%2Fbar%3Apull&service=gcr.io: UNKNOWN: Service 'containerregistry.googleapis.com' is not enabled for consumer 'project:foo'.\n\n","stacktrace":"github.com/tektoncd/chains/vendor/github.com/tektoncd/pipeline/pkg/client/injection/reconciler/pipeline/v1beta1/taskrun.(*reconcilerImpl).Reconcile\n\tgithub.com/tektoncd/chains/vendor/github.com/tektoncd/pipeline/pkg/client/injection/reconciler/pipeline/v1beta1/taskrun/reconciler.go:308\ngithub.com/tektoncd/chains/vendor/knative.dev/pkg/controller.(*Impl).processNextWorkItem\n\tgithub.com/tektoncd/chains/vendor/knative.dev/pkg/controller/controller.go:542\ngithub.com/tektoncd/chains/vendor/knative.dev/pkg/controller.(*Impl).RunContext.func3\n\tgithub.com/tektoncd/chains/vendor/knative.dev/pkg/controller/controller.go:478"} {"level":"error","ts":"2021-11-09T18:00:14.285Z","logger":"watcher","caller":"controller/controller.go:566","msg":"Reconcile error","commit":"9bd7fb8","knative.dev/controller":"github.com.tektoncd.chains.pkg.reconciler.taskrun.Reconciler","knative.dev/kind":"tekton.dev.TaskRun","duration":0.2414034,"error":"1 error occurred:\n\t* uploading: GET https://gcr.io/v2/token?scope=repository%3Afoo%2Fbar%3Apull&service=gcr.io: UNKNOWN: Service 'containerregistry.googleapis.com' is not enabled for consumer 'project:foo'.\n\n","stacktrace":"github.com/tektoncd/chains/vendor/knative.dev/pkg/controller.(*Impl).handleErr\n\tgithub.com/tektoncd/chains/vendor/knative.dev/pkg/controller/controller.go:566\ngithub.com/tektoncd/chains/vendor/knative.dev/pkg/controller.(*Impl).processNextWorkItem\n\tgithub.com/tektoncd/chains/vendor/knative.dev/pkg/controller/controller.go:543\ngithub.com/tektoncd/chains/vendor/knative.dev/pkg/controller.(*Impl).RunContext.func3\n\tgithub.com/tektoncd/chains/vendor/knative.dev/pkg/controller/controller.go:478"} {"level":"info","ts":"2021-11-09T18:00:14.285Z","logger":"watcher","caller":"taskrun/taskrun.go:57","msg":"taskrun default/build-push-run-output-image-bfxtg has been reconciled","commit":"9bd7fb8","knative.dev/controller":"github.com.tektoncd.chains.pkg.reconciler.taskrun.Reconciler","knative.dev/kind":"tekton.dev.TaskRun","knative.dev/traceid":"4ab64e15-4382-4b93-8782-cee8678cd3e5","knative.dev/key":"default/build-push-run-output-image-bfxtg"} {"level":"info","ts":"2021-11-09T18:00:14.285Z","logger":"watcher","caller":"controller/controller.go:550","msg":"Reconcile succeeded","commit":"9bd7fb8","knative.dev/controller":"github.com.tektoncd.chains.pkg.reconciler.taskrun.Reconciler","knative.dev/kind":"tekton.dev.TaskRun","knative.dev/traceid":"4ab64e15-4382-4b93-8782-cee8678cd3e5","knative.dev/key":"default/build-push-run-output-image-bfxtg","duration":0.000104} {"level":"info","ts":"2021-11-09T18:00:14.285Z","logger":"watcher.event-broadcaster","caller":"record/event.go:282","msg":"Event(v1.ObjectReference{Kind:\"TaskRun\", Namespace:\"default\", Name:\"build-push-run-output-image-bfxtg\", UID:\"6f0edb08-b19a-4106-8c3c-920e82d21a90\", APIVersion:\"tekton.dev/v1beta1\", ResourceVersion:\"373438\", FieldPath:\"\"}): type: 'Warning' reason: 'InternalError' 1 error occurred:\n\t* uploading: GET https://gcr.io/v2/token?scope=repository%3Afoo%2Fbar%3Apull&service=gcr.io: UNKNOWN: Service 'containerregistry.googleapis.com' is not enabled for consumer 'project:foo'.\n\n","commit":"9bd7fb8"}
Here is my configmap for chains:
Name: chains-config Namespace: tekton-chains Labels: app.kubernetes.io/component=chains app.kubernetes.io/instance=default app.kubernetes.io/part-of=tekton-pipelines pipeline.tekton.dev/release=devel version=v0.5.0 Annotations: <none> Data ==== artifacts.taskrun.format: ---- tekton-provenance artifacts.taskrun.storage: ---- tekton Events: <none>
Cosgin Verify Error:
cosign verify-blob -key cosign.pub -signature ./signature ./payload error: verifying blob [./payload]: failed to verify signature
Figured out my issue was that i was using an old version of cosign. Once I updated it solved the issue.
@pxp928 glad it's working again! @mtcolman yup looks like it's trying to store the signature in oci -- i think if you change the storage back to tekton
it should work --
kubectl patch configmap chains-config -n tekton-chains -p='{"data":{"artifacts.taskrun.storage": "tekton"}}'
let me know if you still see errors!
@priyawadhwa I think I might still have this issue...
I'm running the "getting started" tutorial.
And this is the current state of tekton-chains configmap "chains-config" (i.e. storage is Tekton):
kubectl describe cm -n tekton-chains chains-config
Name: chains-config
Namespace: tekton-chains
Labels: app.kubernetes.io/component=chains
app.kubernetes.io/instance=default
app.kubernetes.io/part-of=tekton-pipelines
pipeline.tekton.dev/release=devel
version=v0.5.0
Annotations: <none>
Data
====
artifacts.taskrun.format:
----
tekton-provenance
artifacts.taskrun.storage:
----
tekton
transparency.enabled:
----
true
Events: <none>
I've just run: kubectl create -f https://raw.githubusercontent.com/tektoncd/chains/main/examples/taskruns/task-output-image.yaml
and it tells me taskrun.tekton.dev/build-push-run-output-image-s98hw created
.
So I use the taskRun ID in this command:
oc get tr build-push-run-output-image-s98hw -o=json | jq -r ".metadata.annotations[\"chains.tekton.dev/signed\"]"
failed
And it informs me the status is failed
. ("chains.tekton.dev/signed": "failed"
).
Here's the full output of annotations
:
oc get tr build-push-run-output-image-s98hw -o=json | jq -r ".metadata.annotations"
{
"chains.tekton.dev/cert-taskrun-1ba06026-0865-4c02-94cf-3e5bdf192107": "",
"chains.tekton.dev/chain-taskrun-1ba06026-0865-4c02-94cf-3e5bdf192107": "",
"chains.tekton.dev/payload-taskrun-1ba06026-0865-4c02-94cf-3e5bdf192107": "<removed for brevity>",
"chains.tekton.dev/retries": "3",
"chains.tekton.dev/signature-taskrun-1ba06026-0865-4c02-94cf-3e5bdf192107": "<removed for brevity>",
"chains.tekton.dev/signed": "failed",
"chains.tekton.dev/transparency": "https://rekor.sigstore.dev/858199",
"pipeline.tekton.dev/release": "v0.22.0"
}
And here's me double checking I'm definitely looking at the taskRun I've just asked to be created:
oc get tr build-push-run-output-image-s98hw
NAME SUCCEEDED REASON STARTTIME COMPLETIONTIME
build-push-run-output-image-s98hw True Succeeded 9m16s 8m51s
This is the output of oc get events
:
oc get events
LAST SEEN TYPE REASON OBJECT MESSAGE
20m Normal Scheduled pod/build-push-run-output-image-s98hw-pod-5tnzn Successfully assigned tekton-chains/build-push-run-output-image-s98hw-pod-5tnzn to 10.112.78.35
20m Normal AddedInterface pod/build-push-run-output-image-s98hw-pod-5tnzn Add eth0 [172.30.224.163/32]
20m Normal Pulled pod/build-push-run-output-image-s98hw-pod-5tnzn Container image "registry.access.redhat.com/ubi8/ubi-minimal@sha256:fdfb0770bff33e0f97d78583efd68b546a19d0a4b0ac23eef25ef261bca3e975" already present on machine
20m Normal Created pod/build-push-run-output-image-s98hw-pod-5tnzn Created container place-scripts
20m Normal Started pod/build-push-run-output-image-s98hw-pod-5tnzn Started container place-scripts
20m Normal Pulled pod/build-push-run-output-image-s98hw-pod-5tnzn Container image "registry.redhat.io/openshift-pipelines/pipelines-entrypoint-rhel8@sha256:4989b910b5beae615a360deb31c8a4951e7b867a93374dd29d42815cf3480cc0" already present on machine
20m Normal Created pod/build-push-run-output-image-s98hw-pod-5tnzn Created container place-tools
20m Normal Started pod/build-push-run-output-image-s98hw-pod-5tnzn Started container place-tools
20m Normal Pulled pod/build-push-run-output-image-s98hw-pod-5tnzn Container image "registry.access.redhat.com/ubi8/ubi-minimal@sha256:fdfb0770bff33e0f97d78583efd68b546a19d0a4b0ac23eef25ef261bca3e975" already present on machine
20m Normal Created pod/build-push-run-output-image-s98hw-pod-5tnzn Created container step-create-dir-builtimage-ftmmv
20m Normal Started pod/build-push-run-output-image-s98hw-pod-5tnzn Started container step-create-dir-builtimage-ftmmv
20m Normal Pulled pod/build-push-run-output-image-s98hw-pod-5tnzn Container image "registry.redhat.io/openshift-pipelines/pipelines-git-init-rhel8@sha256:afc5d3f9efe26c7042635d43b8ffd09d67936e3d0b6b901dc08a33e20313d361" already present on machine
20m Normal Created pod/build-push-run-output-image-s98hw-pod-5tnzn Created container step-git-source-sourcerepo-xcsrp
20m Normal Started pod/build-push-run-output-image-s98hw-pod-5tnzn Started container step-git-source-sourcerepo-xcsrp
20m Normal Pulling pod/build-push-run-output-image-s98hw-pod-5tnzn Pulling image "busybox"
20m Normal Pulled pod/build-push-run-output-image-s98hw-pod-5tnzn Successfully pulled image "busybox" in 4.555106292s
20m Normal Created pod/build-push-run-output-image-s98hw-pod-5tnzn Created container step-build-and-push
20m Normal Started pod/build-push-run-output-image-s98hw-pod-5tnzn Started container step-build-and-push
20m Normal Pulling pod/build-push-run-output-image-s98hw-pod-5tnzn Pulling image "busybox"
20m Normal Pulled pod/build-push-run-output-image-s98hw-pod-5tnzn Successfully pulled image "busybox" in 1.714015821s
20m Normal Created pod/build-push-run-output-image-s98hw-pod-5tnzn Created container step-echo
20m Normal Started pod/build-push-run-output-image-s98hw-pod-5tnzn Started container step-echo
20m Normal Pulled pod/build-push-run-output-image-s98hw-pod-5tnzn Container image "registry.redhat.io/openshift-pipelines/pipelines-imagedigestexporter-rhel8@sha256:1c8ef68a540856de81657b47c28c632f43030a2566229b0c22c5e6ab0a5adce3" already present on machine
20m Normal Created pod/build-push-run-output-image-s98hw-pod-5tnzn Created container step-image-digest-exporter-r2s2l
20m Normal Started pod/build-push-run-output-image-s98hw-pod-5tnzn Started container step-image-digest-exporter-r2s2l
20m Normal Started taskrun/build-push-run-output-image-s98hw
20m Normal FinalizerUpdate taskrun/build-push-run-output-image-s98hw Updated "build-push-run-output-image-s98hw" finalizers
20m Normal Pending taskrun/build-push-run-output-image-s98hw Pending
20m Normal Pending taskrun/build-push-run-output-image-s98hw pod status "Initialized":"False"; message: "containers with incomplete status: [place-scripts place-tools]"
20m Normal Pending taskrun/build-push-run-output-image-s98hw pod status "Initialized":"False"; message: "containers with incomplete status: [place-tools]"
20m Normal Pending taskrun/build-push-run-output-image-s98hw pod status "Ready":"False"; message: "containers with unready status: [step-create-dir-builtimage-ftmmv step-git-source-sourcerepo-xcsrp step-build-and-push step-echo step-image-digest-exporter-r2s2l]"
20m Normal Running taskrun/build-push-run-output-image-s98hw Not all Steps in the Task have finished executing
19m Normal Succeeded taskrun/build-push-run-output-image-s98hw All Steps have completed executing
19m Warning InternalError taskrun/build-push-run-output-image-s98hw 1 error occurred:
* uploading: GET https://gcr.io/v2/token?scope=repository%3Afoo%2Fbar%3Apull&service=gcr.io: UNKNOWN: Service 'containerregistry.googleapis.com' is not enabled for consumer 'project:foo'.
Note the error:
InternalError taskrun/build-push-run-output-image-s98hw 1 error occurred:
* uploading: GET https://gcr.io/v2/token?scope=repository%3Afoo%2Fbar%3Apull&service=gcr.io: UNKNOWN: Service 'containerregistry.googleapis.com' is not enabled for consumer 'project:foo'.
And the taskRun configuration:
outputs:
- name: builtImage
resourceSpec:
type: image
params:
- name: url
value: gcr.io/foo/bar
tekton logs of taskRun doesn't seem to flag anything untoward:
tkn tr logs build-push-run-output-image-s98hw
[git-source-sourcerepo-xcsrp] {"level":"info","ts":1636970808.014247,"caller":"git/git.go:169","msg":"Successfully cloned https://github.com/GoogleContainerTools/skaffold @ 6ed7aad5e8a36052ee5f6079fc91368e362121f7 (grafted, HEAD) in path /workspace/sourcerepo"}
[git-source-sourcerepo-xcsrp] {"level":"info","ts":1636970808.6034794,"caller":"git/git.go:207","msg":"Successfully initialized and updated submodules in path /workspace/sourcerepo"}
[build-and-push] + set -e
[build-and-push] + cat
[echo] + cat /workspace/sourcerepo/index.json
[echo] {
[echo] "schemaVersion": 2,
[echo] "manifests": [
[echo] {
[echo] "mediaType": "application/vnd.oci.image.index.v1+json",
[echo] "size": 314,
[echo] "digest": "sha256:05f95b26ed10668b7183c1e2da98610e91372fa9f510046d4ce5812addad86b5"
[echo] }
[echo] ]
[echo] }
[echo]
[image-digest-exporter-r2s2l] {"severity":"INFO","timestamp":"2021-11-15T10:06:50.254370252Z","caller":"logging/config.go:116","message":"Successfully created the logger."}
[image-digest-exporter-r2s2l] {"severity":"INFO","timestamp":"2021-11-15T10:06:50.254978848Z","caller":"logging/config.go:117","message":"Logging level set to: info"}
[image-digest-exporter-r2s2l] {"severity":"INFO","timestamp":"2021-11-15T10:06:50.255141162Z","caller":"logging/config.go:79","message":"Fetch GitHub commit ID from kodata failed","error":"\"KO_DATA_PATH\" does not exist or is empty"}
And here is some output from oc logs tekton-chains-controller-7f94b7fc57-zgfpl
:
{"level":"info","ts":"2021-11-15T10:06:50.227Z","logger":"watcher","caller":"taskrun/taskrun.go:52","msg":"taskrun tekton-chains/build-push-run-output-image-s98hw is still running","commit":"9bd7fb8","knative.dev/controller":"github.com.tektoncd.chains.pkg.reconciler.taskrun.Reconciler","knative.dev/kind":"tekton.dev.TaskRun","knative.dev/traceid":"826be2ed-43da-4724-9da1-a8dc3e01b4b0","knative.dev/key":"tekton-chains/build-push-run-output-image-s98hw"}
{"level":"info","ts":"2021-11-15T10:06:50.227Z","logger":"watcher","caller":"controller/controller.go:550","msg":"Reconcile succeeded","commit":"9bd7fb8","knative.dev/controller":"github.com.tektoncd.chains.pkg.reconciler.taskrun.Reconciler","knative.dev/kind":"tekton.dev.TaskRun","knative.dev/traceid":"826be2ed-43da-4724-9da1-a8dc3e01b4b0","knative.dev/key":"tekton-chains/build-push-run-output-image-s98hw","duration":0.000481135}
{"level":"info","ts":"2021-11-15T10:06:51.416Z","logger":"watcher","caller":"x509/x509.go:113","msg":"Found cosign key...","commit":"9bd7fb8","knative.dev/controller":"github.com.tektoncd.chains.pkg.reconciler.taskrun.Reconciler","knative.dev/kind":"tekton.dev.TaskRun","knative.dev/traceid":"6e1152d6-15e4-491d-8c0a-0ecf34d678a3","knative.dev/key":"tekton-chains/build-push-run-output-image-s98hw"}
{"level":"warn","ts":"2021-11-15T10:06:52.291Z","logger":"watcher","caller":"chains/signing.go:68","msg":"error configuring kms signer with config {}: no provider found for that key reference","commit":"9bd7fb8","knative.dev/controller":"github.com.tektoncd.chains.pkg.reconciler.taskrun.Reconciler","knative.dev/kind":"tekton.dev.TaskRun","knative.dev/traceid":"6e1152d6-15e4-491d-8c0a-0ecf34d678a3","knative.dev/key":"tekton-chains/build-push-run-output-image-s98hw"}
{"level":"info","ts":"2021-11-15T10:06:52.306Z","logger":"watcher","caller":"chains/signing.go:163","msg":"Created payload of type tekton-provenance for TaskRun tekton-chains/build-push-run-output-image-s98hw","commit":"9bd7fb8","knative.dev/controller":"github.com.tektoncd.chains.pkg.reconciler.taskrun.Reconciler","knative.dev/kind":"tekton.dev.TaskRun","knative.dev/traceid":"6e1152d6-15e4-491d-8c0a-0ecf34d678a3","knative.dev/key":"tekton-chains/build-push-run-output-image-s98hw"}
{"level":"info","ts":"2021-11-15T10:06:52.314Z","logger":"watcher","caller":"chains/signing.go:178","msg":"Using wrapped envelope signer for tekton-provenance","commit":"9bd7fb8","knative.dev/controller":"github.com.tektoncd.chains.pkg.reconciler.taskrun.Reconciler","knative.dev/kind":"tekton.dev.TaskRun","knative.dev/traceid":"6e1152d6-15e4-491d-8c0a-0ecf34d678a3","knative.dev/key":"tekton-chains/build-push-run-output-image-s98hw"}
{"level":"info","ts":"2021-11-15T10:06:52.314Z","logger":"watcher","caller":"chains/signing.go:182","msg":"Signing object with x509","commit":"9bd7fb8","knative.dev/controller":"github.com.tektoncd.chains.pkg.reconciler.taskrun.Reconciler","knative.dev/kind":"tekton.dev.TaskRun","knative.dev/traceid":"6e1152d6-15e4-491d-8c0a-0ecf34d678a3","knative.dev/key":"tekton-chains/build-push-run-output-image-s98hw"}
{"level":"info","ts":"2021-11-15T10:06:52.387Z","logger":"watcher","caller":"tekton/tekton.go:58","msg":"Storing payload on TaskRun tekton-chains/build-push-run-output-image-s98hw","commit":"9bd7fb8","knative.dev/controller":"github.com.tektoncd.chains.pkg.reconciler.taskrun.Reconciler","knative.dev/kind":"tekton.dev.TaskRun","knative.dev/traceid":"6e1152d6-15e4-491d-8c0a-0ecf34d678a3","knative.dev/key":"tekton-chains/build-push-run-output-image-s98hw"}
{"level":"info","ts":"2021-11-15T10:06:53.750Z","logger":"watcher","caller":"chains/signing.go:214","msg":"Uploaded entry to https://rekor.sigstore.dev with index 858184","commit":"9bd7fb8","knative.dev/controller":"github.com.tektoncd.chains.pkg.reconciler.taskrun.Reconciler","knative.dev/kind":"tekton.dev.TaskRun","knative.dev/traceid":"6e1152d6-15e4-491d-8c0a-0ecf34d678a3","knative.dev/key":"tekton-chains/build-push-run-output-image-s98hw"}
{"level":"info","ts":"2021-11-15T10:06:53.753Z","logger":"watcher","caller":"chains/signing.go:163","msg":"Created payload of type simplesigning for TaskRun tekton-chains/build-push-run-output-image-s98hw","commit":"9bd7fb8","knative.dev/controller":"github.com.tektoncd.chains.pkg.reconciler.taskrun.Reconciler","knative.dev/kind":"tekton.dev.TaskRun","knative.dev/traceid":"6e1152d6-15e4-491d-8c0a-0ecf34d678a3","knative.dev/key":"tekton-chains/build-push-run-output-image-s98hw"}
{"level":"info","ts":"2021-11-15T10:06:53.753Z","logger":"watcher","caller":"chains/signing.go:182","msg":"Signing object with x509","commit":"9bd7fb8","knative.dev/controller":"github.com.tektoncd.chains.pkg.reconciler.taskrun.Reconciler","knative.dev/kind":"tekton.dev.TaskRun","knative.dev/traceid":"6e1152d6-15e4-491d-8c0a-0ecf34d678a3","knative.dev/key":"tekton-chains/build-push-run-output-image-s98hw"}
{"level":"info","ts":"2021-11-15T10:06:53.754Z","logger":"watcher","caller":"oci/oci.go:72","msg":"Storing payload on TaskRun tekton-chains/build-push-run-output-image-s98hw","commit":"9bd7fb8","knative.dev/controller":"github.com.tektoncd.chains.pkg.reconciler.taskrun.Reconciler","knative.dev/kind":"tekton.dev.TaskRun","knative.dev/traceid":"6e1152d6-15e4-491d-8c0a-0ecf34d678a3","knative.dev/key":"tekton-chains/build-push-run-output-image-s98hw"}
{"level":"info","ts":"2021-11-15T10:06:53.756Z","logger":"watcher","caller":"oci/oci.go:96","msg":"Uploading gcr.io/foo/bar@sha256:05f95b26ed10668b7183c1e2da98610e91372fa9f510046d4ce5812addad86b5 signature","commit":"9bd7fb8","knative.dev/controller":"github.com.tektoncd.chains.pkg.reconciler.taskrun.Reconciler","knative.dev/kind":"tekton.dev.TaskRun","knative.dev/traceid":"6e1152d6-15e4-491d-8c0a-0ecf34d678a3","knative.dev/key":"tekton-chains/build-push-run-output-image-s98hw"}
E1115 10:06:53.801975 1 config.go:141] while trying to canonicalize : stat .: permission denied
E1115 10:06:53.802865 1 config.go:141] while trying to canonicalize : stat .: permission denied
E1115 10:06:53.803699 1 config.go:107] while trying to canonicalize : stat .: permission denied
E1115 10:06:53.803976 1 config.go:107] while trying to canonicalize : stat .: permission denied
{"level":"error","ts":"2021-11-15T10:06:54.478Z","logger":"watcher","caller":"chains/signing.go:204","msg":"uploading: GET https://gcr.io/v2/token?scope=repository%3Afoo%2Fbar%3Apull&service=gcr.io: UNKNOWN: Service 'containerregistry.googleapis.com' is not enabled for consumer 'project:foo'.","commit":"9bd7fb8","knative.dev/controller":"github.com.tektoncd.chains.pkg.reconciler.taskrun.Reconciler","knative.dev/kind":"tekton.dev.TaskRun","knative.dev/traceid":"6e1152d6-15e4-491d-8c0a-0ecf34d678a3","knative.dev/key":"tekton-chains/build-push-run-output-image-s98hw","stacktrace":"github.com/tektoncd/chains/pkg/chains.(*TaskRunSigner).SignTaskRun\n\tgithub.com/tektoncd/chains/pkg/chains/signing.go:204\ngithub.com/tektoncd/chains/pkg/reconciler/taskrun.(*Reconciler).FinalizeKind\n\tgithub.com/tektoncd/chains/pkg/reconciler/taskrun/taskrun.go:61\ngithub.com/tektoncd/chains/pkg/reconciler/taskrun.(*Reconciler).ReconcileKind\n\tgithub.com/tektoncd/chains/pkg/reconciler/taskrun/taskrun.go:42\ngithub.com/tektoncd/chains/vendor/github.com/tektoncd/pipeline/pkg/client/injection/reconciler/pipeline/v1beta1/taskrun.(*reconcilerImpl).Reconcile\n\tgithub.com/tektoncd/chains/vendor/github.com/tektoncd/pipeline/pkg/client/injection/reconciler/pipeline/v1beta1/taskrun/reconciler.go:249\ngithub.com/tektoncd/chains/vendor/knative.dev/pkg/controller.(*Impl).processNextWorkItem\n\tgithub.com/tektoncd/chains/vendor/knative.dev/pkg/controller/controller.go:542\ngithub.com/tektoncd/chains/vendor/knative.dev/pkg/controller.(*Impl).RunContext.func3\n\tgithub.com/tektoncd/chains/vendor/knative.dev/pkg/controller/controller.go:478"}
{"level":"info","ts":"2021-11-15T10:06:54.870Z","logger":"watcher","caller":"chains/signing.go:214","msg":"Uploaded entry to https://rekor.sigstore.dev with index 858185","commit":"9bd7fb8","knative.dev/controller":"github.com.tektoncd.chains.pkg.reconciler.taskrun.Reconciler","knative.dev/kind":"tekton.dev.TaskRun","knative.dev/traceid":"6e1152d6-15e4-491d-8c0a-0ecf34d678a3","knative.dev/key":"tekton-chains/build-push-run-output-image-s98hw"}
{"level":"error","ts":"2021-11-15T10:06:55.027Z","logger":"watcher","caller":"taskrun/reconciler.go:308","msg":"Returned an error","commit":"9bd7fb8","knative.dev/controller":"github.com.tektoncd.chains.pkg.reconciler.taskrun.Reconciler","knative.dev/kind":"tekton.dev.TaskRun","knative.dev/traceid":"6e1152d6-15e4-491d-8c0a-0ecf34d678a3","knative.dev/key":"tekton-chains/build-push-run-output-image-s98hw","targetMethod":"ReconcileKind","error":"1 error occurred:\n\t* uploading: GET https://gcr.io/v2/token?scope=repository%3Afoo%2Fbar%3Apull&service=gcr.io: UNKNOWN: Service 'containerregistry.googleapis.com' is not enabled for consumer 'project:foo'.\n\n","stacktrace":"github.com/tektoncd/chains/vendor/github.com/tektoncd/pipeline/pkg/client/injection/reconciler/pipeline/v1beta1/taskrun.(*reconcilerImpl).Reconcile\n\tgithub.com/tektoncd/chains/vendor/github.com/tektoncd/pipeline/pkg/client/injection/reconciler/pipeline/v1beta1/taskrun/reconciler.go:308\ngithub.com/tektoncd/chains/vendor/knative.dev/pkg/controller.(*Impl).processNextWorkItem\n\tgithub.com/tektoncd/chains/vendor/knative.dev/pkg/controller/controller.go:542\ngithub.com/tektoncd/chains/vendor/knative.dev/pkg/controller.(*Impl).RunContext.func3\n\tgithub.com/tektoncd/chains/vendor/knative.dev/pkg/controller/controller.go:478"}
{"level":"info","ts":"2021-11-15T10:06:55.028Z","logger":"watcher.event-broadcaster","caller":"record/event.go:282","msg":"Event(v1.ObjectReference{Kind:\"TaskRun\", Namespace:\"tekton-chains\", Name:\"build-push-run-output-image-s98hw\", UID:\"1ba06026-0865-4c02-94cf-3e5bdf192107\", APIVersion:\"tekton.dev/v1beta1\", ResourceVersion:\"311229752\", FieldPath:\"\"}): type: 'Warning' reason: 'InternalError' 1 error occurred:\n\t* uploading: GET https://gcr.io/v2/token?scope=repository%3Afoo%2Fbar%3Apull&service=gcr.io: UNKNOWN: Service 'containerregistry.googleapis.com' is not enabled for consumer 'project:foo'.\n\n","commit":"9bd7fb8"}
{"level":"error","ts":"2021-11-15T10:06:55.028Z","logger":"watcher","caller":"controller/controller.go:566","msg":"Reconcile error","commit":"9bd7fb8","knative.dev/controller":"github.com.tektoncd.chains.pkg.reconciler.taskrun.Reconciler","knative.dev/kind":"tekton.dev.TaskRun","duration":3.751908598,"error":"1 error occurred:\n\t* uploading: GET https://gcr.io/v2/token?scope=repository%3Afoo%2Fbar%3Apull&service=gcr.io: UNKNOWN: Service 'containerregistry.googleapis.com' is not enabled for consumer 'project:foo'.\n\n","stacktrace":"github.com/tektoncd/chains/vendor/knative.dev/pkg/controller.(*Impl).handleErr\n\tgithub.com/tektoncd/chains/vendor/knative.dev/pkg/controller/controller.go:566\ngithub.com/tektoncd/chains/vendor/knative.dev/pkg/controller.(*Impl).processNextWorkItem\n\tgithub.com/tektoncd/chains/vendor/knative.dev/pkg/controller/controller.go:543\ngithub.com/tektoncd/chains/vendor/knative.dev/pkg/controller.(*Impl).RunContext.func3\n\tgithub.com/tektoncd/chains/vendor/knative.dev/pkg/controller/controller.go:478"}
{"level":"info","ts":"2021-11-15T10:06:55.174Z","logger":"watcher","caller":"x509/x509.go:113","msg":"Found cosign key...","commit":"9bd7fb8","knative.dev/controller":"github.com.tektoncd.chains.pkg.reconciler.taskrun.Reconciler","knative.dev/kind":"tekton.dev.TaskRun","knative.dev/traceid":"1870f742-9602-41f7-a665-076e961362fb","knative.dev/key":"tekton-chains/build-push-run-output-image-s98hw"}
{"level":"warn","ts":"2021-11-15T10:06:55.479Z","logger":"watcher","caller":"chains/signing.go:68","msg":"error configuring kms signer with config {}: no provider found for that key reference","commit":"9bd7fb8","knative.dev/controller":"github.com.tektoncd.chains.pkg.reconciler.taskrun.Reconciler","knative.dev/kind":"tekton.dev.TaskRun","knative.dev/traceid":"1870f742-9602-41f7-a665-076e961362fb","knative.dev/key":"tekton-chains/build-push-run-output-image-s98hw"}
Hey @mtcolman thanks for all the info! Chains is still trying to upload to oci
even though you haven't specified that you want that in the config because it is still assuming the oci config defaults.
We probably need some way to tell Chains to ignore OCI artifacts altogether (or any type of artifact really!). Maybe adding a artifacts.taskrun.enabled
and an artifact.oci.enabled
field. Then in the tutorial we could set artifact.oci.enabled=false
and this error shouldn't happen.
I'll go ahead and change the name of this issue to reflect the fix we'd need!
@priyawadhwa I can work on fixing this issue if no one else has started yet.
@pxp928 thank you! I'll assign you :)
@pxp928 now that #302 is merged, can we update the tutorials to reflect this?
@concaf the config document: https://github.com/tektoncd/chains/blob/main/docs/config.md and the tutorials: https://github.com/tektoncd/chains/tree/main/docs/tutorials were both updated as part of the PR. Is there another location that I missed?
@pxp928 ah! I see that the getting started tutorial was updated. I was looking for this in the signed provenance tutorial, shouldn't that be updated as well? 🤔
@pxp928 ah! I see that the getting started tutorial was updated. I was looking for this in the signed provenance tutorial, shouldn't that be updated as well? 🤔
@concaf That will still work for that tutorial.
@concaf @pxp928 I believe this issue can be closed now, can you confirm?
@priyawadhwa I agree it can be closed.
@priyawadhwa sounds good! 👍🏼
Hi, I'm not sure if this is a bug or not, so raising in free-form, apologies if I've used the wrong thing.
I've followed the tutorial here: https://github.com/tektoncd/chains/blob/main/docs/tutorials/getting-started-tutorial.md
And at the end of it I get:
However, when having a look around with:
I can see that :
Shows:
Has something gone wrong here? And if so, why did the
cosign verify-blob
returnVerified OK
?Thanks!