Closed jwcesign closed 3 years ago
julz
commented
3 years ago This is very weird! Do you also see the domain mapping controller pod failing to start or it only the docker images output? (what does kubectl get pods -nknative-serving show?)
julz
commented
3 years ago Could you also confirm that you're running at HEAD of knative (what does git rev-parse HEAD say?)
jwcesign
commented
3 years ago kubectl get pods show ImagePullBackOff, the error like(here is queue img, same with domain-mapping)：
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 3m49s default-scheduler Successfully assigned default/helloworld-go-00001-deployment-575b7cb9fd-fvjmk to minikube
Normal Pulled 3m48s kubelet Container image "dev.local/knative-samples/helloworld-go:latest" already present on machine
Normal Created 3m48s kubelet Created container user-container
Normal Started 3m48s kubelet Started container user-container
Normal Pulling 82s (x3 over 3m48s) kubelet Pulling image "ko.local/queue-39be6f1d08a095bd076a71d288d295b6:71f73134640f50b0601424a65bd75e01862ae9c04193ba7a4a3f6f4821631498"
Warning Failed 37s (x3 over 2m48s) kubelet Failed to pull image "ko.local/queue-39be6f1d08a095bd076a71d288d295b6:71f73134640f50b0601424a65bd75e01862ae9c04193ba7a4a3f6f4821631498": rpc error: code = Unknown desc = Error response from daemon: Get https://ko.local/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
Warning Failed 37s (x3 over 2m48s) kubelet Error: ErrImagePull
Normal BackOff 1s (x5 over 2m47s) kubelet Back-off pulling image "ko.local/queue-39be6f1d08a095bd076a71d288d295b6:71f73134640f50b0601424a65bd75e01862ae9c04193ba7a4a3f6f4821631498"
Warning Failed 1s (x5 over 2m47s) kubelet Error: ImagePullBackOfffor serving repo, I just clone it. nothing did. Here is the output.
jw@cci-network-0003:~/workspace/src/knative.dev/serving$ git rev-parse HEA
HEA
fatal: ambiguous argument 'HEA': unknown revision or path not in the working tree.
Use '--' to separate paths from revisions, like this:
'git <command> [<revision>...] -- [<file>...]'
jw@cci-network-0003:~/workspace/src/knative.dev/serving$does it have connection with ko delete --ignore-not-found=true -Rf config/core/? I use ctrl+c to stop this, because this sometimes stop at some steps so long time.
julz
commented
3 years ago does it have connection with ko delete --ignore-not-found=true -Rf config/core/? I use ctrl+c to stop this, because this sometimes stop at some steps so long time.
It's a rather strange bug, so I wouldn't be surprised if this was the problem. It's possible deleting half-way could have got things in to an odd state where the sha is still there in docker (so ko doesn't try to recreate the image) but the image metadata isn't. This might be more of an issue to discuss with http://github.com/google/ko, though. It's possible docker image prune -a may clean up dangling references, allowing the next run of ko to work properly, if this is indeed the problem.
jwcesign
commented
3 years ago @julz I will try docker image prune -a if I meet the situation again.
when I exec
ko apply -Rf config/core/, here is the output:I can find in the output log:
but when I exec
docker imagesto check, there isn't this image. other works ok.How to fix this?