search

spidernet-io / spiderpool

Underlay and RDMA network solution of the Kubernetes, for bare metal, VM and any public cloud
https://spidernet-io.github.io/spiderpool/
Apache License 2.0
523 stars 76 forks source link

Night CI 2024-02-05: Failed #3191

Closed github-actions[bot] closed 7 months ago

github-actions[bot] commented 7 months ago

action url: https://github.com/spidernet-io/spiderpool/actions/runs/7790199835

ty-dc commented 7 months ago 
error, found error log !!!
failed to run e2e test
ty-dc commented 7 months ago 
I0205 20:13:58.736380       1 alloc.go:330] "allocated clusterIPs" service="kube-system/spiderpool-agent" clusterIPs={"IPv4":"10.233.33.203"}
I0205 20:13:58.737200       1 cidrallocator.go:181] Finished sync for CIDRs took 9.157µs
I0205 20:13:58.737194       1 cidrallocator.go:181] Finished sync for CIDRs took 7.183µs
I0205 20:13:58.802842       1 controller.go:624] quota admission added evaluator for: prometheusrules.monitoring.coreos.com
I0205 20:13:58.809663       1 controller.go:624] quota admission added evaluator for: servicemonitors.monitoring.coreos.com
W0205 20:13:59.883136       1 dispatcher.go:225] Failed calling webhook, failing closed spidercoordinator.spiderpool.spidernet.io: failed calling webhook "spidercoordinator.spiderpool.spidernet.io": failed to call webhook: Post "https://spiderpool-controller.kube-system.svc:5722/mutate-spiderpool-spidernet-io-v2beta1-spidercoordinator?timeout=10s": dial tcp 10.233.2.218:5722: connect: connection refused
I0205 20:14:00.476406       1 controller.go:624] quota admission added evaluator for: sriovnetworknodepolicies.sriovnetwork.openshift.io
I0205 20:14:00.806909       1 controller.go:624] quota admission added evaluator for: sriovoperatorconfigs.sriovnetwork.openshift.io
W0205 20:14:01.886220       1 dispatcher.go:225] Failed calling webhook, failing closed spidercoordinator.spiderpool.spidernet.io: failed calling webhook "spidercoordinator.spiderpool.spidernet.io": failed to call webhook: Post "https://spiderpool-controller.kube-system.svc:5722/mutate-spiderpool-spidernet-io-v2beta1-spidercoordinator?timeout=10s": dial tcp 10.233.2.218:5722: connect: connection refused
W0205 20:14:03.889029       1 dispatcher.go:225] Failed calling webhook, failing closed spidercoordinator.spiderpool.spidernet.io: failed calling webhook "spidercoordinator.spiderpool.spidernet.io": failed to call webhook: Post "https://spiderpool-controller.kube-system.svc:5722/mutate-spiderpool-spidernet-io-v2beta1-spidercoordinator?timeout=10s": dial tcp 10.233.2.218:5722: connect: connection refused
E0205 20:14:04.317325       1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"client disconnected"}: client disconnected
E0205 20:14:04.317404       1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout
E0205 20:14:04.318474       1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout
E0205 20:14:04.318499       1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout
E0205 20:14:04.319653       1 timeout.go:142] post-timeout activity - time-elapsed: 2.253507ms, POST "/api/v1/namespaces/kube-system/events" result: <nil>
E0205 20:14:04.340393       1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"client disconnected"}: client disconnected
E0205 20:14:04.340468       1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout
E0205 20:14:04.341046       1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"client disconnected"}: client disconnected
E0205 20:14:04.342943       1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout
E0205 20:14:04.342956       1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout
E0205 20:14:04.342970       1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout
E0205 20:14:04.344733       1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout
E0205 20:14:04.344784       1 timeout.go:142] post-timeout activity - time-elapsed: 4.290152ms, POST "/api/v1/namespaces/kube-system/events" result: <nil>
E0205 20:14:04.344809       1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout
E0205 20:14:04.345995       1 timeout.go:142] post-timeout activity - time-elapsed: 4.952508ms, POST "/api/v1/namespaces/kube-system/events" result: <nil>
W0205 20:14:05.891472       1 dispatcher.go:225] Failed calling webhook, failing closed spidercoordinator.spiderpool.spidernet.io: failed calling webhook "spidercoordinator.spiderpool.spidernet.io": failed to call webhook: Post "https://spiderpool-controller.kube-system.svc:5722/mutate-spiderpool-spidernet-io-v2beta1-spidercoordinator?timeout=10s": dial tcp 10.233.2.218:5722: connect: connection refused
W0205 20:14:07.894301       1 dispatcher.go:225] Failed calling webhook, failing closed spidercoordinator.spiderpool.spidernet.io: failed calling webhook "spidercoordinator.spiderpool.spidernet.io": failed to call webhook: Post "https://spiderpool-controller.kube-system.svc:5722/mutate-spiderpool-spidernet-io-v2beta1-spidercoordinator?timeout=10s": dial tcp 10.233.2.218:5722: connect: connection refused
W0205 20:14:09.897330       1 dispatcher.go:225] Failed calling webhook, failing closed spidercoordinator.spiderpool.spidernet.io: failed calling webhook "spidercoordinator.spiderpool.spidernet.io": failed to call webhook: Post "https://spiderpool-controller.kube-system.svc:5722/mutate-spiderpool-spidernet-io-v2beta1-spidercoordinator?timeout=10s": dial tcp 10.233.2.218:5722: connect: connection refused
W0205 20:14:11.899609       1 dispatcher.go:225] Failed calling webhook, failing closed spidercoordinator.spiderpool.spidernet.io: failed calling webhook "spidercoordinator.spiderpool.spidernet.io": failed to call webhook: Post "https://spiderpool-controller.kube-system.svc:5722/mutate-spiderpool-spidernet-io-v2beta1-spidercoordinator?timeout=10s": dial tcp 10.233.2.218:5722: connect: connection refused
W0205 20:14:13.902083       1 dispatcher.go:225] Failed calling webhook, failing closed spidercoordinator.spiderpool.spidernet.io: failed calling webhook "spidercoordinator.spiderpool.spidernet.io": failed to call webhook: Post "https://spiderpool-controller.kube-system.svc:5722/mutate-spiderpool-spidernet-io-v2beta1-spidercoordinator?timeout=10s": dial tcp 10.233.2.218:5722: connect: connection refused
W0205 20:14:15.904780       1 dispatcher.go:225] Failed calling webhook, failing closed spidercoordinator.spiderpool.spidernet.io: failed calling webhook "spidercoordinator.spiderpool.spidernet.io": failed to call webhook: Post "https://spiderpool-controller.kube-system.svc:5722/mutate-spiderpool-spidernet-io-v2beta1-spidercoordinator?timeout=10s": dial tcp 10.233.2.218:5722: connect: connection refused
W0205 20:14:17.907126       1 dispatcher.go:225] Failed calling webhook, failing closed spidercoordinator.spiderpool.spidernet.io: failed calling webhook "spidercoordinator.spiderpool.spidernet.io": failed to call webhook: Post "https://spiderpool-controller.kube-system.svc:5722/mutate-spiderpool-spidernet-io-v2beta1-spidercoordinator?timeout=10s": dial tcp 10.233.2.218:5722: connect: connection refused
I0205 20:14:20.023865       1 controller.go:624] quota admission added evaluator for: spidermultusconfigs.spiderpool.spidernet.io
I0205 20:14:20.127610       1 controller.go:624] quota admission added evaluator for: network-attachment-definitions.k8s.cni.cncf.io
E0205 20:14:31.692769       1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout
E0205 20:14:31.692800       1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout
E0205 20:14:31.692817       1 finisher.go:175] FinishRequest: post-timeout activity - time-elapsed: 5.72µs, panicked: false, err: context canceled, panic-reason: <nil>
E0205 20:14:31.693977       1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout
E0205 20:14:31.694047       1 timeout.go:142] post-timeout activity - time-elapsed: 1.180339ms, POST "/api/v1/namespaces/kube-system/events" result: <nil>
I0205 20:14:34.415760       1 trace.go:236] Trace[1066486867]: "Create" accept:application/json,audit-id:044cf703-8bf0-4313-b861-3e7d242de030,client:172.18.0.1,api-group:spiderpool.spidernet.io,api-version:v2beta1,name:,subresource:,namespace:,protocol:HTTP/2.0,resource:spiderippools,scope:resource,url:/apis/spiderpool.spidernet.io/v2beta1/spiderippools,user-agent:kubectl/v1.26.4 (linux/amd64) kubernetes/f89670c,verb:POST (05-Feb-2024 20:14:33.739) (total time: 676ms):
ty-dc commented 7 months ago 
----- check gorouting leak in kube-system/spiderpool-controller-6dbd4cdf45-qw9kp 
maybe goroutine leak, found 351 goroutines in kube-system/spiderpool-controller-6dbd4cdf45-qw9kp , which is bigger than default 350
ty-dc commented 7 months ago

same as #3225