search

chaosblade-io / chaosblade-operator

chaosblade operator for kubernetes experiments
Apache License 2.0
163 stars 101 forks source link

failed calling webhook make the k8s cluster unavailable #113

Open llovizno opened 2 years ago

llovizno commented 2 years ago

Issue Description

Type: bug report or feature request

Describe what happened (or what feature you want)

After installation(chaosblade-operator ),the cluster unavailable.all the k8s yamls cannot to be submitted.the api-server logs shows en error dozens per second:failed calling webhook "chaosblade-operator.chaosblade.svc": Post "https://chaosblade-webhook-server.chaosblade.svc:443/mutating-pods?timeout=30s": context canceled.

Why is that?why this error make the cluster unavailable and how can i solve it. thinks

Describe what you expected to happen

How to reproduce it (as minimally and precisely as possible)

Tell us your environment

kubernetes

Anything else we need to know?

xcaspar commented 2 years ago

Have not encountered chaosblade operator installation causing the cluster to be unavailable. Please provide the version of kubernetes and chaosblade.

Can the kubectl get pods command be executed normally when the cluster is unavailable?

llovizno commented 2 years ago

Have not encountered chaosblade operator installation causing the cluster to be unavailable. Please provide the version of kubernetes and chaosblade.

Can the kubectl get pods command be executed normally when the cluster is unavailable?

kubernetes:1.19.12 chaos:1.2.0 kubectl get pods is normally, but the kubernetes-api timeout when trying to create a pod through the kubernetes interface. when uninstall chaos-operator the api will be normal. I wonder what chaos-webhook-server did when the api unavailable.

heqg commented 2 years ago

I have met the same problem too。 env： k8s 1.22 chaos-operator： 1.5.0

the error info as below： Warning FailedCreate 94s (x17 over 7m2s) replicaset-controller Error creating: Internal error occurred: failed calling webhook "chaosblade-operator.chaosblade.svc": failed to call webhook: Post "https://chaosblade-webhook-server.chaosblade.svc:443/mutating-pods?timeout=10s": no endpoints available for service "chaosblade-webhook-server"

@xcaspar @tiny-x @llovizno
now，Is there any solution at present? thks

heqg commented 2 years ago

cc: @xcaspar

llovizno commented 2 years ago

I have met the same problem too。 env： k8s 1.22 chaos-operator： 1.5.0

the error info as below： Warning FailedCreate 94s (x17 over 7m2s) replicaset-controller Error creating: Internal error occurred: failed calling webhook "chaosblade-operator.chaosblade.svc": failed to call webhook: Post "https://chaosblade-webhook-server.chaosblade.svc:443/mutating-pods?timeout=10s": no endpoints available for service "chaosblade-webhook-server"

@xcaspar @tiny-x @llovizno now，Is there any solution at present? thks

my environment is in production,until now,I don't have any permission to positioning it.Can you share your environment and position together?