search

k8snetworkplumbingwg / kubemacpool

Apache License 2.0
35 stars 34 forks source link

tests sometimes fail with error "connect: connection refused" #116

Closed RamLavi closed 4 years ago

RamLavi commented 4 years ago

occurs on the following tests:

How to recreate:

the test logs + Error:

Pod Name: kubemacpool-mac-controller-manager-0 
{{ } {kubemacpool-mac-controller-manager-0 kubemacpool-mac-controller-manager- kubemacpool-system /api/v1/namespaces/kubemacpool-system/pods/kubemacpool-mac-controller-manager-0 020a0831-f923-4151-9f5c-74fcd4ac7554 1223034 0 2020-04-16 11:20:29 +0300 IDT <nil> <nil> map[app:kubemacpool control-plane:mac-controller-manager controller-revision-hash:kubemacpool-mac-controller-manager-798656b7c7 controller-tools.k8s.io:1.0 kubemacpool-leader:true statefulset.kubernetes.io/pod-name:kubemacpool-mac-controller-manager-0] map[] [{apps/v1 StatefulSet kubemacpool-mac-controller-manager 8a5b7678-2532-479e-b831-2bf0d307b105 0xc00094c05b 0xc00094c05c}] []  []} {[{default-token-ctzmh {nil nil nil nil nil &SecretVolumeSource{SecretName:default-token-ctzmh,Items:[]KeyToPath{},DefaultMode:*420,Optional:nil,} nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil}}] [] [{manager registry:5000/kubevirt/kubemacpool:latest [/manager] [--v=debug --wait-time=10]  [{webhook-server 0 8000 TCP } {healthz 0 9440 TCP }] [] [{POD_NAMESPACE  &EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,}} {POD_NAME  &EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.name,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,}} {RANGE_START  &EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:kubemacpool-mac-range-config,},Key:RANGE_START,Optional:nil,},SecretKeyRef:nil,}} {RANGE_END  &EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:kubemacpool-mac-range-config,},Key:RANGE_END,Optional:nil,},SecretKeyRef:nil,}} {HEALTH_PROBE_HOST 0.0.0.0 nil} {HEALTH_PROBE_PORT 9440 nil}] {map[cpu:{{300 -3} {<nil>} 300m DecimalSI} memory:{{629145600 0} {<nil>}  BinarySI}] map[cpu:{{100 -3} {<nil>} 100m DecimalSI} memory:{{314572800 0} {<nil>} 300Mi BinarySI}]} [{default-token-ctzmh true /var/run/secrets/kubernetes.io/serviceaccount  <nil> }] [] nil &Probe{Handler:Handler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/readyz,Port:{1 0 healthz},Host:,Scheme:HTTP,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,},InitialDelaySeconds:10,TimeoutSeconds:1,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,} nil nil /dev/termination-log File Always nil false false false}] [] Always 0xc00094c290 <nil> ClusterFirst map[] default default <nil> node01 false false false <nil> &PodSecurityContext{SELinuxOptions:nil,RunAsUser:nil,RunAsNonRoot:nil,SupplementalGroups:[],FSGroup:nil,RunAsGroup:nil,Sysctls:[]Sysctl{},WindowsOptions:nil,} [] kubemacpool-mac-controller-manager-0 kubemacpool-service &Affinity{NodeAffinity:nil,PodAffinity:nil,PodAntiAffinity:&PodAntiAffinity{RequiredDuringSchedulingIgnoredDuringExecution:[]PodAffinityTerm{},PreferredDuringSchedulingIgnoredDuringExecution:[]WeightedPodAffinityTerm{WeightedPodAffinityTerm{Weight:1,PodAffinityTerm:PodAffinityTerm{LabelSelector:&v1.LabelSelector{MatchLabels:map[string]string{},MatchExpressions:[]LabelSelectorRequirement{LabelSelectorRequirement{Key:control-plane,Operator:In,Values:[mac-controller-manager],},},},Namespaces:[],TopologyKey:kubernetes.io/hostname,},},},},} default-scheduler [{node.kubernetes.io/not-ready Exists  NoExecute 0xc00094c390} {node.kubernetes.io/unreachable Exists  NoExecute 0xc00094c3b0}] []  0xc00094c3c0 nil [] <nil> 0xc00094c3c4 <nil> map[] []} {Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2020-04-16 11:20:29 +0300 IDT  } {Ready True 0001-01-01 00:00:00 +0000 UTC 2020-04-16 11:20:49 +0300 IDT  } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2020-04-16 11:20:49 +0300 IDT  } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2020-04-16 11:20:29 +0300 IDT  }]    192.168.66.101 10.244.0.84 [{10.244.0.84}] 2020-04-16 11:20:29 +0300 IDT [] [{manager {nil &ContainerStateRunning{StartedAt:2020-04-16 11:20:30 +0300 IDT,} nil} {nil nil nil} true 0 registry:5000/kubevirt/kubemacpool:latest docker-pullable://registry:5000/kubevirt/kubemacpool@sha256:d1ef5456b62b7d94eb1f74320be352b7eb9793d544a8379d97c3eb5ff59ca2e2 docker://acec8554569ca74d35bb74410b519520e2d9156707fae01a0b4fa796904f3975 0xc00094c4d7}] Burstable []}} 
2020-04-16T08:20:30.832Z    INFO    manager Setting up client for manager
2020-04-16T08:20:30.834Z    INFO    manager Setting up leader electionManager
2020-04-16T08:20:31.324Z    INFO    controller-runtime.metrics  metrics server is starting to listen    {"addr": ":8080"}
2020-04-16T08:20:31.326Z    INFO    manager waiting for manager to become leader
I0416 08:20:31.326820       1 leaderelection.go:241] attempting to acquire leader lease  kubemacpool-system/kubemacpool-election...
I0416 08:20:48.733361       1 leaderelection.go:251] successfully acquired lease kubemacpool-system/kubemacpool-election
2020-04-16T08:20:48.733Z    DEBUG   manager.events  Normal  {"object": {"kind":"ConfigMap","namespace":"kubemacpool-system","name":"kubemacpool-election","uid":"c763aff0-5a12-4d33-90c0-d32795f5c953","apiVersion":"v1","resourceVersion":"1223030"}, "reason": "LeaderElection", "message": "kubemacpool-mac-controller-manager-0_f2bf72d5-64e0-4a0c-a33f-f03ddda22d72 became leader"}
2020-04-16T08:20:48.825Z    INFO    manager marked this manager as leader for webhook   {"podName": "kubemacpool-mac-controller-manager-0"}
2020-04-16T08:20:48.825Z    INFO    manager Setting up Manager
2020-04-16T08:20:49.290Z    INFO    controller-runtime.metrics  metrics server is starting to listen    {"addr": ":8080"}
2020-04-16T08:20:49.338Z    DEBUG   PoolManager start InitMaps to reserve existing mac addresses before allocation new ones
2020-04-16T08:20:49.417Z    DEBUG   PoolManager InitMaps for pod    {"podName": "cluster-network-addons-operator-597b8ff899-lrmzs", "podNamespace": "cluster-network-addons-operator"}
2020-04-16T08:20:49.417Z    DEBUG   PoolManager InitMaps for pod    {"podName": "local-volume-provisioner-kvwf2", "podNamespace": "default"}
2020-04-16T08:20:49.418Z    DEBUG   PoolManager InitMaps for pod    {"podName": "coredns-6955765f44-kbst8", "podNamespace": "kube-system"}
2020-04-16T08:20:49.418Z    DEBUG   PoolManager InitMaps for pod    {"podName": "coredns-6955765f44-w8hh6", "podNamespace": "kube-system"}
2020-04-16T08:20:49.418Z    DEBUG   PoolManager InitMaps for pod    {"podName": "etcd-node01", "podNamespace": "kube-system"}
2020-04-16T08:20:49.418Z    DEBUG   PoolManager InitMaps for pod    {"podName": "kube-apiserver-node01", "podNamespace": "kube-system"}
2020-04-16T08:20:49.418Z    DEBUG   PoolManager InitMaps for pod    {"podName": "kube-controller-manager-node01", "podNamespace": "kube-system"}
2020-04-16T08:20:49.418Z    DEBUG   PoolManager InitMaps for pod    {"podName": "kube-flannel-ds-amd64-rxs7b", "podNamespace": "kube-system"}
2020-04-16T08:20:49.418Z    DEBUG   PoolManager InitMaps for pod    {"podName": "kube-proxy-h7hxc", "podNamespace": "kube-system"}
2020-04-16T08:20:49.418Z    DEBUG   PoolManager InitMaps for pod    {"podName": "kube-scheduler-node01", "podNamespace": "kube-system"}
2020-04-16T08:20:49.418Z    DEBUG   PoolManager InitMaps for pod    {"podName": "kubemacpool-mac-controller-manager-0", "podNamespace": "kubemacpool-system"}
2020-04-16T08:20:49.418Z    DEBUG   PoolManager InitMaps for pod    {"podName": "virt-api-57bc76d84d-mcvtt", "podNamespace": "kubevirt"}
2020-04-16T08:20:49.418Z    DEBUG   PoolManager InitMaps for pod    {"podName": "virt-api-57bc76d84d-swkh2", "podNamespace": "kubevirt"}
2020-04-16T08:20:49.418Z    DEBUG   PoolManager InitMaps for pod    {"podName": "virt-controller-775cb4757d-7mw4f", "podNamespace": "kubevirt"}
2020-04-16T08:20:49.418Z    DEBUG   PoolManager InitMaps for pod    {"podName": "virt-controller-775cb4757d-pdp9c", "podNamespace": "kubevirt"}
2020-04-16T08:20:49.418Z    DEBUG   PoolManager InitMaps for pod    {"podName": "virt-handler-t8br8", "podNamespace": "kubevirt"}
2020-04-16T08:20:49.418Z    DEBUG   PoolManager InitMaps for pod    {"podName": "virt-operator-59d8fdd55b-7gxnp", "podNamespace": "kubevirt"}
2020-04-16T08:20:49.418Z    DEBUG   PoolManager InitMaps for pod    {"podName": "virt-operator-59d8fdd55b-t2wxv", "podNamespace": "kubevirt"}
2020-04-16T08:20:49.418Z    DEBUG   PoolManager InitMaps for pod    {"podName": "bridge-marker-d6fqf", "podNamespace": "linux-bridge"}
2020-04-16T08:20:49.418Z    DEBUG   PoolManager InitMaps for pod    {"podName": "kube-cni-linux-bridge-plugin-w2s2r", "podNamespace": "linux-bridge"}
2020-04-16T08:20:49.418Z    DEBUG   PoolManager InitMaps for pod    {"podName": "kube-multus-ds-amd64-ff7sd", "podNamespace": "multus"}
2020-04-16T08:20:49.432Z    INFO    manager Setting up controller
2020-04-16T08:20:49.432Z    INFO    controller-runtime.webhook  registering webhook {"path": "/mutate-pods"}
2020-04-16T08:20:49.432Z    INFO    controller-runtime.webhook  registering webhook {"path": "/mutate-virtualmachines"}
2020-04-16T08:20:49.432Z    INFO    PoolManager starting cleanup loop for waiting mac addresses
2020-04-16T08:20:49.432Z    INFO    controller-runtime.manager  starting metrics server {"path": "/metrics"}
2020-04-16T08:20:49.532Z    INFO    webhook/server  Starting nodenetworkconfigurationpolicy webhook server
2020-04-16T08:20:49.532Z    INFO    webhook/server/certificate/manager  Updating CA bundle for webhook
2020-04-16T08:20:49.533Z    INFO    controller-runtime.controller   Starting EventSource    {"controller": "pod-controller", "source": "kind source: /, Kind="}
2020-04-16T08:20:49.534Z    INFO    controller-runtime.controller   Starting EventSource    {"controller": "virtualmachine-controller", "source": "kind source: /, Kind="}
2020-04-16T08:20:49.633Z    INFO    controller-runtime.controller   Starting Controller {"controller": "pod-controller"}
2020-04-16T08:20:49.634Z    INFO    controller-runtime.controller   Starting Controller {"controller": "virtualmachine-controller"}
2020-04-16T08:20:49.734Z    INFO    controller-runtime.controller   Starting workers    {"controller": "pod-controller", "worker count": 1}
2020-04-16T08:20:49.734Z    DEBUG   Pod Controller  got a pod event in the controller
2020-04-16T08:20:49.735Z    DEBUG   PoolManager AllocatePodMac: Data    {"macmap": {}, "podmap": {}, "currentMac": "02:00:00:00:00:00"}
2020-04-16T08:20:49.735Z    DEBUG   controller-runtime.controller   Successfully Reconciled {"controller": "pod-controller", "request": "kube-system/kube-apiserver-node01"}
2020-04-16T08:20:49.735Z    INFO    controller-runtime.controller   Starting workers    {"controller": "virtualmachine-controller", "worker count": 1}
2020-04-16T08:20:49.735Z    DEBUG   Pod Controller  got a pod event in the controller
2020-04-16T08:20:49.735Z    DEBUG   PoolManager AllocatePodMac: Data    {"macmap": {}, "podmap": {}, "currentMac": "02:00:00:00:00:00"}
2020-04-16T08:20:49.735Z    DEBUG   controller-runtime.controller   Successfully Reconciled {"controller": "pod-controller", "request": "kube-system/etcd-node01"}
2020-04-16T08:20:49.735Z    DEBUG   Pod Controller  got a pod event in the controller
2020-04-16T08:20:49.735Z    DEBUG   controller-runtime.controller   Successfully Reconciled {"controller": "pod-controller", "request": "kubemacpool-system/kubemacpool-mac-controller-manager-1"}
2020-04-16T08:20:49.735Z    DEBUG   Pod Controller  got a pod event in the controller
2020-04-16T08:20:49.735Z    DEBUG   controller-runtime.controller   Successfully Reconciled {"controller": "pod-controller", "request": "linux-bridge/bridge-marker-d6fqf"}
2020-04-16T08:20:49.735Z    DEBUG   Pod Controller  got a pod event in the controller
2020-04-16T08:20:49.735Z    DEBUG   PoolManager AllocatePodMac: Data    {"macmap": {}, "podmap": {}, "currentMac": "02:00:00:00:00:00"}
2020-04-16T08:20:49.735Z    DEBUG   controller-runtime.controller   Successfully Reconciled {"controller": "pod-controller", "request": "kubevirt/virt-api-57bc76d84d-mcvtt"}
2020-04-16T08:20:49.735Z    DEBUG   Pod Controller  got a pod event in the controller
2020-04-16T08:20:49.735Z    DEBUG   PoolManager AllocatePodMac: Data    {"macmap": {}, "podmap": {}, "currentMac": "02:00:00:00:00:00"}
2020-04-16T08:20:49.735Z    DEBUG   controller-runtime.controller   Successfully Reconciled {"controller": "pod-controller", "request": "kube-system/kube-scheduler-node01"}
2020-04-16T08:20:49.735Z    DEBUG   Pod Controller  got a pod event in the controller
2020-04-16T08:20:49.736Z    DEBUG   PoolManager AllocatePodMac: Data    {"macmap": {}, "podmap": {}, "currentMac": "02:00:00:00:00:00"}
2020-04-16T08:20:49.736Z    DEBUG   controller-runtime.controller   Successfully Reconciled {"controller": "pod-controller", "request": "kubevirt/virt-controller-775cb4757d-7mw4f"}
2020-04-16T08:20:49.736Z    DEBUG   Pod Controller  got a pod event in the controller
2020-04-16T08:20:49.736Z    DEBUG   PoolManager AllocatePodMac: Data    {"macmap": {}, "podmap": {}, "currentMac": "02:00:00:00:00:00"}
2020-04-16T08:20:49.736Z    DEBUG   controller-runtime.controller   Successfully Reconciled {"controller": "pod-controller", "request": "kubevirt/virt-controller-775cb4757d-pdp9c"}
2020-04-16T08:20:49.736Z    DEBUG   Pod Controller  got a pod event in the controller
2020-04-16T08:20:49.736Z    DEBUG   controller-runtime.controller   Successfully Reconciled {"controller": "pod-controller", "request": "cluster-network-addons-operator/cluster-network-addons-operator-597b8ff899-lrmzs"}
2020-04-16T08:20:49.736Z    DEBUG   Pod Controller  got a pod event in the controller
2020-04-16T08:20:49.736Z    DEBUG   PoolManager AllocatePodMac: Data    {"macmap": {}, "podmap": {}, "currentMac": "02:00:00:00:00:00"}
2020-04-16T08:20:49.736Z    DEBUG   controller-runtime.controller   Successfully Reconciled {"controller": "pod-controller", "request": "kubevirt/virt-api-57bc76d84d-swkh2"}
2020-04-16T08:20:49.736Z    DEBUG   Pod Controller  got a pod event in the controller
2020-04-16T08:20:49.736Z    DEBUG   controller-runtime.controller   Successfully Reconciled {"controller": "pod-controller", "request": "multus/kube-multus-ds-amd64-ff7sd"}
2020-04-16T08:20:49.736Z    DEBUG   Pod Controller  got a pod event in the controller
2020-04-16T08:20:49.736Z    DEBUG   controller-runtime.controller   Successfully Reconciled {"controller": "pod-controller", "request": "kube-system/kube-flannel-ds-amd64-rxs7b"}
2020-04-16T08:20:49.736Z    DEBUG   Pod Controller  got a pod event in the controller
2020-04-16T08:20:49.736Z    DEBUG   controller-runtime.controller   Successfully Reconciled {"controller": "pod-controller", "request": "kube-system/coredns-6955765f44-kbst8"}
2020-04-16T08:20:49.736Z    DEBUG   Pod Controller  got a pod event in the controller
2020-04-16T08:20:49.736Z    DEBUG   PoolManager AllocatePodMac: Data    {"macmap": {}, "podmap": {}, "currentMac": "02:00:00:00:00:00"}
2020-04-16T08:20:49.736Z    DEBUG   controller-runtime.controller   Successfully Reconciled {"controller": "pod-controller", "request": "kube-system/kube-controller-manager-node01"}
2020-04-16T08:20:49.736Z    DEBUG   Pod Controller  got a pod event in the controller
2020-04-16T08:20:49.736Z    DEBUG   controller-runtime.controller   Successfully Reconciled {"controller": "pod-controller", "request": "kube-system/kube-proxy-h7hxc"}
2020-04-16T08:20:49.736Z    DEBUG   Pod Controller  got a pod event in the controller
2020-04-16T08:20:49.736Z    DEBUG   PoolManager AllocatePodMac: Data    {"macmap": {}, "podmap": {}, "currentMac": "02:00:00:00:00:00"}
2020-04-16T08:20:49.736Z    DEBUG   controller-runtime.controller   Successfully Reconciled {"controller": "pod-controller", "request": "kubevirt/virt-operator-59d8fdd55b-t2wxv"}
2020-04-16T08:20:49.736Z    DEBUG   Pod Controller  got a pod event in the controller
2020-04-16T08:20:49.737Z    DEBUG   PoolManager AllocatePodMac: Data    {"macmap": {}, "podmap": {}, "currentMac": "02:00:00:00:00:00"}
2020-04-16T08:20:49.737Z    DEBUG   controller-runtime.controller   Successfully Reconciled {"controller": "pod-controller", "request": "kubevirt/virt-operator-59d8fdd55b-7gxnp"}
2020-04-16T08:20:49.737Z    DEBUG   Pod Controller  got a pod event in the controller
2020-04-16T08:20:49.737Z    DEBUG   controller-runtime.controller   Successfully Reconciled {"controller": "pod-controller", "request": "linux-bridge/kube-cni-linux-bridge-plugin-w2s2r"}
2020-04-16T08:20:49.737Z    DEBUG   Pod Controller  got a pod event in the controller
2020-04-16T08:20:49.737Z    DEBUG   PoolManager AllocatePodMac: Data    {"macmap": {}, "podmap": {}, "currentMac": "02:00:00:00:00:00"}
2020-04-16T08:20:49.737Z    DEBUG   controller-runtime.controller   Successfully Reconciled {"controller": "pod-controller", "request": "kubevirt/virt-handler-t8br8"}
2020-04-16T08:20:49.737Z    DEBUG   Pod Controller  got a pod event in the controller
2020-04-16T08:20:49.737Z    DEBUG   controller-runtime.controller   Successfully Reconciled {"controller": "pod-controller", "request": "kube-system/coredns-6955765f44-w8hh6"}
2020-04-16T08:20:49.737Z    DEBUG   Pod Controller  got a pod event in the controller
2020-04-16T08:20:49.737Z    DEBUG   controller-runtime.controller   Successfully Reconciled {"controller": "pod-controller", "request": "kubemacpool-system/kubemacpool-mac-controller-manager-0"}
2020-04-16T08:20:49.737Z    DEBUG   Pod Controller  got a pod event in the controller
2020-04-16T08:20:49.737Z    DEBUG   controller-runtime.controller   Successfully Reconciled {"controller": "pod-controller", "request": "default/local-volume-provisioner-kvwf2"}
2020-04-16T08:20:49.742Z    INFO    webhook/server/certificate/manager  Starting cert manager
2020-04-16T08:20:49.742Z    INFO    webhook/server/certificate/manager  Wait for cert/key to be created

the server rejected our request for an unknown reason (get pods kubemacpool-mac-controller-manager-1)
• Failure [27.280 seconds]
Virtual Machines
/root/github.com/k8snetworkplumbingwg/kubemacpool/tests/virtual_machines_test.go:28
  Check the client
  /root/github.com/k8snetworkplumbingwg/kubemacpool/tests/virtual_machines_test.go:38
    When trying to create a VM after all MAC addresses in range have been occupied
    /root/github.com/k8snetworkplumbingwg/kubemacpool/tests/virtual_machines_test.go:219
      should return an error because no MAC address is available [It]
      /root/github.com/k8snetworkplumbingwg/kubemacpool/tests/virtual_machines_test.go:220

      Unexpected error:
          <*errors.StatusError | 0xc0005cb0e0>: {
              ErrStatus: {
                  TypeMeta: {Kind: "", APIVersion: ""},
                  ListMeta: {
                      SelfLink: "",
                      ResourceVersion: "",
                      Continue: "",
                      RemainingItemCount: nil,
                  },
                  Status: "Failure",
                  Message: "Internal error occurred: failed calling webhook \"mutatevirtualmachines.kubemacpool.io\": Post https://kubemacpool-service.kubemacpool-system.svc:443/mutate-virtualmachines?timeout=30s: dial tcp 10.96.42.251:443: connect: connection refused",
                  Reason: "InternalError",
                  Details: {
                      Name: "",
                      Group: "",
                      Kind: "",
                      UID: "",
                      Causes: [
                          {
                              Type: "",
                              Message: "failed calling webhook \"mutatevirtualmachines.kubemacpool.io\": Post https://kubemacpool-service.kubemacpool-system.svc:443/mutate-virtualmachines?timeout=30s: dial tcp 10.96.42.251:443: connect: connection refused",
                              Field: "",
                          },
                      ],
                      RetryAfterSeconds: 0,
                  },
                  Code: 500,
              },
          }
          Internal error occurred: failed calling webhook "mutatevirtualmachines.kubemacpool.io": Post https://kubemacpool-service.kubemacpool-system.svc:443/mutate-virtualmachines?timeout=30s: dial tcp 10.96.42.251:443: connect: connection refused
      occurred

      /root/github.com/k8snetworkplumbingwg/kubemacpool/vendor/github.com/onsi/gomega/internal/assertion/assertion.go:43

also attaching kube-apiserver pod relevant logs:

W0416 11:44:48.656771       1 dispatcher.go:180] Failed calling webhook, failing closed mutatevirtualmachines.kubemacpool.io: failed calling webhook "mutatevirtualmachines.kubemacpool.io": Post https://kubemacpool-service.kubemacpool-system.svc:443/mutate-virtualmachines?timeout=30s: dial tcp 10.96.248.132:443: connect: connection refused
W0416 11:44:48.775357       1 dispatcher.go:180] Failed calling webhook, failing closed mutatevirtualmachines.kubemacpool.io: failed calling webhook "mutatevirtualmachines.kubemacpool.io": Post https://kubemacpool-service.kubemacpool-system.svc:443/mutate-virtualmachines?timeout=30s: dial tcp 10.96.248.132:443: connect: connection refused
W0416 11:44:48.777780       1 dispatcher.go:180] Failed calling webhook, failing closed mutatevirtualmachines.kubemacpool.io: failed calling webhook "mutatevirtualmachines.kubemacpool.io": Post https://kubemacpool-service.kubemacpool-system.svc:443/mutate-virtualmachines?timeout=30s: dial tcp 10.96.248.132:443: connect: connection refused
RamLavi commented 4 years ago

2020-04-16T08:20:49.742Z INFO webhook/server/certificate/manager Wait for cert/key to be created The issue is with kube-addmisions that isn't ready with the certs - which causes the tests to sometimes fail.

RamLavi commented 4 years ago

fixed in #124. closing issue