The deletion of federated resources will be blocked forever by not-ready clusters

[RainbowMango]

What happened: The deletion of federated resources will be blocked forever by Irrelevant/not-ready clusters.

What you expected to happen: When deleting a federated resource, should ignore the irrelevant clusters no matter it's status is ready or not.

How to reproduce it (as minimally and precisely as possible):

• S1: Prepare two clusters and cluster2 is not-ready.

  [root@ecs-d8b6 kubefed]# kubectl get kubefedclusters.core.kubefed.io -n kube-federation-system 
  NAME       AGE     READY
  cluster1   3h57m   True
  cluster2   3h57m   

• S2: Deploy a federated deployment which spec.placement.clusters only matches cluster1

  placement:
  clusters:
  - name: cluster1

  For now, everything works as expected, the deployment has been propagated only to cluster1.

  [root@ecs-d8b6 kubefed]# kubectl get deployments.apps -n test-namespace --context cluster1
  NAME              READY   UP-TO-DATE   AVAILABLE   AGE
  test-deployment   3/3     3            3           5m50s
  [root@ecs-d8b6 kubefed]# kubectl get deployments.apps -n test-namespace --context cluster2
  Unable to connect to the server: net/http: TLS handshake timeout

• S3: Delete the federated deployment

The delete operation will be hanging there forever.

[root@ecs-d8b6 kubefed]# kubectl delete federateddeployments.types.kubefed.io test-deployment -n test-namespace
federateddeployment.types.kubefed.io "test-deployment" deleted
^C

From the log, we can see the sync controller still trying to touch the not ready cluster.

E0720 07:03:01.481483       1 controller.go:479] failed to delete FederatedDeployment "test-namespace/test-deployment": the following clusters were not ready: cluster2
E0720 07:03:11.481740       1 controller.go:479] failed to delete FederatedDeployment "test-namespace/test-deployment": the following clusters were not ready: cluster2
E0720 07:03:31.482041       1 controller.go:479] failed to delete FederatedDeployment "test-namespace/test-deployment": the following clusters were not ready: cluster2

Anything else we need to know?:

Environment:

• Kubernetes version (use kubectl version)
• KubeFed version: v0.3.0
• Scope of installation (namespaced or cluster)
• Others

/kind bug

[RainbowMango]

/assign

[fejta-bot]

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta. /lifecycle stale

[jimmidyson]

/remove-lifecycle stale

[fejta-bot]

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta. /lifecycle stale

[relaxtheo]

I have same issue in deleting federatedDaemonset

[hectorj2f]

/remove-lifecycle stale

[hectorj2f]

@relaxtheo Could you share the logs ?

[relaxtheo]

root@fedhost-master:/home/ubuntu# kubectl get fds --all-namespaces NAMESPACE NAME AGE fedns-e2 fed-laundry-test-ds 3d4h fedns-e2 fedds-nano 18d fedns-e2 laundry-fed-test-ds 3d fedns-e2 laundry-fed-test1-ds 2d22h

root@fedhost-master:/home/ubuntu# kubectl delete fds fedds-nano -n fedns-e2 federateddaemonset.types.kubefed.io "fedds-nano" deleted ^[c^C

root@fedhost-master:/home/ubuntu# kubectl delete fds fedds-nano -n fedns-e2 --force --grace-period=0 warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. federateddaemonset.types.kubefed.io "fedds-nano" force deleted ^C

root@fedhost-master:/home/ubuntu# kubectl delete fds fedds-nano -n fedns-e2 --force --grace-period=0 -v 10 I0213 13:56:07.117755 226612 loader.go:375] Config loaded from file: /root/.kube/config I0213 13:56:07.118993 226612 cached_discovery.go:114] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/servergroups.json I0213 13:56:07.119540 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/apiregistration.k8s.io/v1beta1/serverresources.json I0213 13:56:07.119635 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/types.kubefed.io/v1beta1/serverresources.json I0213 13:56:07.119762 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/extensions/v1beta1/serverresources.json I0213 13:56:07.119906 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/v1/serverresources.json I0213 13:56:07.119980 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/apiregistration.k8s.io/v1/serverresources.json I0213 13:56:07.119980 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/apps/v1/serverresources.json I0213 13:56:07.120089 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/events.k8s.io/v1/serverresources.json I0213 13:56:07.120105 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/storage.k8s.io/v1/serverresources.json I0213 13:56:07.120193 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/events.k8s.io/v1beta1/serverresources.json I0213 13:56:07.120250 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/authentication.k8s.io/v1/serverresources.json I0213 13:56:07.120324 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/authentication.k8s.io/v1beta1/serverresources.json I0213 13:56:07.120393 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/authorization.k8s.io/v1/serverresources.json I0213 13:56:07.120394 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/storage.k8s.io/v1beta1/serverresources.json I0213 13:56:07.120477 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/authorization.k8s.io/v1beta1/serverresources.json I0213 13:56:07.120528 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/admissionregistration.k8s.io/v1/serverresources.json I0213 13:56:07.120548 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/autoscaling/v1/serverresources.json I0213 13:56:07.120620 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/autoscaling/v2beta1/serverresources.json I0213 13:56:07.120680 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/autoscaling/v2beta2/serverresources.json I0213 13:56:07.120744 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/batch/v1/serverresources.json I0213 13:56:07.120726 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/admissionregistration.k8s.io/v1beta1/serverresources.json I0213 13:56:07.120810 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/batch/v1beta1/serverresources.json I0213 13:56:07.120870 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/apiextensions.k8s.io/v1/serverresources.json I0213 13:56:07.120874 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/certificates.k8s.io/v1/serverresources.json I0213 13:56:07.120939 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/certificates.k8s.io/v1beta1/serverresources.json I0213 13:56:07.121034 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/networking.k8s.io/v1/serverresources.json I0213 13:56:07.121078 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/apiextensions.k8s.io/v1beta1/serverresources.json I0213 13:56:07.121180 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/networking.k8s.io/v1beta1/serverresources.json I0213 13:56:07.121185 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/scheduling.k8s.io/v1/serverresources.json I0213 13:56:07.121285 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/scheduling.k8s.io/v1beta1/serverresources.json I0213 13:56:07.121310 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/policy/v1beta1/serverresources.json I0213 13:56:07.121420 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/rbac.authorization.k8s.io/v1/serverresources.json I0213 13:56:07.121631 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/rbac.authorization.k8s.io/v1beta1/serverresources.json I0213 13:56:07.121688 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/coordination.k8s.io/v1/serverresources.json I0213 13:56:07.121751 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/core.kubefed.io/v1beta1/serverresources.json I0213 13:56:07.121838 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/coordination.k8s.io/v1beta1/serverresources.json I0213 13:56:07.121974 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/core.kubefed.io/v1alpha1/serverresources.json I0213 13:56:07.121997 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/node.k8s.io/v1beta1/serverresources.json I0213 13:56:07.122069 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/discovery.k8s.io/v1beta1/serverresources.json I0213 13:56:07.122262 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/multiclusterdns.kubefed.io/v1alpha1/serverresources.json I0213 13:56:07.122369 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/crd.projectcalico.org/v1/serverresources.json I0213 13:56:07.122466 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/scheduling.kubefed.io/v1alpha1/serverresources.json I0213 13:56:07.122514 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/reliablesyncs.kubeedge.io/v1alpha1/serverresources.json I0213 13:56:07.122548 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/devices.kubeedge.io/v1alpha2/serverresources.json I0213 13:56:07.123069 226612 cached_discovery.go:114] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/servergroups.json I0213 13:56:07.123317 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/apps/v1/serverresources.json I0213 13:56:07.123381 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/types.kubefed.io/v1beta1/serverresources.json I0213 13:56:07.123523 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/events.k8s.io/v1/serverresources.json I0213 13:56:07.123618 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/v1/serverresources.json I0213 13:56:07.123662 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/events.k8s.io/v1beta1/serverresources.json I0213 13:56:07.123713 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/apiregistration.k8s.io/v1/serverresources.json I0213 13:56:07.123788 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/authentication.k8s.io/v1/serverresources.json I0213 13:56:07.123849 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/apiregistration.k8s.io/v1beta1/serverresources.json I0213 13:56:07.123903 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/authentication.k8s.io/v1beta1/serverresources.json I0213 13:56:07.123943 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/extensions/v1beta1/serverresources.json I0213 13:56:07.124049 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/admissionregistration.k8s.io/v1/serverresources.json I0213 13:56:07.124121 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/admissionregistration.k8s.io/v1beta1/serverresources.json I0213 13:56:07.124135 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/authorization.k8s.io/v1/serverresources.json I0213 13:56:07.124220 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/authorization.k8s.io/v1beta1/serverresources.json I0213 13:56:07.124302 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/apiextensions.k8s.io/v1/serverresources.json I0213 13:56:07.124338 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/autoscaling/v1/serverresources.json I0213 13:56:07.124406 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/autoscaling/v2beta1/serverresources.json I0213 13:56:07.124485 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/apiextensions.k8s.io/v1beta1/serverresources.json I0213 13:56:07.124508 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/autoscaling/v2beta2/serverresources.json I0213 13:56:07.124611 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/batch/v1/serverresources.json I0213 13:56:07.124677 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/scheduling.k8s.io/v1/serverresources.json I0213 13:56:07.124706 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/batch/v1beta1/serverresources.json I0213 13:56:07.124797 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/certificates.k8s.io/v1/serverresources.json I0213 13:56:07.124890 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/certificates.k8s.io/v1beta1/serverresources.json I0213 13:56:07.124890 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/scheduling.k8s.io/v1beta1/serverresources.json I0213 13:56:07.124997 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/coordination.k8s.io/v1/serverresources.json I0213 13:56:07.125033 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/networking.k8s.io/v1/serverresources.json I0213 13:56:07.125144 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/networking.k8s.io/v1beta1/serverresources.json I0213 13:56:07.125193 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/coordination.k8s.io/v1beta1/serverresources.json I0213 13:56:07.125271 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/policy/v1beta1/serverresources.json I0213 13:56:07.125329 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/node.k8s.io/v1beta1/serverresources.json I0213 13:56:07.125371 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/rbac.authorization.k8s.io/v1/serverresources.json I0213 13:56:07.125468 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/discovery.k8s.io/v1beta1/serverresources.json I0213 13:56:07.125499 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/rbac.authorization.k8s.io/v1beta1/serverresources.json I0213 13:56:07.125629 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/storage.k8s.io/v1/serverresources.json I0213 13:56:07.125713 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/storage.k8s.io/v1beta1/serverresources.json I0213 13:56:07.125793 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/crd.projectcalico.org/v1/serverresources.json I0213 13:56:07.125909 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/multiclusterdns.kubefed.io/v1alpha1/serverresources.json I0213 13:56:07.126020 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/core.kubefed.io/v1beta1/serverresources.json I0213 13:56:07.126112 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/core.kubefed.io/v1alpha1/serverresources.json I0213 13:56:07.126168 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/scheduling.kubefed.io/v1alpha1/serverresources.json I0213 13:56:07.126265 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/devices.kubeedge.io/v1alpha2/serverresources.json I0213 13:56:07.125916 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/reliablesyncs.kubeedge.io/v1alpha1/serverresources.json I0213 13:56:07.128294 226612 cached_discovery.go:114] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/servergroups.json I0213 13:56:07.129108 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/types.kubefed.io/v1beta1/serverresources.json I0213 13:56:07.129750 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/v1/serverresources.json I0213 13:56:07.130196 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/apiregistration.k8s.io/v1/serverresources.json I0213 13:56:07.130401 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/apiregistration.k8s.io/v1beta1/serverresources.json I0213 13:56:07.130584 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/extensions/v1beta1/serverresources.json I0213 13:56:07.130690 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/apps/v1/serverresources.json I0213 13:56:07.131060 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/events.k8s.io/v1/serverresources.json I0213 13:56:07.131104 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/events.k8s.io/v1beta1/serverresources.json I0213 13:56:07.131144 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/authentication.k8s.io/v1/serverresources.json I0213 13:56:07.131198 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/authentication.k8s.io/v1beta1/serverresources.json I0213 13:56:07.131252 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/authorization.k8s.io/v1/serverresources.json I0213 13:56:07.131301 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/authorization.k8s.io/v1beta1/serverresources.json I0213 13:56:07.131346 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/autoscaling/v1/serverresources.json I0213 13:56:07.131402 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/autoscaling/v2beta1/serverresources.json I0213 13:56:07.131450 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/autoscaling/v2beta2/serverresources.json I0213 13:56:07.131487 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/batch/v1/serverresources.json I0213 13:56:07.131557 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/batch/v1beta1/serverresources.json I0213 13:56:07.131609 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/certificates.k8s.io/v1/serverresources.json I0213 13:56:07.131663 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/certificates.k8s.io/v1beta1/serverresources.json I0213 13:56:07.131717 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/networking.k8s.io/v1/serverresources.json I0213 13:56:07.131769 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/networking.k8s.io/v1beta1/serverresources.json I0213 13:56:07.131830 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/policy/v1beta1/serverresources.json I0213 13:56:07.131887 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/rbac.authorization.k8s.io/v1/serverresources.json I0213 13:56:07.132223 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/rbac.authorization.k8s.io/v1beta1/serverresources.json I0213 13:56:07.132303 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/storage.k8s.io/v1/serverresources.json I0213 13:56:07.132371 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/storage.k8s.io/v1beta1/serverresources.json I0213 13:56:07.132424 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/admissionregistration.k8s.io/v1/serverresources.json I0213 13:56:07.132468 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/admissionregistration.k8s.io/v1beta1/serverresources.json I0213 13:56:07.132523 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/apiextensions.k8s.io/v1/serverresources.json I0213 13:56:07.132584 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/apiextensions.k8s.io/v1beta1/serverresources.json I0213 13:56:07.132623 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/scheduling.k8s.io/v1/serverresources.json I0213 13:56:07.132662 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/scheduling.k8s.io/v1beta1/serverresources.json I0213 13:56:07.132699 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/coordination.k8s.io/v1/serverresources.json I0213 13:56:07.132775 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/coordination.k8s.io/v1beta1/serverresources.json I0213 13:56:07.132814 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/node.k8s.io/v1beta1/serverresources.json I0213 13:56:07.132855 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/discovery.k8s.io/v1beta1/serverresources.json I0213 13:56:07.132985 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/crd.projectcalico.org/v1/serverresources.json I0213 13:56:07.133066 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/core.kubefed.io/v1beta1/serverresources.json I0213 13:56:07.133131 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/core.kubefed.io/v1alpha1/serverresources.json I0213 13:56:07.133196 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/multiclusterdns.kubefed.io/v1alpha1/serverresources.json I0213 13:56:07.133257 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/reliablesyncs.kubeedge.io/v1alpha1/serverresources.json I0213 13:56:07.133304 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/scheduling.kubefed.io/v1alpha1/serverresources.json I0213 13:56:07.133346 226612 cached_discovery.go:71] returning cached discovery info from /root/.kube/cache/discovery/10.0.0.80_6443/devices.kubeedge.io/v1alpha2/serverresources.json warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. I0213 13:56:07.136268 226612 request.go:1097] Request Body: {"gracePeriodSeconds":0,"propagationPolicy":"Background"} I0213 13:56:07.136349 226612 round_trippers.go:423] curl -k -v -XDELETE -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: kubectl/v1.19.5 (linux/amd64) kubernetes/e338cf2" 'https://10.0.0.80:6443/apis/types.kubefed.io/v1beta1/namespaces/fedns-e2/federateddaemonsets/fedds-nano' I0213 13:56:07.147660 226612 round_trippers.go:443] DELETE https://10.0.0.80:6443/apis/types.kubefed.io/v1beta1/namespaces/fedns-e2/federateddaemonsets/fedds-nano 200 OK in 11 milliseconds I0213 13:56:07.147695 226612 round_trippers.go:449] Response Headers: I0213 13:56:07.147707 226612 round_trippers.go:452] Content-Type: application/json I0213 13:56:07.147719 226612 round_trippers.go:452] Date: Sat, 13 Feb 2021 13:56:07 GMT I0213 13:56:07.147748 226612 round_trippers.go:452] Cache-Control: no-cache, private I0213 13:56:07.147946 226612 request.go:1097] Response Body: {"apiVersion":"types.kubefed.io/v1beta1","kind":"FederatedDaemonSet","metadata":{"annotations":{"kubectl.kubernetes.io/last-applied-configuration":"{\"apiVersion\":\"types.kubefed.io/v1beta1\",\"kind\":\"FederatedDaemonSet\",\"metadata\":{\"annotations\":{},\"name\":\"fedds-nano\",\"namespace\":\"fedns-e2\"},\"spec\":{\"placement\":{\"clusters\":[{\"name\":\"tc1-context\"},{\"name\":\"tc2-context\"}]},\"template\":{\"metadata\":{\"labels\":{\"k8s-app\":\"laundry\"},\"name\":\"fed-laundry-edge-ds\",\"namespace\":\"fedns-e2\"},\"spec\":{\"revisionHistoryLimit\":5,\"selector\":{\"matchLabels\":{\"k8s-app\":\"laundry\"}},\"template\":{\"metadata\":{\"creationTimestamp\":null,\"labels\":{\"k8s-app\":\"laundry\"}},\"spec\":{\"containers\":[{\"command\":[\"/brightai/laundry/app/bright\"],\"image\":\"brightai/laundry:0.0-TST-131\",\"imagePullPolicy\":\"IfNotPresent\",\"livenessProbe\":{\"httpGet\":{\"httpHeaders\":[{\"name\":\"Custom-Header\",\"value\":\"ok\"}],\"path\":\"/ctl?cmd=k8s_check\u0026param=live\",\"port\":8080},\"initialDelaySeconds\":3,\"periodSeconds\":3},\"name\":\"laundry\",\"readinessProbe\":{\"httpGet\":{\"httpHeaders\":[{\"name\":\"Custom-Header\",\"value\":\"ok\"}],\"path\":\"/ctl?cmd=k8s_check\u0026param=ready\",\"port\":8080},\"initialDelaySeconds\":3,\"periodSeconds\":3},\"resources\":{},\"securityContext\":{\"privileged\":true},\"startupProbe\":{\"httpGet\":{\"httpHeaders\":[{\"name\":\"Custom-Header\",\"value\":\"ok\"}],\"path\":\"/ctl?cmd=k8s_check\u0026param=init\",\"port\":8080},\"initialDelaySeconds\":3,\"periodSeconds\":3},\"terminationMessagePath\":\"/dev/termination-log\",\"terminationMessagePolicy\":\"File\",\"volumeMounts\":[{\"mountPath\":\"/logs\",\"name\":\"laundry-log\"},{\"mountPath\":\"/dev/video0\",\"name\":\"usb-camera\"},{\"mountPath\":\"/brightai/laundry/record\",\"name\":\"record-dir\"},{\"mountPath\":\"/brightai/laundry/app/model_b1_gpu0_fp16.engine\",\"name\":\"model-engine\"},{\"mountPath\":\"/brightai/laundry/app/serialized_engine.trt\",\"name\":\"serialized-engine-trt\"}]}],\"dnsPolicy\":\"ClusterFirst\",\"hostNetwork\":true,\"nodeSelector\":{\"node-role.kubernetes.io/edge\":\"\"},\"priorityClassName\":\"system-node-critical\",\"restartPolicy\":\"Always\",\"schedulerName\":\"default-scheduler\",\"securityContext\":{},\"terminationGracePeriodSeconds\":30,\"volumes\":[{\"hostPath\":{\"path\":\"/logs\"},\"name\":\"laundry-log\"},{\"hostPath\":{\"path\":\"/dev/video0\"},\"name\":\"usb-camera\"},{\"hostPath\":{\"path\":\"/laundry-records\",\"type\":\"DirectoryOrCreate\"},\"name\":\"record-dir\"},{\"hostPath\":{\"path\":\"/laundry-engine/model_131.engine\",\"type\":\"FileOrCreate\"},\"name\":\"model-engine\"},{\"hostPath\":{\"path\":\"/laundry-engine/serialized_131.trt\",\"type\":\"FileOrCreate\"},\"name\":\"serialized-engine-trt\"}]}},\"updateStrategy\":{\"rollingUpdate\":{\"maxUnavailable\":1},\"type\":\"RollingUpdate\"}}}}}\n"},"creationTimestamp":"2021-01-26T11:56:02Z","deletionGracePeriodSeconds":0,"deletionTimestamp":"2021-02-10T09:13:19Z","finalizers":["kubefed.io/sync-controller"],"generation":12,"managedFields":[{"apiVersion":"types.kubefed.io/v1beta1","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:annotations":{".":{},"f:kubectl.kubernetes.io/last-applied-configuration":{}}},"f:spec":{".":{},"f:placement":{".":{},"f:clusters":{}},"f:template":{".":{},"f:metadata":{},"f:spec":{}}}},"manager":"kubectl-client-side-apply","operation":"Update","time":"2021-01-26T11:56:02Z"},{"apiVersion":"types.kubefed.io/v1beta1","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:finalizers":{".":{},"v:\"kubefed.io/sync-controller\"":{}}},"f:status":{".":{},"f:clusters":{},"f:conditions":{},"f:observedGeneration":{}}},"manager":"controller-manager","operation":"Update","time":"2021-01-26T11:56:03Z"}],"name":"fedds-nano","namespace":"fedns-e2","resourceVersion":"3708987","selfLink":"/apis/types.kubefed.io/v1beta1/namespaces/fedns-e2/federateddaemonsets/fedds-nano","uid":"1f136b51-3808-4c9e-b5ff-17c20ffd854a"},"spec":{"placement":{"clusters":[{"name":"tc1-context"},{"name":"tc2-context"}]},"template":{"metadata":{"labels":{"k8s-app":"laundry"},"name":"fed-laundry-edge-ds","namespace":"fedns-e2"},"spec":{"revisionHistoryLimit":5,"selector":{"matchLabels":{"k8s-app":"laundry"}},"template":{"metadata":{"creationTimestamp":null,"labels":{"k8s-app":"laundry"}},"spec":{"containers":[{"command":["/brightai/laundry/app/bright"],"image":"brightai/laundry:0.0-TST-131","imagePullPolicy":"IfNotPresent","livenessProbe":{"httpGet":{"httpHeaders":[{"name":"Custom-Header","value":"ok"}],"path":"/ctl?cmd=k8s_check\u0026param=live","port":8080},"initialDelaySeconds":3,"periodSeconds":3},"name":"laundry","readinessProbe":{"httpGet":{"httpHeaders":[{"name":"Custom-Header","value":"ok"}],"path":"/ctl?cmd=k8s_check\u0026param=ready","port":8080},"initialDelaySeconds":3,"periodSeconds":3},"resources":{},"securityContext":{"privileged":true},"startupProbe":{"httpGet":{"httpHeaders":[{"name":"Custom-Header","value":"ok"}],"path":"/ctl?cmd=k8s_check\u0026param=init","port":8080},"initialDelaySeconds":3,"periodSeconds":3},"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","volumeMounts":[{"mountPath":"/logs","name":"laundry-log"},{"mountPath":"/dev/video0","name":"usb-camera"},{"mountPath":"/brightai/laundry/record","name":"record-dir"},{"mountPath":"/brightai/laundry/app/model_b1_gpu0_fp16.engine","name":"model-engine"},{"mountPath":"/brightai/laundry/app/serialized_engine.trt","name":"serialized-engine-trt"}]}],"dnsPolicy":"ClusterFirst","hostNetwork":true,"nodeSelector":{"node-role.kubernetes.io/edge":""},"priorityClassName":"system-node-critical","restartPolicy":"Always","schedulerName":"default-scheduler","securityContext":{},"terminationGracePeriodSeconds":30,"volumes":[{"hostPath":{"path":"/logs"},"name":"laundry-log"},{"hostPath":{"path":"/dev/video0"},"name":"usb-camera"},{"hostPath":{"path":"/laundry-records","type":"DirectoryOrCreate"},"name":"record-dir"},{"hostPath":{"path":"/laundry-engine/model_131.engine","type":"FileOrCreate"},"name":"model-engine"},{"hostPath":{"path":"/laundry-engine/serialized_131.trt","type":"FileOrCreate"},"name":"serialized-engine-trt"}]}},"updateStrategy":{"rollingUpdate":{"maxUnavailable":1},"type":"RollingUpdate"}}}},"status":{"clusters":[{"name":"tc1-context"},{"name":"tc2-context"}],"conditions":[{"lastTransitionTime":"2021-02-06T23:38:10Z","lastUpdateTime":"2021-02-10T09:01:22Z","status":"True","type":"Propagation"}],"observedGeneration":11}} federateddaemonset.types.kubefed.io "fedds-nano" force deleted I0213 13:56:07.148477 226612 round_trippers.go:423] curl -k -v -XGET -H "Accept: application/json" -H "User-Agent: kubectl/v1.19.5 (linux/amd64) kubernetes/e338cf2" 'https://10.0.0.80:6443/apis/types.kubefed.io/v1beta1/namespaces/fedns-e2/federateddaemonsets?fieldSelector=metadata.name%3Dfedds-nano' I0213 13:56:07.152020 226612 round_trippers.go:443] GET https://10.0.0.80:6443/apis/types.kubefed.io/v1beta1/namespaces/fedns-e2/federateddaemonsets?fieldSelector=metadata.name%3Dfedds-nano 200 OK in 3 milliseconds I0213 13:56:07.152043 226612 round_trippers.go:449] Response Headers: I0213 13:56:07.152056 226612 round_trippers.go:452] Content-Type: application/json I0213 13:56:07.152067 226612 round_trippers.go:452] Date: Sat, 13 Feb 2021 13:56:07 GMT I0213 13:56:07.152079 226612 round_trippers.go:452] Cache-Control: no-cache, private I0213 13:56:07.152191 226612 request.go:1097] Response Body: {"apiVersion":"types.kubefed.io/v1beta1","items":[{"apiVersion":"types.kubefed.io/v1beta1","kind":"FederatedDaemonSet","metadata":{"annotations":{"kubectl.kubernetes.io/last-applied-configuration":"{\"apiVersion\":\"types.kubefed.io/v1beta1\",\"kind\":\"FederatedDaemonSet\",\"metadata\":{\"annotations\":{},\"name\":\"fedds-nano\",\"namespace\":\"fedns-e2\"},\"spec\":{\"placement\":{\"clusters\":[{\"name\":\"tc1-context\"},{\"name\":\"tc2-context\"}]},\"template\":{\"metadata\":{\"labels\":{\"k8s-app\":\"laundry\"},\"name\":\"fed-laundry-edge-ds\",\"namespace\":\"fedns-e2\"},\"spec\":{\"revisionHistoryLimit\":5,\"selector\":{\"matchLabels\":{\"k8s-app\":\"laundry\"}},\"template\":{\"metadata\":{\"creationTimestamp\":null,\"labels\":{\"k8s-app\":\"laundry\"}},\"spec\":{\"containers\":[{\"command\":[\"/brightai/laundry/app/bright\"],\"image\":\"brightai/laundry:0.0-TST-131\",\"imagePullPolicy\":\"IfNotPresent\",\"livenessProbe\":{\"httpGet\":{\"httpHeaders\":[{\"name\":\"Custom-Header\",\"value\":\"ok\"}],\"path\":\"/ctl?cmd=k8s_check\u0026param=live\",\"port\":8080},\"initialDelaySeconds\":3,\"periodSeconds\":3},\"name\":\"laundry\",\"readinessProbe\":{\"httpGet\":{\"httpHeaders\":[{\"name\":\"Custom-Header\",\"value\":\"ok\"}],\"path\":\"/ctl?cmd=k8s_check\u0026param=ready\",\"port\":8080},\"initialDelaySeconds\":3,\"periodSeconds\":3},\"resources\":{},\"securityContext\":{\"privileged\":true},\"startupProbe\":{\"httpGet\":{\"httpHeaders\":[{\"name\":\"Custom-Header\",\"value\":\"ok\"}],\"path\":\"/ctl?cmd=k8s_check\u0026param=init\",\"port\":8080},\"initialDelaySeconds\":3,\"periodSeconds\":3},\"terminationMessagePath\":\"/dev/termination-log\",\"terminationMessagePolicy\":\"File\",\"volumeMounts\":[{\"mountPath\":\"/logs\",\"name\":\"laundry-log\"},{\"mountPath\":\"/dev/video0\",\"name\":\"usb-camera\"},{\"mountPath\":\"/brightai/laundry/record\",\"name\":\"record-dir\"},{\"mountPath\":\"/brightai/laundry/app/model_b1_gpu0_fp16.engine\",\"name\":\"model-engine\"},{\"mountPath\":\"/brightai/laundry/app/serialized_engine.trt\",\"name\":\"serialized-engine-trt\"}]}],\"dnsPolicy\":\"ClusterFirst\",\"hostNetwork\":true,\"nodeSelector\":{\"node-role.kubernetes.io/edge\":\"\"},\"priorityClassName\":\"system-node-critical\",\"restartPolicy\":\"Always\",\"schedulerName\":\"default-scheduler\",\"securityContext\":{},\"terminationGracePeriodSeconds\":30,\"volumes\":[{\"hostPath\":{\"path\":\"/logs\"},\"name\":\"laundry-log\"},{\"hostPath\":{\"path\":\"/dev/video0\"},\"name\":\"usb-camera\"},{\"hostPath\":{\"path\":\"/laundry-records\",\"type\":\"DirectoryOrCreate\"},\"name\":\"record-dir\"},{\"hostPath\":{\"path\":\"/laundry-engine/model_131.engine\",\"type\":\"FileOrCreate\"},\"name\":\"model-engine\"},{\"hostPath\":{\"path\":\"/laundry-engine/serialized_131.trt\",\"type\":\"FileOrCreate\"},\"name\":\"serialized-engine-trt\"}]}},\"updateStrategy\":{\"rollingUpdate\":{\"maxUnavailable\":1},\"type\":\"RollingUpdate\"}}}}}\n"},"creationTimestamp":"2021-01-26T11:56:02Z","deletionGracePeriodSeconds":0,"deletionTimestamp":"2021-02-10T09:13:19Z","finalizers":["kubefed.io/sync-controller"],"generation":12,"managedFields":[{"apiVersion":"types.kubefed.io/v1beta1","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:annotations":{".":{},"f:kubectl.kubernetes.io/last-applied-configuration":{}}},"f:spec":{".":{},"f:placement":{".":{},"f:clusters":{}},"f:template":{".":{},"f:metadata":{},"f:spec":{}}}},"manager":"kubectl-client-side-apply","operation":"Update","time":"2021-01-26T11:56:02Z"},{"apiVersion":"types.kubefed.io/v1beta1","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:finalizers":{".":{},"v:\"kubefed.io/sync-controller\"":{}}},"f:status":{".":{},"f:clusters":{},"f:conditions":{},"f:observedGeneration":{}}},"manager":"controller-manager","operation":"Update","time":"2021-01-26T11:56:03Z"}],"name":"fedds-nano","namespace":"fedns-e2","resourceVersion":"3708987","selfLink":"/apis/types.kubefed.io/v1beta1/namespaces/fedns-e2/federateddaemonsets/fedds-nano","uid":"1f136b51-3808-4c9e-b5ff-17c20ffd854a"},"spec":{"placement":{"clusters":[{"name":"tc1-context"},{"name":"tc2-context"}]},"template":{"metadata":{"labels":{"k8s-app":"laundry"},"name":"fed-laundry-edge-ds","namespace":"fedns-e2"},"spec":{"revisionHistoryLimit":5,"selector":{"matchLabels":{"k8s-app":"laundry"}},"template":{"metadata":{"creationTimestamp":null,"labels":{"k8s-app":"laundry"}},"spec":{"containers":[{"command":["/brightai/laundry/app/bright"],"image":"brightai/laundry:0.0-TST-131","imagePullPolicy":"IfNotPresent","livenessProbe":{"httpGet":{"httpHeaders":[{"name":"Custom-Header","value":"ok"}],"path":"/ctl?cmd=k8s_check\u0026param=live","port":8080},"initialDelaySeconds":3,"periodSeconds":3},"name":"laundry","readinessProbe":{"httpGet":{"httpHeaders":[{"name":"Custom-Header","value":"ok"}],"path":"/ctl?cmd=k8s_check\u0026param=ready","port":8080},"initialDelaySeconds":3,"periodSeconds":3},"resources":{},"securityContext":{"privileged":true},"startupProbe":{"httpGet":{"httpHeaders":[{"name":"Custom-Header","value":"ok"}],"path":"/ctl?cmd=k8s_check\u0026param=init","port":8080},"initialDelaySeconds":3,"periodSeconds":3},"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","volumeMounts":[{"mountPath":"/logs","name":"laundry-log"},{"mountPath":"/dev/video0","name":"usb-camera"},{"mountPath":"/brightai/laundry/record","name":"record-dir"},{"mountPath":"/brightai/laundry/app/model_b1_gpu0_fp16.engine","name":"model-engine"},{"mountPath":"/brightai/laundry/app/serialized_engine.trt","name":"serialized-engine-trt"}]}],"dnsPolicy":"ClusterFirst","hostNetwork":true,"nodeSelector":{"node-role.kubernetes.io/edge":""},"priorityClassName":"system-node-critical","restartPolicy":"Always","schedulerName":"default-scheduler","securityContext":{},"terminationGracePeriodSeconds":30,"volumes":[{"hostPath":{"path":"/logs"},"name":"laundry-log"},{"hostPath":{"path":"/dev/video0"},"name":"usb-camera"},{"hostPath":{"path":"/laundry-records","type":"DirectoryOrCreate"},"name":"record-dir"},{"hostPath":{"path":"/laundry-engine/model_131.engine","type":"FileOrCreate"},"name":"model-engine"},{"hostPath":{"path":"/laundry-engine/serialized_131.trt","type":"FileOrCreate"},"name":"serialized-engine-trt"}]}},"updateStrategy":{"rollingUpdate":{"maxUnavailable":1},"type":"RollingUpdate"}}}},"status":{"clusters":[{"name":"tc1-context"},{"name":"tc2-context"}],"conditions":[{"lastTransitionTime":"2021-02-06T23:38:10Z","lastUpdateTime":"2021-02-10T09:01:22Z","status":"True","type":"Propagation"}],"observedGeneration":11}}],"kind":"FederatedDaemonSetList","metadata":{"continue":"","resourceVersion":"4481849","selfLink":"/apis/types.kubefed.io/v1beta1/namespaces/fedns-e2/federateddaemonsets"}} I0213 13:56:07.152968 226612 round_trippers.go:423] curl -k -v -XGET -H "Accept: application/json" -H "User-Agent: kubectl/v1.19.5 (linux/amd64) kubernetes/e338cf2" 'https://10.0.0.80:6443/apis/types.kubefed.io/v1beta1/namespaces/fedns-e2/federateddaemonsets?fieldSelector=metadata.name%3Dfedds-nano&resourceVersion=4481849&watch=true' I0213 13:56:07.153550 226612 round_trippers.go:443] GET https://10.0.0.80:6443/apis/types.kubefed.io/v1beta1/namespaces/fedns-e2/federateddaemonsets?fieldSelector=metadata.name%3Dfedds-nano&resourceVersion=4481849&watch=true 200 OK in 0 milliseconds I0213 13:56:07.153572 226612 round_trippers.go:449] Response Headers: I0213 13:56:07.153583 226612 round_trippers.go:452] Cache-Control: no-cache, private I0213 13:56:07.153594 226612 round_trippers.go:452] Content-Type: application/json I0213 13:56:07.153605 226612 round_trippers.go:452] Date: Sat, 13 Feb 2021 13:56:07 GMT ^C

[fejta-bot]

Stale issues rot after 30d of inactivity. Mark the issue as fresh with /remove-lifecycle rotten. Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-contributor-experience at kubernetes/community. /lifecycle rotten

[hectorj2f]

/remove-lifecycle rotten

[fejta-bot]

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-contributor-experience at kubernetes/community. /lifecycle stale

[fejta-bot]

Stale issues rot after 30d of inactivity. Mark the issue as fresh with /remove-lifecycle rotten. Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-contributor-experience at kubernetes/community. /lifecycle rotten

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Reopen this issue or PR with /reopen
• Mark this issue or PR as fresh with /remove-lifecycle rotten
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close

[k8s-ci-robot]

@k8s-triage-robot: Closing this issue.

In response to [this](https://github.com/kubernetes-sigs/kubefed/issues/1254#issuecomment-898347958): >The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs. > >This bot triages issues and PRs according to the following rules: >- After 90d of inactivity, `lifecycle/stale` is applied >- After 30d of inactivity since `lifecycle/stale` was applied, `lifecycle/rotten` is applied >- After 30d of inactivity since `lifecycle/rotten` was applied, the issue is closed > >You can: >- Reopen this issue or PR with `/reopen` >- Mark this issue or PR as fresh with `/remove-lifecycle rotten` >- Offer to help out with [Issue Triage][1] > >Please send feedback to sig-contributor-experience at [kubernetes/community](https://github.com/kubernetes/community). > >/close > >[1]: https://www.kubernetes.dev/docs/guide/issue-triage/ Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.

[jonathanbeber]

/reopen

This issue is still valid in v0.9.0.

Example log where `my-cluster` was not related to the specified role. ``` E0112 17:56:43.508977 1 controller.go:272] failed to remove the label "kubefed.io/managed" from Role "mynamespace/my-role" in member clusters: the following clusters were not ready: my-cluster ```

[k8s-ci-robot]

@jonathanbeber: You can't reopen an issue/PR unless you authored it or you are a collaborator.

In response to [this](https://github.com/kubernetes-sigs/kubefed/issues/1254#issuecomment-1012176326): >/reopen > >This issue is still valid in `v0.9.0`. > >

> >Example log where `my-cluster` was not related to the specified role. > >``` >E0112 17:56:43.508977 1 controller.go:272] failed to remove the label "kubefed.io/managed" from Role "mynamespace/my-role" in member clusters: the following clusters were not ready: my-cluster >``` > >

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[jonathanbeber]

@RainbowMango I guess you are not working on this one, right? Do you mind if I give it a shot? I'd be interested in contributing.

[jimmidyson]

/reopen

[k8s-ci-robot]

@jimmidyson: Reopened this issue.

In response to [this](https://github.com/kubernetes-sigs/kubefed/issues/1254#issuecomment-1012259656): >/reopen Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.

[jimmidyson]

/remove-lifecycle rotten

[jonathanbeber]

/assign

[RainbowMango]

Thanks @jonathanbeber for picking this up. /unassign

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle stale
• Mark this issue or PR as rotten with /lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[jonathanbeber]

/remove-lifecycle stale

1499 is ready for review.